https://pantsbuild.org/ logo
w

wide-midnight-78598

03/31/2022, 1:04 AM
Is this the best way I should be doing this?
Copy code
ansible_pex = await Get(
        Pex,
        PexRequest,
        ansible.to_pex_request(),
    )
This takes about 200 seconds on my macbook, while pip installing ansible is about 45 seconds. I know this is doing more than just installing, but I feel like maybe I'm missing an optimization
is
ansible
something that derives from
PythonToolRequirementsBase
?
w

wide-midnight-78598

03/31/2022, 1:09 AM
Derives from
PythonToolBase
- I think I basically copied what I did for PyOxidizer
f

fast-nail-55400

03/31/2022, 1:09 AM
ah TIL
PythonToolBase
derives from
PythonToolRequirementsBase
😆 1
w

wide-midnight-78598

03/31/2022, 1:11 AM
I'll try VenvPex - and see if that speeds anything up
h

happy-kitchen-89482

03/31/2022, 1:31 AM
FWIW pip installing ansible on my macbook takes 140-180 seconds (without Pants involved)
Almost all that time is on building wheels from the ansible and ansible-core sdists
w

wide-midnight-78598

03/31/2022, 1:37 AM
Wow, that feels slow. But, how bad is it through Pants, is the question. 🙂
f

fast-nail-55400

03/31/2022, 1:56 AM
maybe prebuild the ansible sdists into wheels and store them somewhere? also, why wouldn’t ansible provide wheels?
(I guess they can’t provide manylinux wheels for some reason?)
p

proud-dentist-22844

03/31/2022, 4:05 AM
Do you really need
ansible
? I recommend relying on
ansible-core
for almost everything.
ansible
includes around 100 collections which takes awhile to download and install. It is much better to install
ansible-core
and just the collections you need.
❤️ 2
ansible
can't provide wheels because it depends on symlinks and the zip format does not support that.
w

wide-midnight-78598

03/31/2022, 12:09 PM
@proud-dentist-22844
ansible
is core + community, correct? Does
core
come with playbooks? The goal here is to streamline end user experience, without re-building too much of what already exists
Rather,
ansible
is core + some Collections from community?
If so, using straight core seems reasonable - since there will need to be some ansible galaxy functionality anyways at some point, for this plugin to be usable
Upon further research @proud-dentist-22844 - you're right, I should be using
ansible-core
, while prototyping, I grabbed whatever I use in some of my client projects, and in those I use a bunch of community packages as well, so it makes slightly more sense. However, for a plugin like this,
ansible-core
is definitely the way to go. Thanks!
p

proud-dentist-22844

03/31/2022, 3:43 PM
Rather,
ansible
is core + some Collections from community?
where some = about 100 😉 (quite a bit more than a bunch) There are so many collections, I prefer to get just the collections I depend on on a case by case basis.
w

wide-midnight-78598

03/31/2022, 3:44 PM
Yeah, it's a good call - I was reviewing those 99ish today. Feel like that has creeped substantially over time. We'll need a way to pull in collections anyways, so there definitely isn't any benefit by pulling in
ansible
w

witty-crayon-22786

03/31/2022, 4:45 PM
also, as a general point: PEX uses
pip
under the hood, so in general it should take roughly the same time. some factors that can make it different: 1. cold caches: PEX puts the pip cache in a non-default location (so that we can monitor usage, and more easily clear it) 2. building the PEX for more interpreters than you need it for. for a PEX that you won’t be publishing anywhere, you should ensure that you’re building an “internal_only” PEX: it will only work on a single python interpreter selected by the Pants process, and that avoids building wheels for other interpreters if user’s interpreter constraints are more broad
1
as to venv PEXes: they don’t build any faster, but they run on the order of 200ms (iirc) faster
w

wide-midnight-78598

03/31/2022, 4:47 PM
Thanks @witty-crayon-22786 I was using
internal=True
which is a big reason for my confusion. I didn't know about the cold caches though, that's helpful. Either way, with the switch to
ansible-core
, we're down to like 10 seconds of build time 🚀
🎉 2
@proud-dentist-22844 If you have any recommendations to add here for prototyping this deployment functionality: https://github.com/sureshjoshi/pants-plugins/issues/46 I need to update the documentation of the plugins, which I'm aiming to do tonight
p

proud-dentist-22844

03/31/2022, 4:50 PM
I’ll take a look later 🙂
👍 1