In my lengthy python requirements I have a dependency that changes a lot more frequently than the rest. Is there anything I can configure so that this dependency changing has minimal impact on bootstrapping time i.e. invalidating the cache of requirements .pex files? Can I split this requirements into a separate target or something, or will these requirements sets always need to be regenerated given a change to one of their dependencies?

Unfortunately not, PEX supports incremental updating of lockfiles but Pants does not yet: https://github.com/pantsbuild/pants/issues/15704

That feature request, whilst useful, isn’t exactly what I’m looking for. Rather than a

—no-update

option, I’d like to update a single dependency that is already specified and have Pants’ environment .pex files e.g. requirements.pex not need to be completely regenerated.

Ah okay - as far as I know this isn’t available through Pants/PEX today

currently there is one big

requirements.pex

per lockfile which is then subsetted as needed. But this would be a really good requirement to mention in the new python backend wishlist discussion, if you can

> currently there is one big

requirements.pex

per lockfile So am I correct in thinking that when I see pants’ output displaying

generating requirements.pex

for multiple processes at once, that’s just a figment of the parallelisation? @happy-kitchen-89482. What does “subsetted as needed” refer to exactly?

Sorry, I should have phrased this more clearly. There is a single big resolve (per lockfile) and then Pants takes the necessary subset of deps for each situation (e.g., for each test) and produces a requirements.txt from it

Okay nice, that’s a lot more clear. That’s close to how I assumed this worked under the hood. I read through that feature wishlist page you linked and saw a few comments covering something like an

update

option for the resolves. 🙂