So, a week ago I asked whether pex files are built in two steps (one with requirements, which change rarely, and one with the code), since I knew I saw something about that. It turns out this was a pants v1 feature (thanks @worried-pager-41972 for figuring that out). My question is, wouldn't this be something that should be in the current pants? So, for tests, we think we figured it out, circleci caching is hard. But, for packaging, you have to start from scratch every time.

In this blog post https://blog.pantsbuild.org/optimizing-python-docker-deploys-using-pants/ @bitter-ability-32190 talks about leveraging two PEXs using the options

include_sources

and

include_requirements

. Perhaps that is close to what you’re looking for @ancient-france-42909?

Oh, we discovered the venv bit at built time ourselves 🙂 But, that sounds like what we're looking for, I didn't think of combining these two, though, heh

We can all thank Facebook for

torch

😛

Yeah, we have tensorflow sprinkled all over our docker images because someone needed to import an

Enum

from a package that depended on that.

Juicy

Amazingly, we're actually working towards fixing this, but still, I get sad every time I look at how long it takes to pull one of these images without cache, heh

yep, it should all be nicely cached.

If we manage to get the pants resolving transitive deps time down from the 3+ minutes it is sometimes, that'd be great, but from my past conversations, that's probably a matter of using lockfiles