Hi, how does pants handle VCS dependency using pex during test? My tests are taking quite a long time (~30 mins). Is it building this dependency from source for every test ? I am using multiple resolves if that's something that might be relevant.

Yeah, Pex clones the repo and builds it like it's an sdist. But it should be reusing the work from other runs. Sounds like you're not seeing that?

I am not sure. The only thing I saw was pants accumulating all these dependencies for each test and it took forever. Earlier I was using a global

requirements.txt

where this was a dependency and the tests used to run much quicker. But after I switched to multiple resolves using pex, these tests are taking forever

Yeah the old approach was: build a PEX once w/ everything, then extract out smaller subset PEXes from that Now the approach is: directly build the smallest PEX necessary for the task. Share Pex's cache of downloading wheels & building sdists etc across those different runs.

Share Pex's cache of downloading wheels & building sdists etc across those different runs.

It's crucial that that is working properly - so definitely please do keep us informed on this!

It does take consistently ~30 minutes when running with multiple resolves on aws. Earlier without multiple resolves it used take like 5-6 minutes. All my 3rd party dependencies except this one are defined inside

pyproject.toml

which is then used by

poetry_requirements

On the other hand I use

python_requirement

for installing prophet from git. I do provide the same resolve for

poetry_requirements

and

python_requirement

are you using

lockfile_generator='pex'

? see https://www.pantsbuild.org/docs/reference-python#section-lockfile-generator