https://pantsbuild.org/ logo
#general
Title
# general
w

worried-painter-31382

07/31/2022, 9:52 PM
Is there a way to make pants run pytest tests in a batch? The OH of running each in a separate process and creating a pex for it causes about 3 times slow down for a "small" lib by just switching out
pytest <module>
with
./pants test <module>::
in our github actions.
👀 1
A target just globbing all python_test targets that are children of current directory would be nice enough, but I suppose that isn't a thing? Edit: https://github.com/pantsbuild/pants/issues/14941 No, it isnt.
Could I write this plugin myself, or is it a bad idea?
w

wide-midnight-78598

07/31/2022, 10:52 PM
Have you tested this out in your CI? https://www.pantsbuild.org/docs/using-pants-in-ci#tuning-resource-consumption-advanced You can run tests sequentially instead of parallel.
h

happy-kitchen-89482

08/01/2022, 6:42 PM
There's been a lot of discussion of this. I personally think test batching is an important feature to add, but not all agree.
1
That said, in this case, I'm curious about the "creating a pex for it" part of the overhead
Is this a 3rdparty requirements pex?
Are you using a lockfile?
With a (pex) lockfile, the 3rdparty requirements for each test are created by subsetting the lockfile, and that should be really fast.
👍 1
which eliminates that entire thing by running each test against the entire lockfile
so you get worse caching - every test will be invalidated if any requirement changes, but possibly faster performance when no requirements have changed
w

worried-painter-31382

08/01/2022, 7:10 PM
Mm, I found this (lockfile having perf implications for dependencies) scouring the source and backtracking to the docs got me to the FYI https://www.pantsbuild.org/docs/python-third-party-dependencies#lockfiles here. The lockfile does speed it up a fair bit, which is nice! I started out without it because we've maintained our own "lockfile" via pip-compile and bash, and I didnt realize the benefits of a (pex) lockfile vs a single requirements file pointing out exact dependencies for every source (pip-compile result). We're still suffering from scoped pytest fixtures re-computing for every test file (its aws-cdk templates which are SLOW to generate). Thankfully they are a realitively small subset of the module. Ill follow up this thread when I have tested the other option, although the docs seem tell me not to now that I use a lockfile :)
h

happy-kitchen-89482

08/01/2022, 7:25 PM
Yeah, lockfiles should generally sort you out, that other option is for when even that case is slow