Is there a way to make pants run pytest tests in a...
# general
Is there a way to make pants run pytest tests in a batch? The OH of running each in a separate process and creating a pex for it causes about 3 times slow down for a "small" lib by just switching out
pytest <module>
./pants test <module>::
in our github actions.
👀 1
A target just globbing all python_test targets that are children of current directory would be nice enough, but I suppose that isn't a thing? Edit: No, it isnt.
Could I write this plugin myself, or is it a bad idea?
Have you tested this out in your CI? You can run tests sequentially instead of parallel.
There's been a lot of discussion of this. I personally think test batching is an important feature to add, but not all agree.
That said, in this case, I'm curious about the "creating a pex for it" part of the overhead
Is this a 3rdparty requirements pex?
Are you using a lockfile?
With a (pex) lockfile, the 3rdparty requirements for each test are created by subsetting the lockfile, and that should be really fast.
👍 1
which eliminates that entire thing by running each test against the entire lockfile
so you get worse caching - every test will be invalidated if any requirement changes, but possibly faster performance when no requirements have changed
Mm, I found this (lockfile having perf implications for dependencies) scouring the source and backtracking to the docs got me to the FYI here. The lockfile does speed it up a fair bit, which is nice! I started out without it because we've maintained our own "lockfile" via pip-compile and bash, and I didnt realize the benefits of a (pex) lockfile vs a single requirements file pointing out exact dependencies for every source (pip-compile result). We're still suffering from scoped pytest fixtures re-computing for every test file (its aws-cdk templates which are SLOW to generate). Thankfully they are a realitively small subset of the module. Ill follow up this thread when I have tested the other option, although the docs seem tell me not to now that I use a lockfile :)
Yeah, lockfiles should generally sort you out, that other option is for when even that case is slow