Is there a way to make pants run pytest tests in a batch? The OH of running each in a separate process and creating a pex for it causes about 3 times slow down for a "small" lib by just switching out

pytest <module>

with

./pants test <module>::

in our github actions.

Have you tested this out in your CI? https://www.pantsbuild.org/docs/using-pants-in-ci#tuning-resource-consumption-advanced You can run tests sequentially instead of parallel.

There's been a lot of discussion of this. I personally think test batching is an important feature to add, but not all agree.

Mm, I found this (lockfile having perf implications for dependencies) scouring the source and backtracking to the docs got me to the FYI https://www.pantsbuild.org/docs/python-third-party-dependencies#lockfiles here. The lockfile does speed it up a fair bit, which is nice! I started out without it because we've maintained our own "lockfile" via pip-compile and bash, and I didnt realize the benefits of a (pex) lockfile vs a single requirements file pointing out exact dependencies for every source (pip-compile result). We're still suffering from scoped pytest fixtures re-computing for every test file (its aws-cdk templates which are SLOW to generate). Thankfully they are a realitively small subset of the module. Ill follow up this thread when I have tested the other option, although the docs seem tell me not to now that I use a lockfile :)

Yeah, lockfiles should generally sort you out, that other option is for when even that case is slow