Hi everyone, I’m evaluating migrating a monorepo to pants. So far I’ve managed to get everything building and it’s looking great. A lot of services were written under the assumption a single DB connection would be kept/shared amongst all the tests for that service (I’m sure this is not uncommon in python webdev) i.e.

service/a/test/test_a.py
service/a/test/test_b.py 
service/a/test/test_c.py

Will re-use the same DB. Since pants runs each pytest in a separate process I had to create a unique DB name for each and run any initial migrations xN times. This overhead unfortunately makes the tests slower than they were before, even with parallel execution. Is there’s any mechanism to force pytest to run tests in a single process? Passing

--debug

still appears to run the DB setup/teardown once per test file. Whereas ideally I’d like it to run once for the whole suite.

Have you had a chance to try this? https://www.pantsbuild.org/docs/troubleshooting#controlling-test-parallelism

I think this is exactly what I’m looking for ☝️

🙂

@dazzling-elephant-33766 that scenario is the exact use-case that motivated the “pytest batching” feature added in Pants 2.15: https://www.pantsbuild.org/v2.15/docs/python-test-goal#batching-tests

PS You can have the best of both worlds by running multiple concurrent batches, and using execution_slot_var (https://www.pantsbuild.org/docs/reference-pytest#section-execution-slot-var) to name a database that is unique to the concurrency slot

i upgraded to 2.15 and started using an unique batch_compatibility_tag for tests under a directory. but I still have test failures for db table conflicts when running

./pants test ::

but each test passes when I run them individually like

./pants test file/to/tests/foo.py

Have not tried execution_slot_var yet.

A combination of batching +

execution_slot_var

is almost certainly what I’m looking for here. So I can run the db/init/migration setup once per service, but test each service against a different DB. For now I’d just like to get the batching working, and worry about execution slots + different DB’s later. I’m on

2.15.0rc1

and I’ve added the following to the

BUILD

file in the same dir as my

tests/conftest.py

as detailed in https://www.pantsbuild.org/v2.15/docs/python-test-goal#batching-tests

python_test_utils(
    name="test_utils",
)

__defaults__({(python_test, python_tests): dict(batch_compatibility_tag="your-tag-here"),})

Are there additional steps required here? Pants still appears to be running tests in multiple processes, because half the suite is failing (as the db already exists)

psycopg2.errors.DuplicateDatabase: database "blah" already exists

I’m a bit wary of this snippet:

Compatible tests may not end up in the same

pytest

batch if:

• There are “too many” tests with the same

batch_compatibility_tag

, as determined by the

[test].batch_size

setting.

• Compatible tests have some incompatibility in Pants metadata (i.e. different

resolve

or

extra_env_vars

).

perhaps I’m hitting some sort of batch limit, but ideally I’d like this to be unbounded.

@plain-night-51324 that is exactly what execution_slot_var is designed to solve...

Adding this in my

pants.toml

in the root of my repo hasn’t changed the behaviour. I’m still seeing what appears to be separate pytest processes for each test file. I’m invoking the tests with

./pants test services/a/tests:: -- -s

and observing pytest trying to create the same test DB over and over.

[test]
batch_size = 1000

I’m traveling most of today but I can help debug tomorrow if you’re still hitting problems. At first glance I don’t see an obvious issue

Many thanks for the offer Dan, I’ll continue playing around, no time pressure at all.