I'm trying to run our Python tests, but they're ac...
# general
a
I'm trying to run our Python tests, but they're accessing the db and when pants runs pytest in parallel I get a bunch of concurrency-related errors. What is the shortest path approach to get such tests to run?
I'm considering having a pytest fixture that brings up an isolated postgres intsance per test case
b
In https://www.pantsbuild.org/2.21/docs/python/goals/test#setting-environment-variables there's some advice about
PANTS_EXECUTION_SLOT
to manage concurrent execution. What you say is another option: ensure that tests work fully concurrently (for us, we have a single postgres instance, but unique database names within it. https://pantsbuild.slack.com/archives/C046T6T9U/p1699499512255419?thread_ts=1699480512.369739&cid=C046T6T9U )
a
We do the same as in that message you linked. We apply all our migrations and build a PostgreSQL image, at the end we create template databases from those we need, then we have a pytest plugin that takes care of creating the db on demand and changing things around so that every session gets its own version. It's a bit of a pain to set it up, but once it's set up, it Just Works. It even solved a lot of issues with tests that don't properly clean up, though, in the same file you still need to (our db is huge so it takes ~500ms to create a new version, we want to avoid that per test).
I mean, not exactly the same, but the same idea.
It gets a bit tricky if you have more than one way of creating database sessions (we have a mix of sqlalchemy and asyncpg, and, frankly, the way we use the database is an abomination, we sometimes have 3+ sessions in one app), but it's a one time thing, and I'm a huge fan of the idea
a
Thanks, I was able to use
[pytest].execution_slot_var
for now to get around concurrency issues against pg
👍 1
Should hold us over until after moorepo migration to improve tests