Hello I’m starting to integrate pantsbuild into ou...
# general
j
Hello I’m starting to integrate pantsbuild into our python monorepo (cause it seems great!) and was wondering - We’re using testcontainers in our test suite and that sometimes means there’s a sequentiality needed in our case. i.e. 2 tests using the same PSQL table - currently we would load the table for test1, run the test, then truncate the table and move on to test2 which will do the same. Do you guys have any suggestions regarding migrating such a case? How would you approach this on your repo (perhaps you’d also opt against using containers?)
e
Pants aside, I'd structure my tests to neither depend on database host/port nor database name - just depend on a db connection and then table names, etc down from there. This generally allows parallelizing db tests by creating throwaway dbs per-test. Now, if db initialization is expensive compared to test runtime, you may still need to serialize, but, at base, not relying on these 3 things is a good start.
j
Hi John, thank you for the response. We indeed don’t depend on the database host/port etc, we inject a test connection using a fixture. I just am wondering - if we run parallel, it means we basically will have multiple sessions right? So each session will spin up a container. WDYM by serialize the db initialization?
e
I just mean avoid this:
i.e. 2 tests using the same PSQL table - currently we would load the table for test1, run the test, then truncate the table and move on to test2 which will do the same.
Test 1 and test 2 should be using two different databases unless the initialization of the database (applying the schema and loading data) is more expensive than the tests.
Generally the same database server can hold N databases inside in parallel - depends on the db server though.
j
Oh I see what you mean now, cool, thanks
h
If you run N tests concurrently and you have N logical databases (in the same DB server, as John suggests) then you can assign DBs to tests using the
execution_slot_var
option: https://www.pantsbuild.org/docs/reference-pytest#execution_slot_var
You use that option to name an env var that, at test runtime, will contain an integer [0, N) that is guaranteed not to overlap with that of any other currently executing process
That is if your tests themselves can't create unique databases on the fly (for example, Django's test framework will create unique DBs for you and you don't need this)
🙏 1