https://pantsbuild.org/ logo
r

rapid-exabyte-76685

05/17/2022, 6:27 AM
Separately from the above,… is there a way to force Pants to run tests in a particular order? e.g. I’m running tests in a particular folder, which has 3
test_<xxx>.py
files - and my tests fail depending on the order in which Pants decides to run the files, so making it a bit time-consuming to force a run that triggers the race condition
1
c

curved-television-6568

05/17/2022, 6:38 AM
Ouch. I’m not aware of any way to order the test runs. But I’d feel like there’s something fishy with the tests when there are dependencies of this kind between tests.
r

rapid-exabyte-76685

05/17/2022, 6:58 AM
Oh I agree that there is. Just trying to debug it. Would be helpful if I got debug output from within the fixtures and the successful tests, but alas I’m not getting any at the moment.
👍 1
c

curved-television-6568

05/17/2022, 7:24 AM
Sorry I’m not of more help here..
r

rapid-exabyte-76685

05/17/2022, 7:43 AM
All good, I have a way to make my asserts more forgiving.
👌 1
But to explain a bit more about my context, I’m running all the tests in a directory/package. This contains three modules,
A
,
S
, and
Q
. So there are six combinations…
ASQ
AQS
SAQ
SQA
QAS
QSA
. I think if
S
runs before
Q
then that is the cause of the problem, but I’m not 100% sure if both
SAQ
and
ASQ
are misbehaving as its time-consuming to force both of these to appear randomly.
Add to that a shared database that should be being cleared between each test… and a package-level fixture, then there’s a lot going on 😅
But the ability to force the order would be useful so that I can fully understand what is going on.
💯 1
c

curved-television-6568

05/17/2022, 9:09 AM
Hmm I wonder if running the pytest sessions interactively will force them to be run sequentially in a defined order (as provided on the cmd line for instance):
./pants test --debug A S Q
w

wide-midnight-78598

05/17/2022, 11:45 AM
What about using Andreas's suggestion of sequential, interactive runs (instead of pytest's parallel running) + something like https://pytest-ordering.readthedocs.io/en/develop/ Also, not sure if re-naming your files is a possibility. I can't recall how pytest enumerates files by default, but maybe there is something you could do to change the ordering via config? Or force the ordering? https://docs.pytest.org/en/7.1.x/example/pythoncollection.html
r

rapid-exabyte-76685

05/17/2022, 12:31 PM
I’ll give these a go… I am running with
--process-execution-local-parallelism=1
anyway
h

hundreds-father-404

05/17/2022, 12:38 PM
yeah
./pants test --debug
makes things run sequentially. I'm not certain the ordering is stable, but we could make it be probably!
4 Views