https://pantsbuild.org/ logo
Join Slack
Powered by
This is probably a bit of a nebulous question, but...
# general
i
This is probably a bit of a nebulous question, but was there any parallelization added to unit test running between pants 1.8 and 1.26? For some context, I have a few transient unit test failures that are occurring, but only when I run our entire test suite from scratch. Trying to pin down what might be causing these issues.
a
i believe the short answer is “yes” and that there is an option to disable this. which language is this?
i
Python
a
let me check the options. i’m pretty sure the default was made to running all tests in parallel before 1.25, and it would be an option probably on the pytest subsystem
w
the v1 backend will never run python tests in parallel, afaik.
👍 1
a
what is the command line you’re using to run tests?
thanks stu
i was wrong, sorry
w
options that did change in that area are 
--[no-]fast
and 
--[no-]chroot
(both of which are on by default in v2/2.0)
i
Ah ok. I will check out those two options you mentioned. Thanks 😄
a
let us know if the help isn’t sufficient!
h
We did change the default to be 
--no-fast
, though, which means that tests have more isolation than they did before. It’s likely that that was what impacted this. 
--no-fast
means that each test runs in its own invocation. It’s not in parallel per se, but it is isolated. So, you likely had shared state between the tests. You can run with 
fast = true
on the 
[test.pytest]
scope to confirm.
w
a strange thing though is that `--no-fast`should be more deterministic, rather than less.
so the failures being “transient” is surprising.
i
@hundreds-father-404 I will try that out and update here if it affects things at all
h
so the failures being “transient” is surprising.
Yeah, agreed there. Same with 
--chroot
. The impact of 
--chroot
being the default is that you now run in a temporary directory, rather than directly in the build root. This means you can’t access any random files in your build root anymore, unless they are declared via 
dependencies
. You either have a file included or you don’t; that failure should be consistent.
i
Somewhat of an update, using the 
fast = true
flag caused the unit tests to run indefinitely.
👀 1
h
That’s strange. We used to default to fast = true, so I wouldn’t expect that because it means that your CI was using fast = true for a while before you upgraded to more modern Pants.
i
Right, I assumed it would behave like the older version. Should I try changing the 
chroot
option as well?
h
Worth trying, but yeah as mentioned above, I would expect chroot failures to be deterministic. Agreed, and I’m also surprised because we haven’t touched the V1 code in a long time, outside or changing the defaults of those flags. I don’t think the different behavior would be from a certain code change we made.
i
Yeah, it certainly is strange that the two versions appear to have different behaviors. Granted, we did see similar errors on the older version, but only when they were run in a much faster environment. I'll have to track down what causes these failures before completing our upgrade. Thanks for the help!
h
I wish the help resulted in an answer, but you’re welcome nonetheless! Do you know if the problematic tests share any state between the tests, such as accessing a database?
A major motivation for us eventually removing the fast flag is that it makes it possible for your tests to only pass depending on the execution order of those tests 😬
i
They do not share a state as far as I can tell. There may be a library somewhere that I am missing, but they usually fail with a non descriptive 
asyncio:Task was destroyed but it is pending!
. So, it seems to be some async issue
4 Views
Open in Slack
PreviousNext
Powered by