https://pantsbuild.org/ logo
Join Slack
Powered by
I'm seeing scala/jvm tests using the vintage engin...
# general
f
I'm seeing scala/jvm tests using the vintage engine to interop with JUnit4 succeeding when running 
pants test ::
, when they should fail. While running the test via BSP with Intellij correctly fails. Any ideas, could results be parsed incorrectly?
f
Is IntelliJ just running the tests directly using the classpath Pants told IntellIj about over BSP?
f
Yes should be
f
If you are willing to run Pants from source, you could insert code at https://github.com/pantsbuild/pants/blob/d9a764bd709707ff57ee26dcb268bf8d65277021/src/python/pants/jvm/test/junit.py#L191 to dump what the exit code coming back from junit's 
ConsoleLauncher
is.
If 
ConsoleLauncher
is not returning a non-zero exit code on test failure, then Pants will think the tests succeeded.
f
Ah that's interesting, Intellij detects it as a failed test, but the exit code is:
Process finished with exit code 255
I can run it from source and check on the pants side
👍 1
r
I've seen this issue before as well. We have tests that rely on docker containers, and as a 
scala_junit_test
they would succeed even when the containers hadn't been set up. Converting them to 
scalatest_test
seems to have improved their reliability so I've been using that instead.
f
Didn't have time to get pants from source running yet, but will when able to get more details. Yeah we unfortunately can't because of compatibility
> If you are willing to run Pants from source, you could insert code at https://github.com/pantsbuild/pants/blob/d9a764bd709707ff57ee26dcb268bf8d65277021/src/python/pants/jvm/test/junit.py#L191 to dump what the exit code coming back from junit's 
ConsoleLauncher
is. Sorry for being super slow on that. Yes the exit code is successful (0), while the stdout actually includes test failures
Not the most fun to read, but here's the dump
Untitled
bit of cleanup
Untitled
Copy code
2 tests found
0 tests skipped
2 tests started
0 tests aborted
1 tests successful
0 tests failed
Is a bit interesting, the failing test is just not accounted for at all
This was not an issue with pants but with multiple interop-hops and a best-effort vintage runner, (for ZIO specifically), which is simply not working properly with JUnit5 with the vintage engine, I suspect due to the changes to how reporting is supposed to be done.
We solved the weirdness by putting together a new engine for JUnit5 instead
Open in Slack
PreviousNext
Powered by