I observe `coverage.py` behaving in a weird way wh...
# general
f
I observe
coverage.py
behaving in a weird way when running tests. It’s being confused about sources for some code:
Copy code
pants.engine.process.ProcessExecutionFailure: Process 'Generate Pytest xml coverage report.' failed with exit code 1.
stdout:
No source for code: '/private/var/folders/k0/3nssm8v15r5b08k1k1wyhs2r0000gp/T/process-executionzCCweC/(builtin)'.
Aborting report output, consider using -i.
I have a few tests which do imports from all kinds of places and despite passing, they may cause
coverage.py
to trip. I wouldn’t post it here given it’s a pure
coverage.py
problem, but the path looks dodgy having
(builtin)
which looks like some variable expansion / string interpolation issue? Have seen Eric had problems with https://github.com/pantsbuild/pants/pull/11129, not sure if that’s relevant or not. It feels wrong that
(builtin)
is in the constructed path. Ignoring errors when generating coverage HTML reports (https://coverage.readthedocs.io/en/coverage-5.5/config.html#report --
ignore_errors
) works fine and the results look sensible.
h
huh, it literally is saying
(builtin)
?
f
yep 😄
I confirm that I’ve copied/pasted the output without any editing
Copy code
pants_version = "2.5.1rc3"
and tried on
Copy code
pants_version = "2.5.1"
h
this is strange. I think it'd be helpful to run with
--no-process-execution-local-cleanup
to inspect what the files look like: https://www.pantsbuild.org/docs/troubleshooting#debug-tip-inspect-the-chroot-with---no-process-execution-local-cleanup
f
oh I did that, looks legit:
Copy code
├── __run.sh
├── coverage.pex
├── coverage.pex_bin_python_shim.sh
├── coverage.pex_pex_shim.sh
and the source directories that were required to run the tests along with the tests modules
👍 1
the shims contents is standard, what it used to be really
so this path
(builtin)
has been collected into the
.coverage
file and of course when producing report it can’t find it
h
Are you able to narrow down by chance which files it's happening on? Like if you're now running
./pants test ::
, try
./pants test dir1::
and see if it reproduces. I'm curious if we can narrow down which specific test is resulting in
(builtin)
being added FYI Pants's coverage works by each test generating its own data, then using
coverage combine
to get a single
.coverage
file
f
Are you able to narrow down by chance which files it’s happening on?
oh yes, I know exactly which file is the problem maker 🙂 wanted to check in with you before continuing narrowing down if that problem looked familiar. Will let you know my findings later this week.
h
Interesting, does that file have anything fishy? I'd love to figure out if Pants is doing anything wrong
f
fishy
it’s importing tons of stuff from various peer projects, a bit too nosy 🙂 we will hunt it down
Found it 🙂 PySide2 is guilty, it seems! You should be able to reproduce it on a example-python repo (https://github.com/pantsbuild/example-python) by adding a few lines of code:
Copy code
diff --git a/BUILD b/BUILD
index 9c0a8bc..9e6a482 100644
--- a/BUILD
+++ b/BUILD
@@ -20,5 +20,7 @@ python_requirements(
   module_mapping={
     "ansicolors": ["colors"],
     "setuptools": ["pkg_resources"],
+    "PySide2": ["PySide2"],
+
   },
 )
diff --git a/helloworld/greet/greeting_test.py b/helloworld/greet/greeting_test.py
index 4faa9e8..7e6de79 100644
--- a/helloworld/greet/greeting_test.py
+++ b/helloworld/greet/greeting_test.py
@@ -3,6 +3,7 @@
 
 from helloworld.greet.greeting import Greeter
 
+from PySide2.QtWidgets import QApplication
 
 def test_greeter() -> None:
     greeter = Greeter(languages=["es"], greetings=["good morning"])
diff --git a/pants.toml b/pants.toml
index b5ee007..b5a06bb 100644
--- a/pants.toml
+++ b/pants.toml
@@ -36,3 +36,9 @@ interpreter_search_paths = ["<PATH>", "<PYENV>"]
 # See <https://www.pantsbuild.org/docs/protobuf>.
 runtime_dependencies = ["//:protobuf"]
 mypy_plugin = true
+
+[test]
+use_coverage = true
+
+[coverage-py]
+report = ["html"]
diff --git a/requirements.txt b/requirements.txt
index ffab1be..f11aee9 100644
--- a/requirements.txt
+++ b/requirements.txt
@@ -5,3 +5,4 @@ ansicolors>=1.0.2
 setuptools<54.0,>=50.3.0
 translate>=3.2.1
 protobuf>=3.11.3
+PySide2
1
h
Great find! I don't think that's a Pants Issue then, but a PySide2 x Coverage.py issue. If you're motivated, next step would be trying to reproduce this without using Pants at all, a trivial repo that uses PySide2 and coverage.py. I suspect you'll have the same error. Then you could file a bug with https://bugreports.qt.io/projects/PYSIDE/issues/PYSIDE-1628?filter=allopenissues (Or, go with your workaround you found and call it a day. But documenting the issue w/ PySide2 could also save future developers time by seeing it's an issue and your workaround)
f
thank you, Eric. I’ll try to reproduce without Pants to see if the
builtin
thing will pop-up. Thanks for helping out with this quest.
❤️ 1
👍 1
I am back 🙂 so it turns out that
coverage.py
runs the code with PySide2 just fine and it looks like Pants may be doing something that causes coverage to be confused with the paths. 😞 I’ve authored a minimal repro in a public GitHub repo — https://github.com/alexey-tereshenkov-oxb/monorepo-coverage-pants. It showcases how
coverage.py
works and how Pants fails. I’ll make
coverage.py
ignore the errors to be able to go on with my life, but thought it may be worth sharing with you the repo in case you’d ever want to step through what Pants is doing on the minimal monorepo. @hundreds-father-404
❤️ 1
h
Thanks for the repo! Could you please open a GitHub issue? It's been a busy week and I'm not sure I'll be able to get to this today, but definitely don't want to forget about it
1
f
oh this is super low priority! Just wanted to be helpful and provide the necessary reference to explore further, if time permits.
e
This is definitely a Pants issue in its use of coverage. I've updated the ticket.
❤️ 1