Hi, We are facing problems with coverage-py that I was hoping someone knew a solution to. We have a monorepo type repo where one of the projects run with 100% test coverage. We are omitting the other subprojects to be included in the report:

# .coveragerc
[run]
omit =
    project_0/python/*
    ...

# pants.toml
[coverage-py]
config = ".coveragerc"
install_from_resolve = "tools"
fail_under = 100

The issue occurs when no changes to the tested project that is tested. In this case, since no tests are run, no report is generated and coverage-py fails the coverage check:

23:41:59.73 [INFO] Starting: Building coverage_py.pex
23:42:01.54 [INFO] Completed: Building coverage_py.pex
23:42:01.91 [INFO] Preserving local process execution dir /tmp/pants-sandbox-vS2dyM for Generate Pytest report coverage report.
Error: 1.91 [ERROR] 1 Exception encountered:

Engine traceback:
  in `test` goal

ProcessExecutionFailure: Process 'Generate Pytest report coverage report.' failed with exit code 1.
stdout:
No data to report.

A quick solution is to add a new dummy test that always run in the project to force coverage-py to run on that. But we are not really fan of that solution as it is kind of hacky. Maybe someone with more knowledge in pytest knows how to do this?

Hey Martin, What currently work for us is:

# pants.toml
[coverage-py]
global_report = true

so that all files are included on the final report, even the ones who weren't tested, and:

# .coveragerc
[report]
omit = 
    */__init__.py
    */*_test.py

Thank you, This helped on the first issue by generating the report regardless. But we are now getting the issue that since none of the tests run it misses all and the fail_under fails 😞 The first image is

pants test ::

while the second is

pants test path_to_file_not_tested_in_report

I'm assuming this is only when running pants locally, right? Maybe you could use the

fail_under = 100

only when running the tests on your CI?

It also fails in CI, since we currently run with

--changed-since=origin/main

. The issue occured in CI because no change to the reported files was changed. We will be moving to caching at some point, I just need to prioritize it😅

Coverage is tricky when tests are run selectively. Coverage, by definition, needs to know about the entire repo, and tools like Pants try not to run things in the context of the entire repo, for better caching. there could be attempts to split up and reassemble coverage data, but it’s quite fragile.

Yes. To some degree I'm surprised that I cannot set different limits on different folder/file/project level. To me this seems like a basic feature if using a tool to check coverage. It also turned out the method above did not work either. I recently added coverage in replacement of my own script that parsed my files for function names and matched with imports in the test directory. I was happy that I could remove custom, boilerplate code, but I realize that I probably will need to revert that decision.