within the test ? I’ve tried two ways -

psutil.Popen('./pants run {target}', shell=True)

and

psutil.Popen([os.getcwd() + '/pants', 'run', '{target}'])

For the sake of debugging, it might help to start by trying to get it working directly on the command line, rather than using

psutil.Popen

. Once you get that working, then switch to

psutil.Popen

Here,

psutil.Popen

is a little distracting from the underlying issue

the binary runs fine, are you suggesting i try to run coverage on the executable manually ?

To clarify: is the

Exception: ModuleNotFoundError("No module named '__pants_backend_python_tasks_pytest_prep_coverage_plugin__'",)

error happening within your subprocess, or just during the test which uses the subprocess?

it’s happening in the test, I believe I’ve tried doing what pytest-cov does manually (as far as I understand, set

$COVERAGE_PROCESS_START

and then in my initialization routine invoke

coverage.process_startup()

), but I can try that again and see what happens

are you suggesting i try to run coverage on the executable manually ?

I mean try running

./pants run target

directly on the command line. Is that working? If so, but it doesn’t work with

subprocess.Popen

, then we know the issue has something to do with

subprocess.Popen

rather than an underlying issue with the call to Pants

the target standalone runs without any issues, the issue is when pytest-cov tries to start running coverage on the subprocess, which is what i could try explicitly doing myself

You should not need to call coverage directly.

that’s true if the process is started within a situation where coverage is already running, if i just

./pants run

, I believe coverage doesn’t get invoked

Are you trying to get coverage to run when running

./pants run target

inside of a test?

sort of, to clarify, i’m running an integration test, and need to run the code being profiled in a separate process outside the test itself

The target environment likely does not include the coverage module as it is not a test.

to be clear, running the test in this fashion and running coverage on it does work correctly if I use pytest

It's all good, I'm just trying to understand what you're doing and how we can help.

since this an inherited codebase i’m working on, i’ve noticed there’s some settings in pants.ini that i don’t quite understand and can’t find docs for -

[GLOBAL]
pants_version: 1.21.0

plugins: +[
    "pantsbuild.pants.contrib.node==%(pants_version)s",
    "pantsbuild.pants.contrib.python.checks==%(pants_version)s",
    "pantsbuild.pants.contrib.mypy==%(pants_version)s",
    "pylint==1.9.3",
  ]

I’ve tried setting packages here (had run into a no module coverage issue, and if i include it here it seems to work) and here -

[DEFAULT]
[pytest]
requirements: pytest==3.6.4
cov_requirements: pytest-cov==2.5.1

Hah

when it works with pytest, i’m not using pants in the subprocess.

I have some answers, I think. The pants.ini values you're seeing should be the ones that are being used. You can also pass these on the command line: with

--pytest-version=<str>

and

--pytest-pytest-plugins=<str>

BUT: these things are only available when you are running

./pants test

they are not available in the environment when you

./pants run

makes sense generally, so if i use a pex or something i’ll run into the same issue

I believe so.

ok, i was trying to figure out how the pants integration tests run

I'm not sure that what you're attempting is currently possible with pants. If you can run your binary with something other than

./pants run

from within your integration test you may get the behavior you're looking for.

hmm, ok

To clarify for myself, it sounds like you have an integration test that happens to fork out to

./pants run

and you want to run coverage on that

./pants run

invocation?

yes, as supported by pytest-cov and coverage

I see, so you're actually not directly using the coverage support in

./pants test

but setting up coverage manually yourself in your

./pants run

invocation

well, i’m setting the options in ./pants test, i just want that to include the data from the subprocess

But you want the resulting coverage results to be merged back in with pants's regular

./pants test

coverage?

yeah, basically

Hmmm

yeah, pytest-cov does some crazy crap to get this to work, but i think pants does somethings to further isolate than standard python

And are you able to get coverage data when you run your process, but you can't make

./pants test

pick it up and merge it into the test's coverage? Is that an accurate representation of the problem?

yes, something is generated, i haven’t taken a look at what exactly to be honest

I.e., your forked process does in fact produce a coverage file, but Pants doesn't do anything with it, and you want it to?

i’ve mostly been focused on trying to get the merged thing to work since it seems easiest

Have your process drop coverage files in some well-known location

but pants doesn’t seem to generate .coverage files, i just see html and xml

It generates them, but only internally

ok

In the short term that's probably right

ok, that’s a good starting point, thanks

But this is great feedback, we can probably provide this as a feature in v2

Of note: running

coverage combine

on

.coverage

files will delete the

.coverage

files. I suggest copying them to some other location before doing your merge

ok, is there some failsafe way to find the .coverage files ?

In v1 I am not sure, sorry 😞

another coverage question … for a non subprocess/crazy scenario, what i’m seeing is that when i run

./pants test {target} — --cov=targetdir --

pretty much nothing is recorded (init files are 100% and everything else is more or less 0) Then, I switched to trying to supply coverage targets via --test-pytest-coverage=modules:, and then frequently modules that I know for a fact are included don’t get any hits. Any guidance on the best approach here ?