hey guys! Context I’m trying to produce a combined coverage from unit, integration, blackbox tests for multiple projects that share same code and having a bit of trouble understanding how it can be done properly using

pants

. Details - unit and integration tests are collecting coverage via

./pants test

with

--test-junit-coverage-processor=jacoco --test-junit-coverage

options - for blackbox I couldn’t figure out how to use pants natively, so it’s done using

JaCoCo

agent to create

foo.exec

and

bar.exec

files with

java -javaagent:jacocoagent.jar -jar dist/<path to foo.jar>

for bundles that are created by

./pants bundle

and then combined report can be generated via

java -jar jacococli.jar report foo.exec bar.exec --classfiles dist/bar.jar --classfiles dist/baz.jar --html <dir to output report to>

Questions - Is there a way to run bundles using pants and collect coverage data and merge reports for multiple different bundles? - if not, where one can find classfiles that

pants

is using to produce report so that one can specify them when creating a combined report?

jacoco.exec

can be found in

.pants.d/junit/_runs/..

folder but I don’t see any classfiles there,

tmp*.jar

don’t seem to contain the info needed as well, and .pants.d/compile folder contains too many classes I don’t care about and even with filters it seems like some deduplication might be needed

how are you planning to create the combined report (like what command lines would you run to do that)? i would start perhaps by subclassing the pants task that produces jacoco reports instead of trying to hop around

.pants.d/

(which is maybe 1/4 of an answer to your first question)

java -jar jacococli.jar merge foo.exec bar.exec

creates merged

.exec

file, after that report can be generated using

java -jar jacococli.jar report combined.exec --classfile <list of classfiles>

or just

java -jar jacococli.jar report foo.exec bar.exec --classfiles dist/bar.jar --classfiles dist/baz.jar --html <dir to output report to>

the trouble I have is getting classfiles for unit and integration tests

tasks registered for

./pants bundle

will typically (should) write their result to the

'deployable_archives'

product.

src/python/pants/backend/jvm/tasks/bundle_create.py

. it uses the older

ProductMapping

api, which essentially is a two-level map from

(target -> base dir) -> [list of objects]

. you can consume that product by using

round_manager.require('deployable_archives')

in a

prepare()

classmethod override, then getting the product mapping from

self.context.products.get('deployable_archives')

(which gives you a ProductMapping. there are some examples in the repo of how to use the ProductMapping API, but the class is also pretty well-documented in the repo source. you would probably want to then get your target set with

self.context.targets(...)

and

with self.invalidated(...)

to get invalid targets, and then you can run jacoco on each of these in turn, and then merging them, by using the jvm tool api, which i'm assuming you're already using.

FWIW, the jacoco integration in pants by default specifies the classfiles for all dependent targets when generating the report. consequently jacoco reports by default include coverage info for all deps that get loaded during the test run. There’s a new-ish flag that allows a user to specify a set of regexes to identify targets that should be included. targets whose canonical id matches one of the regexes will be passed to jacococli during report generation, and jacoco will only report on a class if it’s specified (i.e. even if it has execution data, if it’s not specified as --classfiles it won’t be in the report)

@cuddly-architect-27013 ty for response, yeah it works like magic for unit and it tests, the issue I had is how to make it work with blackbox testing i.e. having standalone executable jar/jars, run them and run tests against them, I don’t see how one can do this using

pants

built-in commands to make offline/on-the-fly instrumentation like

javaagent

does and produce coverage reports for it. Is there an option to run instrumented jar bundles using pants instead of doing

java -javaagent:jacocoagent.jar=includes=<pattern> -jar foo.jar

? let me know if it makes sense 😅

hmmm. i guess i’m not 100% clear on the structure of your project. if you have a jar you’re doing blackbox testing on, and the tests are pants junit targets, then the blackbox jars should be some binary dep (I guess?). pants doesn’t specify any includes to the jacoco agent, so all jars loaded into the test should be instrumented at runtime, and execution data should be gathered. the trick then is getting a report that doesn’t have a bunch of noise.

yeah, I think you got it right

i’m rereading your original questions to make sure i’m understanding this

ah, i was assuming it was produced by

./pants bundle

for no reason

oh wait no i see…they are bundles

I’m using

./pants bundle

to make executable jars for bunch of services

yes ok, sorry

the

'deployable_archives'

product definitely works for that just would require a separate pants task

I have like 5 servers that run simultaneously and communicate with each other via grpc or http, and I have a bunch of api end-to-end tests which I want to collect coverage on

gotcha

@cuddly-architect-27013 it’s cool, ty for your response and help! it’s a path I’m following currently just wanted to know that I’m not reinventing 💡 😃 btw while I got your attention, by any chance do you know how to configure

scoverage

scalac plugin to use with pants for similar stuff? JaCoCo is pretty good but it has some caveats that scoverage does better for scala code

HAH ACTUALLY! I am planning on looking into that over the next few days. we’ve been interested in using scoverage as well

wooo!

pants does support scalac plugins, so there are a few ways i think would work off the bat

would be very curious to see example project or some guide on integration, I wanted to follow

scoverage

path but already got pretty far on current one to change gears, not enough sprint time 😅

ok, well if possible i’ll share what i do…plz don’t bank on it or anything, but maybe i’ll come up with something helpful, or even a plan to integrate it into pants

ty ty 🙏🏼 I’ll follow your advice and will keep my excitement about it to myself 😃