https://pantsbuild.org/ logo
#general
Title
# general
c

curved-vase-73456

12/28/2018, 3:44 AM
hey guys! Context I’m trying to produce a combined coverage from unit, integration, blackbox tests for multiple projects that share same code and having a bit of trouble understanding how it can be done properly using
pants
. Details - unit and integration tests are collecting coverage via
./pants test
with
--test-junit-coverage-processor=jacoco --test-junit-coverage
options - for blackbox I couldn’t figure out how to use pants natively, so it’s done using
JaCoCo
agent to create
foo.exec
and
bar.exec
files with
java -javaagent:jacocoagent.jar -jar dist/<path to foo.jar>
for bundles that are created by
./pants bundle
and then combined report can be generated via
java -jar jacococli.jar report foo.exec bar.exec --classfiles dist/bar.jar --classfiles dist/baz.jar --html <dir to output report to>
Questions - Is there a way to run bundles using pants and collect coverage data and merge reports for multiple different bundles? - if not, where one can find classfiles that
pants
is using to produce report so that one can specify them when creating a combined report?
jacoco.exec
can be found in
.pants.d/junit/_runs/..
folder but I don’t see any classfiles there,
tmp*.jar
don’t seem to contain the info needed as well, and .pants.d/compile folder contains too many classes I don’t care about and even with filters it seems like some deduplication might be needed
a

aloof-angle-91616

12/28/2018, 3:51 AM
how are you planning to create the combined report (like what command lines would you run to do that)? i would start perhaps by subclassing the pants task that produces jacoco reports instead of trying to hop around
.pants.d/
(which is maybe 1/4 of an answer to your first question)
c

curved-vase-73456

12/28/2018, 3:54 AM
java -jar jacococli.jar merge foo.exec bar.exec
creates merged
.exec
file, after that report can be generated using
java -jar jacococli.jar report combined.exec --classfile <list of classfiles>
or just
Copy code
java -jar jacococli.jar report foo.exec bar.exec --classfiles dist/bar.jar --classfiles dist/baz.jar --html <dir to output report to>
the trouble I have is getting classfiles for unit and integration tests
btw @aloof-angle-91616 ty for jumping in and helping out! appreciate it 🙂
👖 1
❤️ 1
a

aloof-angle-91616

12/28/2018, 4:05 AM
tasks registered for
./pants bundle
will typically (should) write their result to the
'deployable_archives'
product.
src/python/pants/backend/jvm/tasks/bundle_create.py
. it uses the older
ProductMapping
api, which essentially is a two-level map from
(target -> base dir) -> [list of objects]
. you can consume that product by using
round_manager.require('deployable_archives')
in a
prepare()
classmethod override, then getting the product mapping from
self.context.products.get('deployable_archives')
(which gives you a ProductMapping. there are some examples in the repo of how to use the ProductMapping API, but the class is also pretty well-documented in the repo source. you would probably want to then get your target set with
self.context.targets(...)
and
with self.invalidated(...)
to get invalid targets, and then you can run jacoco on each of these in turn, and then merging them, by using the jvm tool api, which i'm assuming you're already using.
Copy code
with self.invalidated(...) as invalidation_check:
  invalid_versioned_targets = invalidation_check.invalid_vts # list of VersionedTarget
  for vt in invalid_versioned_targets:
    vt.target # the target
    vt.results_dir # where you would put the output of your task for that target -- VersionedTargetSet also lets you generate a `results_dir` for a list of targets (which you would use for the merge step
c

cuddly-architect-27013

12/28/2018, 5:06 AM
FWIW, the jacoco integration in pants by default specifies the classfiles for all dependent targets when generating the report. consequently jacoco reports by default include coverage info for all deps that get loaded during the test run. There’s a new-ish flag that allows a user to specify a set of regexes to identify targets that should be included. targets whose canonical id matches one of the regexes will be passed to jacococli during report generation, and jacoco will only report on a class if it’s specified (i.e. even if it has execution data, if it’s not specified as --classfiles it won’t be in the report)
👖 1
☝️ 1
c

curved-vase-73456

12/28/2018, 5:21 AM
@cuddly-architect-27013 ty for response, yeah it works like magic for unit and it tests, the issue I had is how to make it work with blackbox testing i.e. having standalone executable jar/jars, run them and run tests against them, I don’t see how one can do this using
pants
built-in commands to make offline/on-the-fly instrumentation like
javaagent
does and produce coverage reports for it. Is there an option to run instrumented jar bundles using pants instead of doing
java -javaagent:jacocoagent.jar=includes=<pattern> -jar foo.jar
? let me know if it makes sense 😅
c

cuddly-architect-27013

12/28/2018, 5:32 AM
hmmm. i guess i’m not 100% clear on the structure of your project. if you have a jar you’re doing blackbox testing on, and the tests are pants junit targets, then the blackbox jars should be some binary dep (I guess?). pants doesn’t specify any includes to the jacoco agent, so all jars loaded into the test should be instrumented at runtime, and execution data should be gathered. the trick then is getting a report that doesn’t have a bunch of noise.
which i assume is why you were asking about classfiles
c

curved-vase-73456

12/28/2018, 5:34 AM
yeah, I think you got it right
c

cuddly-architect-27013

12/28/2018, 5:34 AM
i’m rereading your original questions to make sure i’m understanding this
i suspect we don’t have a clean way to do this, tbh
so blackbox is some jar that’s produced by someone else or some other process?
a

aloof-angle-91616

12/28/2018, 5:37 AM
ah, i was assuming it was produced by
./pants bundle
for no reason
c

cuddly-architect-27013

12/28/2018, 5:37 AM
oh wait no i see…they are bundles
c

curved-vase-73456

12/28/2018, 5:37 AM
I’m using
./pants bundle
to make executable jars for bunch of services
c

cuddly-architect-27013

12/28/2018, 5:37 AM
yes ok, sorry
a

aloof-angle-91616

12/28/2018, 5:39 AM
the
'deployable_archives'
product definitely works for that just would require a separate pants task
c

curved-vase-73456

12/28/2018, 5:41 AM
I have like 5 servers that run simultaneously and communicate with each other via grpc or http, and I have a bunch of api end-to-end tests which I want to collect coverage on
c

cuddly-architect-27013

12/28/2018, 5:42 AM
gotcha
so when pants produces the report, i believe it does use the classfiles out of the .pants.d/compile folder. i think for a bundle, assuming it’s an archive, you may need to unpack the archive to find the classes you want to pass to the jacoco cli report step
☝️ 1
i can’t think of an existing way to get what you want, so you’ll have to probably write some scripting that works with the bundles and jacoco cli, or alternately add functionality into pants
sorry 😞
c

curved-vase-73456

12/28/2018, 5:48 AM
@cuddly-architect-27013 it’s cool, ty for your response and help! it’s a path I’m following currently just wanted to know that I’m not reinventing 💡 😃 btw while I got your attention, by any chance do you know how to configure
scoverage
scalac plugin to use with pants for similar stuff? JaCoCo is pretty good but it has some caveats that scoverage does better for scala code
c

cuddly-architect-27013

12/28/2018, 5:49 AM
HAH ACTUALLY! I am planning on looking into that over the next few days. we’ve been interested in using scoverage as well
👖 2
c

curved-vase-73456

12/28/2018, 5:49 AM
wooo!
c

cuddly-architect-27013

12/28/2018, 5:49 AM
pants does support scalac plugins, so there are a few ways i think would work off the bat
one is specifying the plugin as part of the target definition
in that case you might have parallel target definitions for the normal scala target, and the scoverage instrumented version
and then you can also specify a scalac plugin in pants.ini
c

curved-vase-73456

12/28/2018, 5:52 AM
would be very curious to see example project or some guide on integration, I wanted to follow
scoverage
path but already got pretty far on current one to change gears, not enough sprint time 😅
👍 1
c

cuddly-architect-27013

12/28/2018, 5:54 AM
ok, well if possible i’ll share what i do…plz don’t bank on it or anything, but maybe i’ll come up with something helpful, or even a plan to integrate it into pants
or maybe just a simple guide, idk
c

curved-vase-73456

12/28/2018, 5:55 AM
ty ty 🙏🏼 I’ll follow your advice and will keep my excitement about it to myself 😃
well, at least out of immediate work
8 Views