https://pantsbuild.org/ logo
#general
Title
# general
f

future-oxygen-10553

10/13/2022, 1:25 PM
I’m curious about how coverage and
--changed-since
interact in testing. Or, rather, I can see that coverage only shows results for the files that actually ran during a
./pants test --changed-since
run. Are there any best practices for using that coverage information in CI? I’d like to use
--changed-since
in our CI to help speed up test runs, but obviously only showing coverage of the files that have been changed doesn’t allow us to gate MRs on increasing overall coverage
b

bitter-ability-32190

10/13/2022, 2:09 PM
I think you'd need a server that "hosts" the overall coverage and is smart enough to be fed a subset of coverage (and carry the remaining metrics form the merge-base)
IIRC https://about.codecov.io/ has something like that
Ah this seems relevant, albeit wouldn't work with `--changed`: https://docs.codecov.com/docs/carryforward-flags
It's probably not prohibitively hard to whip something up, assuming you have a server to host the commit -> full coverage mapping. But I digress...
f

future-oxygen-10553

10/13/2022, 3:47 PM
Thanks @bitter-ability-32190! Unfortunately, the nature of our work is that we can’t integrate third party services like CodeCov although that’s a great suggestion in general. The last few comments in the thread here seem like they were meant for a different conversation, though 😄
b

bitter-ability-32190

10/13/2022, 4:03 PM
Oh ooooooops lol I'll delete those 😂
Alternatively, a remote caching server would presumably allow you to run
test ::
and see similar perf to
--changed
f

future-oxygen-10553

10/13/2022, 4:16 PM
Yep, unfortunately we’re working on defense related stuff so external services are generally hard to get approval for 🙂
b

bitter-ability-32190

10/13/2022, 4:24 PM
Ah yeah makes sense. Hosting a coverage metadata server is likely your option then if you absolutely need to use changed
f

future-oxygen-10553

10/13/2022, 4:27 PM
Test time isn’t a problem yet, but this is good to keep in mind for when it does become a problem
Thank you!
4 Views