Hi, we’ve found out recently that touching a BUILD...
# general
f
Hi, we’ve found out recently that touching a BUILD file essentially invalidates all the targets defined in the BUILD file, even the ones that had not been changed in any way. This behavior leads us to run a lot more tests than strictly necessary when we compute the changes of a diff / PR (with
--changed-parent
and
--changed-include-dependees
). Is there a way to compute precisely the targets that have been impacted by a change?
w
hey @fierce-plastic-71129! yes, the
--changed
flags will report those targets as potentially changed.
👍 1
🙏 1
but test caching will not: so if you are using remote caching and/or local caching, you should get cache hits for unaffected processes
if you’re using only local caching, make sure that you’re preserving the cache directories: https://www.pantsbuild.org/docs/using-pants-in-ci#directories-to-cache
h
How often are y'all changing BUILD files? With dependency inference, our hope is that it's rare you need to make a change. But it's plausible folks are changing more than anticipated Another thing that can help to minimize the impact is to use one BUILD file per directory, which is now automated with
./pants tailor
. See https://www.pantsbuild.org/docs/targets#target-granularity-for-first-party-code for the guidance there (to be clear, not required to do)
f
I see. We’re still on an old version of pants (
1.11
) so we’re not using dependency inference. For local tests, it’s indeed not much of a problem since those are cached. But on a build server, you can’t always assume that the test cache will be warm. More generally speaking, I think there is a need to know exactly what targets have changed as a result of a change. For example, we have a custom target type for a service and we want to detect exactly what services need to be redeployed after a change is landed. We were hoping to implement this logic using the pants but due to the false positive, it looks we’re going to have to re-implement all of this ourselves.
w
ah… yea, 1.11 is quite old
For example, we have a custom target type for a service and we want to detect exactly what services need to be redeployed after a change is landed.
one way to approach this is to use the digest of the output binary to skip work. that still relies on caching though.
in general: pants is a monadic build system, and so some amount of user logic needs to run (and some cache hits might be necessary) in order to determine precisely whether an output has changed.
there are plans to make this easier over time, and to better align it with the
--changed
flags: https://github.com/pantsbuild/pants/issues/11206 … but we’re unlikely to be able to start that until later this year.
f
Sounds good. Thanks!
h
@fierce-plastic-71129 would y'all be interested in upgrading to Pants 2.x? It generally is a much more enjoyable and often more performant experience than 1.10 is giving you. For this particular question, dependency inference should substantially cut down on editing BUILD files - it removes about 90% of BUILD file content usually
f
As far as I understand, it would be a rather large migration because use many languages (python, java, custom targets for js, webpack, custom targets for go, and more) and we have over 10K targets. I’m not familiar with the new dependency inference system but we have rather large mono-repo so we would need the inference to be performant and precise.
w
there is more information about dependency inference here: https://blog.pantsbuild.org/dependency-inference/ … BUILD files get smaller, but also: dependency information gets much more accurate.
the 90% smaller BUILD files come with 30% fewer dependencies overall… and those are validated as being correct because v2 runs processes in hermetic sandboxes
but yea… v2 doesn’t support the JVM quite yet: we’ve just kicked off a project to add it last week.