maybe stupid question but does `pants package` tak...
# general
b
maybe stupid question but does
pants package
take advantage of remote caching?
b
Usually yes, e.g. if
pants packages ::
builds a
pex_binary
target, that'll be doing an invocation of a
pex ...
process under the hood, and the outputs of this invocation can be served from a remote cache if available. Are you observing something unexpected?
b
i’m not entirely sure - i’ve just run
pants package ::
in two different gitlab CI jobs at this point, i do have a container of
buchgr/bazel-remote-cache
running and I have pants configured to use it in
pants.ci.toml
. the first CI job took 1m38s according to gitlab, but the second took 2m - but oddly the first job metrics for
remote_cache_total_time_saved_ms
is
491094
while the second is
521192
so i’m not sure if this is a gitlab runner thing or a remote cache thing
other “odd” thing was that when i was watching the job in the gitlab UI, the second one seemed to go faster than the first
i don’t necessarily think there’s an issue yet but i was more or less just curious if the
package
goal also made use of remote caching
b
I guess
package
likely generally has larger artefacts than goals like
test
, so the download might take noticeably longer too?
b
yeah - though ultimately i don’t think i’m asking the right question; what i’m really after is generating artifacts from
pants package
on code (lambdas) that have actually changed - the “using Pants in CI” docs point to using remote caching and running goals “over everything” but in this case I would have to assume that no matter what,
pants package ::
would always spit out every package, even if it has to pull it from the cache
it smells like i’d actually want to use
--changed-since
for the
package
goal
yeah seems like no matter what my gitlab CI is uploading every single lambda zip as artifacts when i use
pants package ::
in my CI yaml even if i’ve only changed the code in a single lambda
h
So
pants package ::
will place all the requested artifacts in
dist/
, so it does do work every time. The ingredients of that work can be cached, but it still has to download them and write them out, and that can be non-trivial if large.
So yeah, sounds like
--changed-since=<gitref> --changed-dependents=transitive
is what you want
🥳 1
b
thanks @happy-kitchen-89482!
i’ll play around with that
h
Basically the end result of
pants package
is a side effect (materializing files to local disk) so it must run every time
Even if the content of those files comes from cache