(I'm a little unsure how to read GitHub action yam...
# general
c
(I'm a little unsure how to read GitHub action yaml.) from https://www.pantsbuild.org/docs/using-pants-in-ci
If you're not using a fine-grained remote caching service, then you may also want to preserve the local Pants cache at $HOME/.cache/pants/lmdb_store.
Does that mean with a REAPI cache there will not be a
lmdb_store
directory, but
pex_root
and friends will all still be under
named_caches
? Under what conditions would you want to use a REAPI cache and cache all of
lmdb_store
? (the "may" from the advice)
c
as I read it, is if your not using a REAPI cache you may still not want to cache that directory (as it is unbounded and could be huge)
b
There's still a local
lmdb_store
cache when using a remote cache, the remote cache is a second cache that can be used to fill the local cache when something isn't there... but if something has already been run locally or already downloaded from a remote cache, pants will be able to find/reuse the result from that local cache. Using a remote cache will be better than coarse-grained "tar a directory and upload it"-style caching of
lmdb_store
(e.g. what github actions does with its fully transient runners), because each job will only download what it needs... whereas lots of stuff in
lmdb_store
will become out of date and just be uselessly uploaded/downloaded. That is, if you're using a CI service like that, and using a remote cache, caching
lmdb_store
is likely wasted time. However, if you're using a CI service with persistent runners (i.e. separate jobs run on the same "physical" machine), preserving
lmdb_store
between runs is likely helpful, even with a remote cache: if something has been downloaded from the remote cache by one job, it'll be saved locally and won't need to be re-downloaded for a future job. But, this is just ensuring that directory isn't cleared out. That said, the "may" there is technically different: it's suggesting that if you're not using a remote cache, there's times when you might not want to preserve
lmdb_store
. I guess you might find doing fresh builds every time is faster than coarse grained caching of that whole large directory. So, in summary, the four options: • use remote cache, preserve `lmdb_store`: for persistent CI runners, where
lmdb_store
can be saved with no effort/cost • use remote cache, no preserve `lmdb_store`: transient CI runners, where
lmdb_store
would need to be archived and uploaded in bulk • no use remote cache, preserve `lmdb_store`: when a remote cache isn't possible (e.g. cost or time), and saving
lmdb_store
makes builds faster/cheaper • no use remote cache, no preserve
lmdb_store
: when a remote cache isn't possible (e.g. cost or time), and saving
lmdb_store
makes builds slower/more expensive
c
Thank you! All that context is very helpful.