Cache (build everything but share cache cross build) vs Incremental Build (only build based on git change), which one is more favorable in case of monorepo? Related question, many of the intermediate result (e.g. pyprep, source, requirements) are not in the

.cache

folder (they can be found in

.pants.d

folder), hence even with remote cache setup, those step would still be re-evaluated. Is this by design?

pants will mirror things from the remote cache into

.pants.d

I found 3 folders, without modifying any cache related config:

~/.cache
~/myproject/.pants.d
~/myproject/.cache

I understand

~/.cache

is global, but not sure about

~/myproject/.cache

i don't remember what

~/myproject/.cache

is for

Also by remote cache, I am referring to this config https://www.pantsbuild.org/setup_repo.html#build-cache

[cache]
read_from = ['<https://myserver.co/pantscache>']
write_to = ['<https://myserver.co/pantscache>']

I assume this determine where to write the content which by default goes to

~/myproject/.cache

i believe it should actually pull things to versioned directories in

.pants.d

at least for v1 (sorry I am trying to pull my repo and see what I got in the cache folder, did pants clean-all recently)

yes yes

.pants.d (I think I might have some v2 content there as well)

zxiong@zxiong-dt-02:~/cruise/ark$ ll .pants.d/
total 128
drwx------  21 zxiong zxiong  4096 Apr 18 22:56 ./
drwxr-xr-x  23 zxiong zxiong  4096 Apr 21 14:22 ../
drwxr-xr-x   3 zxiong zxiong  4096 Apr 18 14:50 binary/
drwxr-xr-x   3 zxiong zxiong  4096 Apr 18 14:50 build_invalidator/
drwxr-xr-x   3 zxiong zxiong  4096 Apr 18 15:02 bundle/
drwxr-xr-x   3 zxiong zxiong  4096 Apr 18 14:50 compile/
drwxr-xr-x   3 zxiong zxiong  4096 Apr 18 15:02 docker/
drwxr-xr-x   3 zxiong zxiong  4096 Apr 18 15:27 gen/
drwxr-xr-x   3 zxiong zxiong  4096 Apr 18 15:02 go/
drwxr-xr-x   3 zxiong zxiong  4096 Apr 18 14:50 link/
drwxr-xr-x   4 zxiong zxiong  4096 Apr 18 14:50 native-compile/
drwxr-xr-x   2 zxiong zxiong 12288 Apr 21 14:21 .pids/
drwxr-xr-x   5 zxiong zxiong  4096 Apr 18 14:50 pyprep/
drwxr-xr-x   4 zxiong zxiong  4096 Apr 18 14:50 python-setup/
drwxrwxr-x   3 zxiong zxiong  4096 Apr 18 22:56 repl/
drwxr-xr-x 147 zxiong zxiong 20480 Apr 21 14:21 reports/
drwxr-xr-x   3 zxiong zxiong  4096 Apr 18 15:01 resolve/
drwxr-xr-x 147 zxiong zxiong 20480 Apr 21 14:21 run-tracker/
drwxr-xr-x   4 zxiong zxiong  4096 Apr 18 14:54 test/
drwx------   2 zxiong zxiong  4096 Apr 18 22:50 tmp13co_3j9/
drwxr-xr-x   3 zxiong zxiong  4096 Apr 18 14:50 unpack-wheels/

ok yes perfect,

.cache/

is fingerprinted

yes

we have it configured a little differently at twitter because we have it used a shared buildcache to support multiple checkouts of our monorepo

.pants.d

too fingerprinted but it is deeper in the folder structure (I think)

yes that's current

yes, read the fine print

please let me know if i can clear anything else up

I was trying to setup a simple remote cache by pointing my

cache.write_to

and

cache.read_from

to a remote web storage. Given that the

.pants.d

content are not persisted and pulled cross build, now I wonder if worth spending time on this (another reason is we are trying pants v2 as well, which may not be done over night neither…)

.cache

is persisted across builds, though!

.cache

 is persisted across builds, though!

it is

i would also love to align with any timelines you may have regarding pants v2 support for any partixular workflow

There is discussion in #gettingstartedv2 and #cruise-folks

ok wonderful

Generally, I am working on it but at meantime keep getting distracted (a different discussion between me an my manager I guess 😉 ) I identified several gaps and @hundreds-father-404 and @happy-kitchen-89482 had been helping me a lot. That’s said, the timeline depends on how fast we can get all features (we currently use in v1) to v2, and how I can align my time

yes absolutely

To clarify, those caching instructions and

.pants.d

are v1 things. Local caching is baked deeply into the design of v2. You don't have to set anything up to get it.

got it