With `pants.backend.awslambda.python` , is there a...
# general
b
With
pants.backend.awslambda.python
, is there a particular caching location that would speed the packaging operation up? Also, could I get some clarification on this wording? Is this referring to things like pants TOML files, or even the source code? I thought it was the latter, meaning caching it is a net negative since you would be generating and uploading a file constantly.
This has to be invalidated on any file that can affect any process, e.g.,
hashFiles('**/*')
on GitHub Actions.
I observed
~/.nce
never being used, and
~/.cache/pants/named_caches
causing lots of issues with that pip.log issue, while building pex files for testing. Something about a pip.log file not being found, which crashes the process.
~/.cache/pants/lmdb_cache
and
~./cache/pants/setup
by themselves seem to help though.
h
Well, that language is purposely cautious, because it's warning about the inherent trickiness of trying to use CI-level caches
because to do so you have to generate cache keys manually
And then you have to reason about what needs to be in those keys
and pretty much any file in your repo can affect
lmdb_cache
, yes
You may still get benefit from it, if your CI provider supports prefix-based stale cache restores
GitHub Actions does, for example
But that language is to warn you to measure, and not assume benefit
b
Using BitBucket (for now), and I’ve defined the LMDB cache key to be the pants TOMLs and requirements files. They expire every 7 days regardless, see how that goes
It speeds up, bit time, the building each pex before testing. Most directories do not change week-to-week, which unfortunately is difficult to cache that granularly
h
This is where remote caching is really valuable
b
Yeah that’s the dream! Unfortunately DX is in limbo because we are moving to GitHub, so I’m trying to speed things up with minimal effort I guess.
Proper caching server will be one of my lobbying points eventually for sure 😄