Question about caching: Why mypy cache is under na...
# general
b
Question about caching: Why mypy cache is under named caches and not the lmdb cache? Intended behaviour? I wanted to leverage the remote cache so we can get faster typechecks.
b
There's two layers of caching in Pants: • LMDB caches exact matches: re-invoking a process with the same files, args and env vars => serve from cache, don't actually execute it. • Named caches are designed to help with partial matches: invoking a process with (slightly) different files, but still able to benefit from previous work. Pants' Mypy uses both: • if you re-run on exactly the same files, mypy isn't even invoked • if you make a change and rerun, it's not an exact match, but mypy might still benefit from its work from a previous run, which pants tells it to store as a "named cache"
b
OK. Problem is that the mypy cache keeps growing like crazy. Is there any tips to optimize? Or any way to send the lmdb cache to the remote caching service?
b
(The contents of lmdb cache is what is cached by a remote cache, I’m going to guess you mean a named cache.) Not sure about optimising. The only general one I can think of is to preserve it usually but regularly clear it when it is too large, so most builds benefit but not all: https://www.pantsbuild.org/2.21/docs/using-pants/using-pants-in-ci “nuking the cache when too big” You could investigate mypy-specific optimisations, eg understand what is in the mypy cache and work out what can be deleted (etc.). This’ll be a balance of pants and mypy behaviour, but mostly mypy I suspect. Re named caches -> remote cache, I don’t think there’s a current possibility for that.
👍 1