How does pants' caching integrate (or not) with mypy caching? Pre pants: I had a mypy script where I would run

mypy . --always-true FEATURE_FLAG

and then

mypy . --always-false FEATURE_FLAG

to typecheck both variants of my code. This would cause a full run of mypy each time since it would invalidate the mypy cache. I changed it to

mypy . --always-true FEATURE_FLAG --cache-dir .mypy_cache.flag_on

(and

.flag_off

). This made mypy do fast incremental runs since it could preserve its cache correctly. With pants: I (naiively) expected I could just run

pants check --mypy-args=["--always-true FEATURE_FLAG"] ::

and

pants check --mypy-args=["--always-false FEATURE_FLAG"] ::

and as expected this works to deliver a fast (pants cached) success. However, I get a full mypy run (roughly 1 minute for each feature flag variant) if any python files have changed. ie. The mypy cache is not being preserved. I added back the

--cache-dir

argument to mypy (as

--mypy-args

) but it still looks like I'm getting the slow mypy runs and not getting the incremental caching. Is there something I need to do to get pants to preserve mypy caches when I'm using non-standard cache names?

Is your cache dir an absolute path? You should make it, since pants runs mypy in a new sandbox every time

Late response, but I've tried using an absolute path here with no luck, the cache file does not get written out.

It looks like there's no current way to customise as you desire: https://github.com/pantsbuild/pants/blob/0392d378d6b9c6c8cecd17ac5bffa0698b5eb454/src/python/pants/backend/python/typecheck/mypy/rules.py#L76-L105 The persisted cache dir is computed with: https://github.com/pantsbuild/pants/blob/0392d378d6b9c6c8cecd17ac5bffa0698b5eb454/src/python/pants/backend/python/typecheck/mypy/rules.py#L224 So, maybe a path forward here would be to make the

{sha256(build_root.path.encode()).hexdigest()}

part of that second link customisable, for instance, a new

cache_path_disambiguator

option on https://github.com/pantsbuild/pants/blob/0392d378d6b9c6c8cecd17ac5bffa0698b5eb454/src/python/pants/backend/python/typecheck/mypy/subsystem.py#L[…]20 with default value

default=lambda _: pants.base.build_environment.get_buildroot()

(This would also solve https://github.com/pantsbuild/pants/issues/18830) Then, for your usecase, you could do

pants check --mypy-args=[...] --mypy-cache-disambiguator='%(buildroot)s:flag_...'

(and/or create an alias.) Are you interested in contributing? 😄

I am if time budget allows haha. In the end I settled with only running a single variant of my FEATURE_FLAG on local machines, and just hardcoding the mypy arguments in pants.toml. Since there is only one variant, the caching works "normally". Then I run all the variants (2 feature flags so 4 versions) in CI, where I would want uncached runs anyways (for now, at least)