I'm trying to switch over to pyright from mypy on a repository with remote cache enabled. That's failIng with the third party dependencies issue identfied elsewhere. Killing the remote cache fixed the issue, but it then re-occurred. My hunch is that having mypy and pyright running on the same cache is a problem. Mypy is running on main, and i'm loathe to merge until I have a fix. Is that hunch plausible and how might I refute it? Refuted, thanks Krishnan.

I have never used Mypy, only Pyright

Well, there goes that theory! Thanks 🙂

We were originally running MyPy but removed it entirely when migrating to Pyright, Nuking the cache didn’t work but removing the lmdb cache from the GitHub CI action seems to have resolved the issues (although it’s too soon to tell if ti will rear its head again)

Yeah, my current belief is that it’s caused by concurrent runs corrupting the cache - we tried forcing GitHub Actions runs to occur sequentially and that seems to have fixed the issue somewhat

Ewwww... Can you disable remote cache for a single invocation of pants?

A simpler thing might be to make the remote cache read-only (which should prevent the corruption). Here’s what’s in my `pants.ci.toml`:

remote_cache_read = true
remote_cache_write = false
remote_store_address = <address here>

Alternatively, within CI you can set the environment variable

PANTS_REMOTE_CACHE_WRITE=false

and that should accomplish the same thing

I might try running

pants --no-remote-cache-read check

and see if that works. The remote cache is super helpful for tests

Yeah this particular issue only seems to affect Pyright, and I’m still not entirely sure why. The read-only solution I point out above is rather ham-fisted, but does seem to work 🤷

I'm good with ham fisted. I am so, so tired of mypy

I agree with Bob's assessment, pyright is seemingly better, and it makes sense since it powers vs codes Python extension, so high stakes high rewards

I have a cache that's still broken, and presumably if I configure my local machine to use it and check out a commit, I'll have a reproduction, so I'm very happy to lend a hand here.

I think part of what's hard for these kinds of tools is that they're slow. So they implement their own daemons and caching, which we don't know about, and therefore haven't disabled/worked around

So, by inspection, this line assumes the venv PEX has been executed once: https://github.com/pantsbuild/pants/blob/062ec627afc474791a4fbdd3e60e12e8abae2624/src/python/pants/backend/python/typecheck/pyright/rules.py#L220 On a cache hit that will not be the case and the pex_root won't have been populated.

Hmm, interestingly the theory didn’t play out here as I expected. Here’s what I ran:

$ pants check ::
21:13:16.59 [INFO] Initializing scheduler...
21:13:19.51 [INFO] Scheduler initialized.
...
21:13:31.55 [INFO] Completed: Typecheck using Pyright - pyright - pyright succeeded.
Loading configuration file at /private/var/folders/0w/zhpm9fs12zzcmt3m5799xb9c0000gn/T/pants-sandbox-SKdiSz/pyrightconfig.json
Searching for source files
Found 197 source files
pyright 1.1.274
0 errors, 0 warnings, 0 informations
Completed in 3.851sec

stubPath /private/var/folders/0w/zhpm9fs12zzcmt3m5799xb9c0000gn/T/pants-sandbox-SKdiSz/typings is not a valid directory.

✓ pyright succeeded.

Then removed the cache:

$ rm -r ~/.cache/pants/named_caches/pex_root

Then ran the check again:

$ pants check ::
21:17:41.82 [INFO] Completed: Typecheck using Pyright - pyright - pyright succeeded.
Loading configuration file at /private/var/folders/0w/zhpm9fs12zzcmt3m5799xb9c0000gn/T/pants-sandbox-SKdiSz/pyrightconfig.json
Searching for source files
Found 197 source files
pyright 1.1.274
0 errors, 0 warnings, 0 informations
Completed in 3.851sec

stubPath /private/var/folders/0w/zhpm9fs12zzcmt3m5799xb9c0000gn/T/pants-sandbox-SKdiSz/typings is not a valid directory.


✓ pyright succeeded.

The second run seems to have benefited from caching on the first run, but Pyright still works just as quickly if I run both the check commands with

pants --no-local-cache

. So I’m not totally sure what exactly the problem is, or if I ran pants with the right arguments. I tried running pants with

pants --no-pantsd check ::

as well, and that seems to work too.

@better-van-82973 sorry - need to

rm -rf ~/.cache/pants/named_caches/pex_root && pkill pantsd

But after that, do I need to run the check command with

--no-pantsd

to ensure that the daemon doesn’t start again?

Hmm, I wonder if pyright has a cache we don't control. The code I pointed to certainly seems like a bug that fits this all just right.

I don’t see any mention of one in their documentation, but it’s certainly possible

Did you look?

~/.cache/*

on Linux and

~/Library/Caches/*

on Mac are well known / well used places to check. Ditto a

ps

to see if it starts a daemon.

It actually does have one -

$ ls ~/.cache/pyright-python/1.1.309/
node_modules      package-lock.json package.json

But unfortunately, this directory only seems to contain the NodeJS package for Pyright and not much else

Ok. Well this is definitely not near the 1st time we're suffering because we don't know the tool we're integrating.

Hey I managed to reproduce the issue! And at the end, I see this log output:

venv venvs/918c9178508a995b0dd4e20e20e83fd4186290da/1a5262eabfce46850648f9fd6fb6bab392a7a9f6 subdirectory not found in venv path /Users/krishnanchandra/.cache/pants/named_caches/pex_root.

Which certainly points to @enough-analyst-54434 theory being correct, as this venv path does not exist locally for me

Thanks. What was missing in the repro steps?

I’m honestly not sure tbh. Nor am I really sure how I managed to repro it just now - the one thing that I changed was running

pants check ::

from a subdirectory, rather than from the top level directory of the repo