Hi folks, I'm getting mypy errors when using proto...
# general
Hi folks, I'm getting mypy errors when using protobufs. Anyone have any suggestions? I've enabled the mypy-plugin in the [python-protobuf] section, but still get:
Copy code
./pants check ::
09:37:48.34 [INFO] Completed: Building requirements.pex with 5 requirements: bitarray==2.5.1, numpy==1.23.4, protobuf==4.21.12, simpy==4.0.1, types-protobuf==
09:37:48.75 [INFO] Completed: Building requirements_venv.pex
09:37:49.18 [ERROR] Completed: Typecheck using MyPy - mypy failed (exit code 1).
fabric/proto/v1/workload_pb2.pyi:6: error: Library stubs not installed for "google.protobuf.descriptor" (or incompatible with Python 3.7)
fabric/proto/v1/workload_pb2.pyi:6: note: Hint: "python3 -m pip install types-protobuf"
fabric/proto/v1/workload_pb2.pyi:6: note: (or run "mypy --install-types" to install all missing stub packages)
As you can see, I tried explicitly adding the types-protobuf dependency. I also tried the recommended --install-types in mypy args, but it doesn't have an auto-yes feature so can't progress. Trying to pass --exclude to mypy args doesn't work when you're in file-list mode (which is what pants seems to do)
So you're saying we can safely ignore the Python 3.7 warning there? You know that's not it for unspecified reasons; so it must be something else?
actually that's weird, I'm not sure where the 3.7 is coming from. My python is 3.9
I have no clue whether the warning is germaine or not, it just seemed like an obvious hint you weren't mentioning; so wanted to be sure it had been investigated.
I tried passing
--python-version 3.9
to mypy, didn't help
but it still seems like mypy is potentially not running in the right environment, if it thinks it's python 3.7 by default?
If you run with
there will be a line printed near the mypy failure that reveals a temporary directory you can examine. That will contain a `__run.sh`script that can be used to emulate and inspect what happened. See: https://www.pantsbuild.org/docs/rules-api-tips#debugging-look-inside-the-chroot That may be the best path forward in your debugging quest.
The more relevant link is probably https://www.pantsbuild.org/docs/troubleshooting#debug-tip-inspect-the-sandbox-with---keep-sandboxes - you're not hacking on rules here, but the concept applies generally for rooting out mystery in Pants actions.
I think I went through something similar. Ended up adding
in [mypy].extra_type_stubs in
(along with one specific lockfile for mypy and one for the extras). I don’t have the error anymore.
Along with what John suggested, you can add
in args, so Mypy gives more details on errors
I'm not seeing anything different/interesting yet. I made a small repo for repro: https://github.com/jonas25007/pants-proto-test
./pants check ::
(adding extra_type_stubs is basically equivalent to what I was trying)
It is not!
There are complicated details behind the scenes.
ah ok, well I tried it anyway and same result
Let me know if you have a chance to confirm repro, want to rule out something specific to my system
Is your system dockerable? Then you yourself can root that out! Will I need to install proto binaries to try your example repo?
that's true, I'll try in docker. doesn't pants manage the proto deps?
I’ve been able to reproduce with your repo.
./pants export-codegen
writes transpiled python classes into
, is there any way to change that? That would probably solve the problem, or at least, help a lot.
I realized that I was trying in docker as well through my CI (an image based on continuumio/miniconda3:latest). Not sure what
would have to do with it - check is finding the .pyi's (like it's supposed to?), the stubs just aren't getting installed
@average-sugar-68948 I believe the issue comes from Mypy, that introduced a breaking change a few versions ago, and I thought it was reverted… Anyway, adding an arg in mypy config should help:
Copy code
args = ["--namespace-packages"]
(in pants.toml)
Yes!! this works for me, thanks so much for hunting it down
🙏 1
While I have you here and the test repo, I noticed that proto dependencies don't seem to resolve for the
goal. For example,
./pants run run.py
in that test repo (pull for update) results in an import error, even though
dependencies --transitive
lists the proto target properly. The
goal doesn't seem to have this problem (
./pants test tests/
). Am I doing something wrong?
I guess there is an interference created by the
file within
folder. As of right now, it’s unclear to me why test works and run does not, I would have expected none to work. I need to dig more in order to give you a satisfying answer, but you can just remove the
file from proto and try again 🙂
sorry just catching up from holidays - yes thanks, removing
does the trick