IIUC, generating

pyi

files for protobufs (necessary for tools like

mypy

and

pyright

to work) doesn't have support for Python 3.12. Generating those

pyi

files happens via the

pants.toml

file:

[python-protobuf]
mypy_plugin = true

However, because Pants uses pip 23 instead of 24, and there doesn't seem to be a way to install this tool from a lockfile, it complains that pip 23 isn't compatible with Python 3.12. So, can anyone confirm that my understanding is correct, that it's either

pyi

generation or Python 3.12, but not both?

You can upgrade Pants to a later version of `pex`: https://www.pantsbuild.org/2.19/docs/python/overview/pex#setting-pex-and-pip-versions

Thanks for the reply, @happy-kitchen-89482! I've followed your advice, and added the following to the pants.toml:

[python]
interpreter_constraints = ["==3.12.*"]  # was there before
enable_resolves = true.  # was there before
pip_version = "24.0"

[pex-cli]
version = "v2.3.1"
known_versions=[
  "v2.3.1|macos_arm64|71690e672871b55323f5d6ef9a3fe9705f1668662652c4081080e7ab27d44de3|4124530"
]

However, it still is using pip 23. I'm guessing the problem is that it is pex that has to be given the argument of what pip to use, and I don't see a way to do that. Here's the exact error:

ProcessExecutionFailure: Process 'Building mypy_protobuf.pex from <resource://pants.backend.codegen.protobuf.python/mypy_protobuf.lock>' failed with exit code 1.
stdout:

stderr:
The Pip requested was pip==23.0.1 but it does not work with the interpreter selected which is CPython 3.12.3 at /opt/homebrew/Cellar/python@3.12/3.12.3/Frameworks/Python.framework/Versions/3.12/bin/python3.12. Pip 23.0.1 requires Python <3.12,>=3.7.

Did you set

pip_version = "latest"

?

Same thing - regardless of what I set the

pip_version

to, it is using

23.0.1

. Perhaps I'm making a mistake in setting in

[python]

and not somewhere else?

You should get warnings or errors if you have unrecognized config keys

Done, same thing. I tried setting it to something invalid, and indeed it makes a difference; pex complains. Based on what I'm seeing, while Pex might be getting the argument for pip 24, something else is then defaulting to 23.1, probably the running of the pex needed for the new mypy part.

So, a bug perhaps... Can you create a small repo that reproduces it, and file an issue with a link to said repo? That'll make it much easier to investigate. Thanks!

sure, will do, thanks for looking at it

I think this is likely to be https://github.com/pantsbuild/pants/issues/20354, which is fixed in 2.21... but that's not yet released. Assuming that's the case, you have a few options • use a 2.21.0.devX pre-release • generate your own lockfile for the

mypy_protobuf

tool, by creating a resolve for it, adding the relevant tool(s) to it with a

python_requirement

and pointing

[mypy-protobuf].install_from_resolve

to that

@broad-processor-92400 thanks - unfortunately there is no

[mypy-protobuf].install_from_resolve

, at least in the Pants version I'm using (

2.19.1

), maybe it was added later?

Sorry, I was a bit ambiguous/abbreviated, it's an option in `pants.toml`: https://www.pantsbuild.org/2.19/reference/subsystems/mypy-protobuf#install_from_resolve

@broad-processor-92400 thanks, that solved the problem! I am now starting to understand things a bit better, with all the different sections of the toml file.