#16044 Unable to guarantee consistent versions for (transitive) dependencies between tools (particularly mypy) and runtime resolves Issue created by huonw Describe the bug We've got a codebase that uses mypy extensively, and various plugins, and we'd prefer if we had good methods for having the versions of our mypy plugin libraries match the ones used at runtime. For instance:

# pyproject.toml
[tool.mypy]
plugins = ["pydantic.mypy", "sqlmypy", "strawberry.ext.mypy_plugin"]
...

Two of these come from libraries also used at runtime (

pydantic

and

strawberry-graphql

) and thus it'd be good to be using a consistent version globally: if we have mypy using plugins for different versions of our runtime libraries who knows if type checking passing is meaningful. There's a few options: • write

extra_requirements = ["pydantic==...", ...]

etc. but that duplicates the version configuration, and doesn't guarantee identical versions of transitive dependencies without listing them all out by hand. • use

source_plugins

pointing to the relevant

python_requirement

target (which isn't what it's documented to be used for), which at least deduplicates version configuration for those direct requirements, but also doesn't guarantee identical versions of transitive dependencies. The second is slightly better than the first, so that's what we've tried so far despite it seemingly being not a proper use of

source_plugins

. For instance:

[GLOBAL]
pants_version = "2.12.0"

backend_packages = [
  "pants.backend.python",
  "pants.backend.python.typecheck.mypy",
]

[python]
interpreter_constraints = ["CPython==3.9.*"]
enable_resolves = true

[mypy]
lockfile = "3rdparty/python/mypy.lock"
source_plugins = [
  "//:pydantic",
]

[anonymous-telemetry]
enabled = false

With top-level

BUILD

that simulates a transitive dependency

typing-extensions

that's stuck on an older version:

python_requirement(name="pydantic", requirements=["pydantic==1.9.0"])
python_requirement(name="typing-extensions", requirements=["typing-extensions==4.0.0"])

This mostly works, after running

./pants generate-lockfiles

, we have

mypy.lock

containing:

...
          "project_name": "pydantic",
          "version": "1.9"
...
          "project_name": "typing-extensions",
          "version": "4.3"
...
  "requirements": [
    "mypy==0.910",
    "pydantic==1.9.0"
  ],
...

However, note that

typing-extensions

is missing from the requirements, and indeed the pinned version of the transitive dependency of

pydantic

is different to the one in the runtime default resolve (which is just

"version": "4"

). Notes: • For this particular

typing-extensions

dependency, this probably doesn't matter so much, but other transitive dependencies of some plugins may make a difference if the version is significantly different (e.g. hypothetically a plugin that validates that a particular string literal with strict formatting requirements (e.g. "the arg is a valid regex" in a call like

re.compile("...")

), in addition to the basic runtime check: if the plugin uses a newer version that might support more features, mypy will incorrectly be successful). • This potentially relates to general issues about

./pants generate-lockfiles

always choosing the latest version (if there's been new releases), such as #12880, #15704 and #15723. • This also potentially relates to #12449. • Maybe we want to be saying that mypy can pull plugins from a particular resolve, but that's potentially weird (e.g. the docs for

source_plugins

call out using a separate resolve for deps of such plugins https://www.pantsbuild.org/docs/reference-mypy#section-source-plugins) As always, thanks for pants! Pants version 2.12.0 OS macOS Additional info Reproducing example: https://github.com/huonw/pants-tool-pinning pantsbuild/pants