When running `check` on python code with mypy interpreter co

Question

When running `check` on python code with mypy interpreter constraints is set we don t actually use those constraints on targets that have their own ICs

happy-kitchen-89482 · Answer

So there is no way to say run mypy using Python 3 9 on code that may be compatible with earlier versions

happy-kitchen-89482 · Answer

I m wondering if setting ` mypy interpreter constraints` explicitly should just bypass the partitioning and all that and just do the simple and obvious thing that is implied by that

happy-kitchen-89482 · Answer

Or rather presumably there is a reason it doesn t do that but it s not clear what it is and right now it looks like we simply ignore a config setting

enough-analyst-54434 · Answer

As a datapoint I find useful over in Pex I run mypy with an effectively arbitrary interpreter that s compatible with mypy of course but I run it over most python versions my code supports I pick 4 versions not all using `mypy python version X Y` In the Pex case this would be like using the Pants mypy ICs to create the right MyPy PEX tool and then running the tool over ~each Python major minor implied by the code being checked ICs And in the Pex case this is important since Pex works with 2 7 which has different type checking concerns than 3 x namely around unicode vs str I m not sure if that generalizes to differences in 3 x version or not and I m pretty darn sure its not performant to just auto cover the IC range like that

enough-analyst-54434 · Answer

For example here in CI <https github com pantsbuild pex runs 6483354481 check suite focus=true step 4 112>

enough-analyst-54434 · Answer

But even if running over the range isn t performant the Pex example points to a fail safe algorithm Use the MyPy ICs to build the MyPy PEX tool pick the minimum Python major minor version from the code under check ICs and run MyPy once with that ` python version`

hundreds-father-404 · Answer

< happy kitchen 89482> Pants lets you explicitly set `python version` to force a version

enough-analyst-54434 · Answer

So Benjy is observing a big then If the code under check is Python 2 7 code that code s IC will be used for the MyPy PEX tool which will fail since MyPy should be run using some Python 3 but told ` python version 2 7`

hundreds-father-404 · Answer

I vaguely remember them changing something about running w Py2 recently but when this code was written not so thanks to `typed ast` MyPy could understand prior Python versions than the interpreter it was run with

enough-analyst-54434 · Answer

It does as Pex use shows I use a Python 3 10 MyPy venv to run against ` python version 2 7 3 5 10 11 `

enough-analyst-54434 · Answer

This is at least since MyPy 0 800 Feb 2021 <https github com pantsbuild pex pull 1233|https github com pantsbuild pex pull 1233>

happy-kitchen-89482 · Answer

Well `python version` is something different it s the version targeted by mypy not the version we run mypy on

happy-kitchen-89482 · Answer

But that s not what we partition on

happy-kitchen-89482 · Answer

we partition on ICs and resolves and then try to set ` python version` dynamically in each partition unless it s already set in mypy ini or args

happy-kitchen-89482 · Answer

But I ve set ` mypy interpreter constraints` explicitly to Python 3 9 yet I still get partitioning and one of the partitions runs mypy on 3 6 because that code has permissive constraints

happy-kitchen-89482 · Answer

Even though it would be fine and in fact required in this case to run on 3 9 everywhere in a single partition

happy-kitchen-89482 · Answer

I know I keep coming back to this but this is a manifestation of the complexity with treating ICs as two different things 1 descriptive metadata about the compatibility of some py files and 2 imperative config for running a tool on that code

happy-kitchen-89482 · Answer

BTW since I set ` mypy interpreter constraints` explicitly the tool lockfile is generated for those constraints so running it on other constraints might not even work

happy-kitchen-89482 · Answer

So it seems like if you set ` mypy interpreter constraints` explicitly we should always run with that but still partitioning if we re setting `python version` dynamically and if you set `python version` explicitly in `mypy ini` then we shouldn t partition at all

happy-kitchen-89482 · Answer

And we probably shouldn t warn about the explicit python version in `mypy ini` unless there s a conflict It s weird to warn about normal use of a tool config file

happy-kitchen-89482 · Answer

Although actually even using ` mypy interpreter constraints` isn t right those are constraints for generating a lockfile and we may want a lockfile compatible with multiple versions of mypy in this case because we need to run mypyc at package time on several different interpreters

happy-kitchen-89482 · Answer

But we would still like a way to say when running mypy as a tool do it on this interpreter version

enough-analyst-54434 · Answer

gt Well `python version` is something different it s the version targeted by mypy not the version we run mypy on Absolutely agreed We need just 2 things 1 A valid Python to run MyPy with this need have 0 to do with what Python your code under check needs to run 2 python version to say which interpreter to run the check using The partitioning and other complications are just that I have 0 clues what that is what we do there but the above 2 facts need to be the basis

hundreds-father-404 · Answer

Yeah good summary John ` mypy interpreter constraints` is meant to be 1 Partitioning + the allowance for `python version` is meant to be 2

happy-kitchen-89482 · Answer

So I think right now those two get more mingled up than we would like

happy-kitchen-89482 · Answer

In that we often ignore ` mypy interpreter constraints` and use the code s constraints instead

happy-kitchen-89482 · Answer

For 1

hundreds-father-404 · Answer

We only do that for 1 because of the typed ast limitation If you run MyPy with Py37 it cannot understand Py38+ code Same story w Black Note we only ignore the option if it was not explicitly st

happy-kitchen-89482 · Answer

But if the user has explicitly said run on this version then we should probably do that

hundreds-father-404 · Answer

If they explicitly set it then no matter what we use the option

happy-kitchen-89482 · Answer

As far as I can tell that is not the case

hundreds-father-404 · Answer

hundreds-father-404 · Answer

happy-kitchen-89482 · Answer

Hmm so I m not sure what I m seeing

hundreds-father-404 · Answer

To make clear the intent w 1 is that users don t need to worry about it as it s ~implementation detail Pants Just Does The Right Thing Modeling for 2 could probably be improved tho Feel free to re envision the partitioning

happy-kitchen-89482 · Answer

The only thing with 2 is not to warn about not being able to set that explicitly unless there s a conflict perhaps

happy-kitchen-89482 · Answer

If you have an existing mypy ini that sets python version it s odd to get a warning

happy-kitchen-89482 · Answer

The partitioning I m getting is correct just possibly not necessary

happy-kitchen-89482 · Answer

Since in the end both partitions are running with the same interpreter the same python version and the same resolve

happy-kitchen-89482 · Answer

But I don t think that s a big deal