Hey all. :wave: In my team's monorepo we support P...
# general
b
Hey all. 👋 In my team's monorepo we support Python 3.9 through 3.12 (
>=3.9,<3.13
). In order to speed up local development, we would like to just test/check with 3.12 and let CI/CD cover the other versions. Is there a way to do that? I tried setting a
.pants.rc
file with
interpreter_constraints = ["==3.12.*"]
but then I just get an error when running tests like this:
Copy code
InvalidFieldException: The target tests/cli/test_cli.py:tests@interpreter_constraints=py310 has the `interpreter_constraints` ('==3.10.*',), which are not a subset of the `interpreter_constraints` of some of its dependencies
b
Another way to do this might be to set your interpreter constraints to
==3.12.*
in your
pants.toml
, but override it to
>=3.9,<3.13
in
pants.ci.toml
. Does that work?
b
When I do that I see the same error. This is what I have defined on my test:
Copy code
python_tests(
    name = "tests",
    interpreter_constraints = parametrize(
        py310 = ["==3.10.*"],
        py311 = ["==3.11.*"],
        py312 = ["==3.12.*"],
        py39 = ["==3.9.*"],
    ),
)
It seems like the issue is that I am saying in pants config that I am only using 3.12 but then the test is asserting it should run on 3.9 through 3.12.
c
We do something like
Copy code
**parametrize(
                "py311",
                interpreter_constraints=["==3.11.*"],
                tags=["py311", "unit", "unit-test"],
And then you could -- I think -- locally always do:
--tag=py311
to only run those tests
b
Ah interesting let me give that a try
This wouldn't help though when running mypy right? One of the problems I am trying to solve is that mypy takes a long time in pants so it would be nice to limit it to just 3.12 too. This does help with tests though.
Thanks @curved-manchester-66006 this worked great for tests! I'm hoping someone has a suggestion for mypy. The mypy check takes a long time.
c
The mypy check takes a long time.
Doe
pants --tag=py311 check foo::
not work the same?
b
Oh thanks for the suggestion. So I tried adding the parameterize logic from above to my python sources, so I could use the tag, but now I get a bunch of these:
Copy code
16:25:29.25 [WARN] The target path/to/file.py@parametrize=py311 imports `some.other.path.Class`, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use: ['some/other/path.py@parametrize=py310', 'some/other/path.py@parametrize=py311', 'some/other/path.py@parametrize=py312', 'some/other/path.py@parametrize=py39'].
So it seems like it doesn't understand the relationship between sources that
py39
should related to
py39
in the dependency. My sources currently don't define an interpreter-constraint at all and just use the project default of
">=3.9,<3.13"
.
c
oh I see, sorry to send you down that wrong turn.. Backing up, if the constraint is
">=3.9,<3.13"
and locally everyone has just 3.12, what does
pants check
do that isn't what you would like?
b
Well everyone locally has 3.9 through 3.12. When I do check I get a report of the different partitions passing. See example here:
Copy code
16:44:05.88 [INFO] Completed: Typecheck using MyPy - mypy - mypy succeeded.
Partition #1 - python-default, ['CPython<3.13,>=3.9']:
Success: no issues found in 261 source files

Partition #2 - python-default, ['CPython==3.12.*']:
Success: no issues found in 74 source files

Partition #3 - python-default, ['CPython==3.10.*']:
Success: no issues found in 25 source files

Partition #4 - python-default, ['CPython==3.11.*']:
Success: no issues found in 25 source files

Partition #5 - python-default, ['CPython==3.9.*']:
Success: no issues found in 25 source files
I guess partitions 3-5 are from python test files that mypy is checking. If I pass in
--tag=py312
then I get this:
Copy code
15:22:26.60 [INFO] Completed: Typecheck using MyPy - mypy - mypy succeeded.
Success: no issues found in 59 source files
Which makes sense in that it is only checking test files. So I can't seem to find a way to say "check all files with mypy with only py312". That being said, I'm not sure how much time I would be saving if I did make pants only process 3.12 since it seems like the overhead of pex + having to set
--no-incremental
for mypy (bug here: https://github.com/pantsbuild/pants/issues/18519) is causing most of extra time.