Is there a way to configure mypy to build per package, instead of globally? Similar to how unit tests work?

can you elaborate a bit..? you can run mypy over a subset of your sources.. but isn't that you want?

If I run

pants check ::

it tries to build a single env for all my packages (seemingly)

Ah.. as far as I know, there is a single env for mypy to execute in, but I'm not sure how it works with pulling in the application 3rd party dependencies for typing purposes..

Yea, it seems to build a single env, which is tough when you have 300+ python packages in a repo, usually with conflicting deps 😅

If you have conflicting deps, you will probably need to use different resolves for that

Right. I guess my comment is,

pants test ::

already works and figures this out. But seems like more extensive work is needed to make this work for mypy

idk, mypy in our repo does the partitioning by resolve and interpreter_constraints out of the box

Partition #1 - auctions, ['CPython==3.11.*']:
Success: no issues found in 15 source files

Partition #2 - auctions, ['CPython==3.11.*', 'CPython==3.12.*']:
Success: no issues found in 917 source files

Partition #3 - black, ['CPython==3.11.*']:
Success: no issues found in 1 source file

Partition #4 - buildkite, ['CPython==3.11.*', 'CPython==3.12.*']:
Success: no issues found in 2 source files

Partition #5 - default, ['CPython==3.11.*']:
Success: no issues found in 568 source files

Partition #6 - default, ['CPython==3.12.*']:
Success: no issues found in 26 source files

Partition #7 - default, ['CPython==3.11.*', 'CPython==3.12.*']:
Success: no issues found in 946 source files

Interesting... I'll have to fiddle around with this more.

are you sure you have resolves defined for python_sources? not just in pants.toml

So the thing is, I have 400+ python packages. Each package has • its own pyproject.toml (some of these will have conflicting python package dependencies) • sometimes interpreter constraints, if the global constraint is invalid And

pants test ::

just kind of works with this setup, builds an env for each package, and then runs tests successfully If I need to define resolves for 400+ packages in order to use

pants check

... I'm probably not going to get that done anytime soon lol.

I don't see resolves enabled, so you're using a single resolve

Oh wow, thanks for taking a stab at this!! Thats a ton of work 😅

Maybe there are ways to make it work, but I've run out of ideas

I really appreciate the help! I know I'm supposed to be using lockfiles to begin with, so it's good to know how to get even that far 🙏

I've created an issue about the high memory consumption https://github.com/pantsbuild/pants/issues/20568

Yikes, thanks for the issue, that will require some urgent examination

Did anything move forward on this?

AFAIK nope