https://pantsbuild.org/ logo
e

eager-dress-66405

12/05/2022, 11:10 PM
A large (and knotted) portion of my monorepo is a django project. We would like start using pants to do mypy type checking on it and use mypy_django_plugin. The problem is mypy config plugin installation applies everywhere and installing this plugin requires that we add our entire django project as a mypy dependency, even when checking a file that is not part of the django code. What I need is to have different mypy configs (or a layer that modifies one) and
extra_requirements
for different segments of the codebase. Is this something pants might want to support natively? Any advise on what a plugin for it might look like?
• Having the django plugin is not absolutely required to run mypy on this code, but it makes the user experience significantly better and would likely speed up fixing all the current errors. • Mypy doesn’t have support for only enabling a plugin for a subset files/roots. There is only one configuration file (
$ROOT/mypy.ini
) and it doesn’t support hierarchical config like pytest’s
conftest.py
. • To enable the django plugin in mypy you need to make the project settings (
django_projects/test/settings.py
) and all transitive deps a
mypy
tool resolve dependency. The plugin throws an error if the project can’t be
django.setup()
.
b

bitter-ability-32190

12/05/2022, 11:15 PM
Hmm we actually could make something like this work, behind flag. If we (optionally) partitioned mypy by config we could do this Cc @sparse-lifeguard-95737 @hundreds-father-404
e

eager-dress-66405

12/05/2022, 11:16 PM
Ya I think similar partitioning to what Dan added for pytest would work, with the addition of keying on a config file. Allowing them to group and share init time the same way would be even better. It's the same motivating problem (django setup).
b

bitter-ability-32190

12/05/2022, 11:17 PM
We already partition by config in another plugin, so all the pieces are there
And this could be useful for any tool not supporting hierarchical configs
1
e

eager-dress-66405

12/05/2022, 11:19 PM
Would I be able to take advantage of those in a plugin, or is this something best done in pants (assuming you want it eventually)?
b

bitter-ability-32190

12/05/2022, 11:19 PM
Both?
s

sparse-lifeguard-95737

12/05/2022, 11:21 PM
I think Michael means, is the plumbing for
check
already generic enough that we could roll our own custom mypy plugin with the batching logic?
Or are there hard-codes things in the core code that would block us?
b

bitter-ability-32190

12/05/2022, 11:21 PM
Ah nope
Check doesn't partition today
Oh but if you roll your own mypy plugin you can do it
s

sparse-lifeguard-95737

12/05/2022, 11:22 PM
🤔 yeah because it already has some built in partitioning by resolve, right?
b

bitter-ability-32190

12/05/2022, 11:23 PM
By IC/resolve
👍 1
s

sparse-lifeguard-95737

12/05/2022, 11:23 PM
So there must be enough of a skeleton in the core check logic to support partitioning of some sort when appropriate
b

bitter-ability-32190

12/05/2022, 11:24 PM
Well not technically. Any rule can invoke any number of other rules as many times as it wants 😌
s

sparse-lifeguard-95737

12/05/2022, 11:24 PM
That is true lol
b

bitter-ability-32190

12/05/2022, 11:25 PM
But I think Check does ask for N Check results from the plugin implementors
👍 1
h

hundreds-father-404

12/05/2022, 11:25 PM
But I think Check does ask for N Check results from the plugin implementors
yes
s

sparse-lifeguard-95737

12/05/2022, 11:25 PM
But there’s no hard rule about what N actually maps to? (Unlike
test
which was pretty locked into 1 result per target)
b

bitter-ability-32190

12/05/2022, 11:26 PM
I think we should just partition check and call it a day
e

eager-dress-66405

12/05/2022, 11:42 PM
Which is the other plugin that partitions by config?
b

bitter-ability-32190

12/05/2022, 11:44 PM
Scalafmt I think
e

eager-dress-66405

12/05/2022, 11:47 PM
I sounds like this is something pants might want to support generally? Should I file an issue on gh?
b

bitter-ability-32190

12/05/2022, 11:49 PM
Please do. A generic "check should partition" as well as one for maybe partitioning more tools by config. We can inventory plugins for the latter
e

eager-dress-66405

12/05/2022, 11:56 PM
b

bitter-ability-32190

12/05/2022, 11:57 PM
he's me
😂 1
e

eager-dress-66405

12/06/2022, 12:01 AM
How does a "check should partition by config" issue linked to https://github.com/pantsbuild/pants/issues/17328 sound?
b

bitter-ability-32190

12/06/2022, 12:03 AM
Different issue, but is blocked by that one
And honestly, it isn't just check, fmt/lint/fix could benefit too
I like this a lot! I'm glad you posted this @eager-dress-66405
e

eager-dress-66405

12/07/2022, 4:22 PM
7 Views