https://pantsbuild.org/ logo
Join Slack
Powered by
A large (and knotted) portion of my monorepo is a ...
# general
e
A large (and knotted) portion of my monorepo is a django project. We would like start using pants to do mypy type checking on it and use mypy_django_plugin. The problem is mypy config plugin installation applies everywhere and installing this plugin requires that we add our entire django project as a mypy dependency, even when checking a file that is not part of the django code. What I need is to have different mypy configs (or a layer that modifies one) and 
extra_requirements
for different segments of the codebase. Is this something pants might want to support natively? Any advise on what a plugin for it might look like?
• Having the django plugin is not absolutely required to run mypy on this code, but it makes the user experience significantly better and would likely speed up fixing all the current errors. • Mypy doesn’t have support for only enabling a plugin for a subset files/roots. There is only one configuration file (
$ROOT/mypy.ini
) and it doesn’t support hierarchical config like pytest’s 
conftest.py
. • To enable the django plugin in mypy you need to make the project settings (
django_projects/test/settings.py
) and all transitive deps a 
mypy
tool resolve dependency. The plugin throws an error if the project can’t be 
django.setup()
.
b
Hmm we actually could make something like this work, behind flag. If we (optionally) partitioned mypy by config we could do this Cc @sparse-lifeguard-95737 @hundreds-father-404
e
Ya I think similar partitioning to what Dan added for pytest would work, with the addition of keying on a config file. Allowing them to group and share init time the same way would be even better. It's the same motivating problem (django setup).
b
We already partition by config in another plugin, so all the pieces are there
And this could be useful for any tool not supporting hierarchical configs
➕ 1
e
Would I be able to take advantage of those in a plugin, or is this something best done in pants (assuming you want it eventually)?
b
Both?
s
I think Michael means, is the plumbing for 
check
already generic enough that we could roll our own custom mypy plugin with the batching logic?
Or are there hard-codes things in the core code that would block us?
b
Ah nope
Check doesn't partition today
Oh but if you roll your own mypy plugin you can do it
s
🤔 yeah because it already has some built in partitioning by resolve, right?
b
By IC/resolve
👍 1
s
So there must be enough of a skeleton in the core check logic to support partitioning of some sort when appropriate
b
Well not technically. Any rule can invoke any number of other rules as many times as it wants 😌
s
That is true lol
b
But I think Check does ask for N Check results from the plugin implementors
👍 1
h
But I think Check does ask for N Check results from the plugin implementors
yes
s
But there’s no hard rule about what N actually maps to? (Unlike 
test
which was pretty locked into 1 result per target)
b
I think we should just partition check and call it a day
e
Which is the other plugin that partitions by config?
b
Scalafmt I think
e
I sounds like this is something pants might want to support generally? Should I file an issue on gh?
b
Please do. A generic "check should partition" as well as one for maybe partitioning more tools by config. We can inventory plugins for the latter
e
b
he's me
😂 1
e
How does a "check should partition by config" issue linked to https://github.com/pantsbuild/pants/issues/17328 sound?
b
Different issue, but is blocked by that one
And honestly, it isn't just check, fmt/lint/fix could benefit too
I like this a lot! I'm glad you posted this @eager-dress-66405
e
7 Views
Open in Slack
PreviousNext
Powered by