https://pantsbuild.org/ logo
#general
Title
# general
n

narrow-vegetable-37489

10/06/2021, 7:01 PM
Hi! After moving type stubs from
[mypy].extra_requirements
to
requriements.txt
(as recommended in https://github.com/pantsbuild/pants/pull/12597), our PEXs have increased in size by roughly 20MB each. This by itself doesn't sound like much but, seeing how they were "only" ~70MB before, it ends up being a significant percentage. Why it happens is quite understandable, but preferably type stubs would be skipped in PEXs since (to me) they don't seem to add much value there. Not sure if this is an oversight or simply a con to the pros of 12597 🙂
h

hundreds-father-404

10/06/2021, 7:04 PM
Hey Jyggen, thanks for reaching out about this! Agreed, 20mb is a lot and I don't think any of us expected that would be a particular concern, hm. Do you know what they are / why they're so huge? Iiuc type stubs should be simply small
.pyi
files Definitely open to exploring alternatives
Instead, [mypy].extra_requirements should only be used for what is needed to run MyPy itself, such as MyPy plugins. It should not include type stubs
I still think this is true, and it's important for the tool lockfile. But we could explore a new option like
[mypy].type_stubs
! It'd be preferable to "Keep It Simple" by not needing that, so would love to understand the particular situation better first
n

narrow-vegetable-37489

10/06/2021, 7:09 PM
The stubs are
grpc-stubs
,
types-contextvars
,
types-decorator
,
types-Flask
,
types-freezegun
,
types-futures
,
types-jwt
,
types-mock
,
types-protobuf
,
types-python-dateutil
,
types-pytz
,
types-PyYAML
,
types-redis
and
types-requests
. While installing them I noticed that mypy was installed as well (maybe due to one of the stubs having it as a dependency?), maybe that could be the main suspect. As you said, the stubs themselves are rather minimal.
h

hundreds-father-404

10/06/2021, 7:10 PM
Oh I bet that is the culprit, MyPy is huge. I gtg for next ~2-3 hours, but I'm wondering if it's possible to determine which is depending on MyPy and perhaps if a newer version of that dep fixes that? That is, if this can be worked around
n

narrow-vegetable-37489

10/06/2021, 7:12 PM
No problem! I'll dig around a bit more 🙂
❤️ 1
Seems
grpc-stubs
is the culprit for the MyPy dependency.
h

hundreds-father-404

10/06/2021, 7:41 PM
Is it like Django-stubs that it's also a plugin?
n

narrow-vegetable-37489

10/06/2021, 7:54 PM
I don't think so. All it does is to provide .pyi files for the various grpc packages (grpc, grpc_health, grpc_reflection etc.).
h

hundreds-father-404

10/07/2021, 4:55 PM
I think there's no reason they need to put MyPy in the
setup.py
! https://github.com/shabbyrobe/grpc-stubs/search?q=mypy Iiuc, it's a "dev dependency" for them, so they should put it in
requirements-dev.txt
Would you be open to opening an issue / submitting a patch?
FYI @narrow-vegetable-37489, thanks for the feedback! https://github.com/pantsbuild/pants/pull/13180
n

narrow-vegetable-37489

10/08/2021, 11:31 PM
No problem! I'll send a MR their way when I find the time. Thanks! :)
4 Views