https://pantsbuild.org/ logo
#general
Title
# general
a

adorable-engine-71736

10/07/2021, 6:02 PM
Hi, i have a working setup with 2.6.1 for mypy. Stub files are listed in
[mypy].extra_requirements
. I updated to 2.7.0 and added a lockfile (which contains my stub deps) The mypy PEX has the stub deps, afaict When i run
typecheck
now it says that the stubs are not installed. I would really rather not add the stubs as runtime deps as they are not, so wondering if/how to get the old behaviour working again
h

hundreds-father-404

10/07/2021, 6:20 PM
Hey Phlax, thanks for reaching out! Relates to this discussion: https://pantsbuild.slack.com/archives/C046T6T9U/p1633546871362700 What's your hesitation with them being runtime deps? Definitely open to revisiting this all - trying to understand your thinking here
a

adorable-engine-71736

10/07/2021, 6:22 PM
simply that they are only used in typechecking, so v reluctant to add unused deps (same eg with pytest deps) the only downside i saw to the old way was that i had to just specify all of the mypy requirements for all of my packages - rather than per-package, but i would still rather that than have unused deps
ie unused deps that end users must depend upon
i read through the posted comment and related ticket/s before posting, but tbh felt that it just underlined the issue, rather than resolved it
(apologies if i sound critical, not meaning too, just trying to explain the issue)
h

hundreds-father-404

10/07/2021, 6:30 PM
What do you think of
[mypy].extra_type_stubs
option? Having it there, rather than in
extra_requirements
, would still solve the issues that were motivating https://github.com/pantsbuild/pants/pull/12597 And the "extra" to speak to how you can still use normal runtime deps. This augments any of those
(apologies if i sound critical, not meaning too, just trying to explain the issue)
Not at all, we love feedback on things like this! Thank you for advocating it and for being thoughtful in how you do so
a

adorable-engine-71736

10/07/2021, 6:33 PM
yep that would work i think
i guess my ideal would be that i had some way to specify per-package so users could do
Copy code
pip install somepackage[mypy]
h

hundreds-father-404

10/07/2021, 6:34 PM
I don't think I understand what you mean with per-package?
a

adorable-engine-71736

10/07/2021, 6:34 PM
or similar, but having the old behaviour back with
extra_type_stubs
would be good
im building ~20 separate packages from the repo im working on
h

hundreds-father-404

10/07/2021, 6:38 PM
I still don't understand what you mean with per-package. Like that the
python_distribution
you build has an
extras
called
[mypy]
which will optionally include the type stubs the project depends on? I think related, I am wondering if a better solution here is for us to have first-class distinction between type-stub only deps vs. normal deps. Like that
python_requirement_library
tracks that. Such that MyPy can install everything, but other contexts like Pytest and packaging ignore those ones
a

adorable-engine-71736

10/07/2021, 6:41 PM
f

fast-nail-55400

10/07/2021, 6:41 PM
do we treat PEP 561 stubs-only packages in any special way? https://www.python.org/dev/peps/pep-0561
a

adorable-engine-71736

10/07/2021, 6:41 PM
most of the folders are separately published pypi packages
so lets say there are 3 (pypi) packages
a
,
b
and
c
end users who just need to use package
a
would do
Copy code
pip install a
if they happened to want to run tests on the package then ideally they should be able to do
Copy code
pip install a[test]
and if they wanted to typecheck then
Copy code
pip install a[mypy]
same with
b
and
c
and each should be able to maintain a separate set of deps for each context
but tbh, my more immediate concern is just to get my build working with 2.7+ and ideally without adding more deps
if i add more deps, then i will have to add them downstream with hashes etc, etc, and get those extra deps reviewed, agreed, landed
and then dependabot wants to update them
it just adds more review and maintenance overhead
h

hundreds-father-404

10/07/2021, 6:49 PM
Okay, thanks for this context. The easiest/quickest would indeed be adding
[mypy].extra_type_stubs
. That more holistic rework of type stubs would require much more design work and be harder to implement
a

adorable-engine-71736

10/07/2021, 6:50 PM
yep
is there a PR for
extra_type_stubs
?
it didnt seem to work locally when i added
h

hundreds-father-404

10/07/2021, 6:52 PM
No, I'm proposing adding it based on your feedback today + jyggen's yesterday
a

adorable-engine-71736

10/07/2021, 6:53 PM
ah k, i see
i can take a look, but v limited time atm, so not sure how far i will get if its not a v easyfix
h

hundreds-father-404

10/07/2021, 6:56 PM
Thanks, although I can try to get to it today or tomorrow - tbh that MyPy code is a bit confusing (which was a big motivator for #12597). What would be most helpful is if you can review the PR
a

adorable-engine-71736

10/07/2021, 6:59 PM
sure, would be happy to - i have not got so far with pants dev, but ive got the basics - feel free to request a review on github - im @adorable-engine-71736 there too
❤️ 1
h

hundreds-father-404

10/08/2021, 10:40 PM
a

adorable-engine-71736

10/09/2021, 12:50 AM
just saw that you added - thanks so much for addressing this! quickly scanning the code and it looks good, ill give it a spin in our env in the morning to confirm thanks again
❤️ 1
hi, i tried installing from the github repo but as its a pyproject.toml based repo i dont see a way to do a local/editable install
is there some magic im missing, ive searched on this previously and only ever found irate users and unhelpful responses
for that reason i tend to stick with setup.* configurations
h

hundreds-father-404

10/09/2021, 3:34 PM
Tried installing what, Pants?
If so, no need to manually test, but I appreciate the offer! Manually testing would require using
./pants_from_sources
, which involves compiling the Rust engine which can be really slow (After this commit is merged, we will publish the wheels for the commit so you can run the commit before an official release)
a

adorable-engine-71736

10/09/2021, 3:38 PM
hmm, ok, no aversion to rust, but happy to wait
h

hundreds-father-404

10/09/2021, 3:39 PM
If you are interested in testing it beforehand, you should be able to clone Pants and then use this script in your repo: https://github.com/pantsbuild/example-python/blob/main/pants_from_sources
a

adorable-engine-71736

10/09/2021, 3:40 PM
im struggling to understand how you guys work with this, i live in pdb and test all my code, both of which necessitate local install
might be helpful to add some dev docs on how to run locally with a git checkout
h

hundreds-father-404

10/09/2021, 3:43 PM
We live it with by using that
pants_from_sources
script, which allows us to test locally on any repository
4 Views