https://pantsbuild.org/ logo
Join Slack
Powered by
the python distribution target. Kris and I have be...
# general
s
the python distribution target. Kris and I have been discussing this and arrived at the conclusion that it would break the model of a setup.py-packaged distribution only providing a package and depending on third party libraries via install_requires
i.e. break the self-contained characteristic of a setup.py-based project
w
so, obviously i haven't reviewed the whole thing, but: in the future, needing to declare the external dependencies of the setup.py (rather than allowing setup.py to hit the internet) would likely be reasonable? cc @rough-minister-58256
anyway, for now it sounds like the answer is to just ban them. but targets generally do eventually sprout deps
r
the idea is to keep the interface of a given setup.py whole to permit use of that both in and outside of pants sans setup.py modification. because at the point it stops being a portable setup.py, it could more easily just become a 
python_library
etc.
and since pants doesnt provide any of the declared deps to the executing setup.py, it doesnt make sense for me for that to be able to declare any since they’d only be used transitively. i.e. a python_dist /provides/ a thing but shouldn’t be able to /depend/ on outward deps, since it can’t consume those other than transitively (which should already be covered by the setup.py interface)
w
I don't see why needing to declare the deps is incompatible. It's redundant, but.
"doesn't provide any of the" ... should it, eventually?
When that becomes an isolated process, the fact that it is hitting the internet is going to become a challenge
r
how would it? it’s an sdist of 1 python module w/ setup.py containing only the code for that module. output is exactly 1 bdist.
it shouldn’t be hitting the internet, ever, except in “bad behavior” cases
w
Gotcha.
r
it doesn’t need those deps to do anything (thats the “install_requires” case) - it just declares them in its wheel metadata which then get transitively handled by the resolver.
so it doesnt really “depend” on them in the context of the interface of python_dist
w
Gotcha. So rather than dependencies it has "install_requires"
And we don't think those should be declared as dependencies...?
r
yeah. and install_requires is in no way plumbed via pants/pex, today.
w
They should match 3rdparty, right?
r
thats not how things work today.
if I depend on 3rdparty/python:a and that transitively depends on ‘b’, ‘c’, ‘d’ - the versions in 3rdparty are not considered unless transitively pinned at the target level.
(and we can currently compose 
target(deps=[3p:a, 3p:b, 3p:c])
for logical transitive
(so feels like it’d be cleaner to leave python_dist in the same boat as all other reqs and then solve w/ 3rdparty freeze at some point)
declaring dep edges outward from a python_dist feels like the wrong model, to me
python_dist provides exactly 1 bdist, to depend on something in that context I think would mean 
install_requires
would need to be plumbed to provide declared dists in the setup.py execution environment to actually manifest something for it to depend on. that might come later, but for the time being feels cleaner to me at least to outlaw deps.
w
ok. but worth recognizing the potential confusion there. every time we add arguments that define things that "quack like" dependencies, we should think carefully about whether they should be dependencies
sometimes they definitely shouldn't be... other times they should be (most recent example was 
resources
... we ended up needing a lot of special casing for that one)
i promised roger some docs about how to think about these, but it wasn't 100% clear to me how to think about them at the time (still isn't). but one thing that might help would be to think about whether the things in the dependencies list define a uniform set of inputs: ie, in the case of python_library, everything in my dependencies list can be converted into a PEX chroot (including resources)
so rather than defining a precise list of input target types for dependencies, it's more about a precise list of output types
for the jvm, everything in the dependencies list can eventually be converted into a Classpath
etc
r
I’m pretty sure disallowing dependencies on python_dist, at least initially, is the best way to keep the model and UX clean. I can just picture a user, day 1, complaining that the deps he declared aren’t being made available to the python_dist’s setup.py as they expected etc.
and I have yet to hear a counter argument for it. so.
would propose we just go that route unless there’s more to discuss. way easier to add that later than to remove it later, imho.
(we’ve been talking about this for what feels like an eternity)
7 Views
Open in Slack
PreviousNext
Powered by