Hello everyone, I have a question about `python_di...
# general
s
Hello everyone, I have a question about
python_distribution
. In my monorepo, I have a package A that depends on package B (both owned by me). When I build package A, I want to build
A.whl
such that it includes
B.whl
and/or knows where to pick it up without relying on PyPI. Similar to how pip operates when we do not use the
--no-deps
option. But I noticed that the generated
A.whl
does not contain any reference to B apart from in the metadata. Does anyone know how to achieve what I want?
For more context, I am letting pants generate the setup.py for me. In my BUILD file, I have tried different combinations- I tried having B be an explicit dependency in the distribution target for A, I tried having B be an inferred dependency, and B being a dependency in the python_sources target for A. None of them had the intended effect.
h
So you want A.whl to include the code that is also in B.whl?
Pants goes to some lengths to prevent that, because it’s generally a bad idea, assuming those wheels are going to be published
That’s why A.whl has a
Requires-Dist
metadata entry for B.whl, and that gets resolved at wheel install time
Is B.whl also used independently? Why does it exist?
s
Yes, I do see B in the
Requires-Dist
metadata and pip does try to install B from PyPI when I install
A.whl
. However, some of our downstream users do not have access to the internet and until we get them to use pexes, they use whls. When I use pip to generate the whls, I can leave out the
--no-deps
and the generated
A.whl
contains all the dependency whls too. We could ship individual whls to them, but is there any way to tell pants that the whls should be all-encompassing?
h
If you manually write the setup.py then yes
You can do whatever you choose
unfortunately then you lose all the other benefits of generated setup.py
because right now it’s all or nothing
So that is not ideal
s
Got it, thanks! I was going to give the setuptools args in here a shot. Is that not worthwhile?
h
I think you will run afoul of “However, you cannot use data_files, install_requires, namespace_packages, package_dir, package_data, or packages because Pants will generate these for you, based on the data derived from your code and dependencies.”
s
Ahh, ok. Thank you. For packages that require an end-to-end whl, I suppose having my own setup.py is preferred.
(or better still, use a pex for such end to end use cases)
h
Yeah, that’s really what Pex is for
wheels are for publishing
and are intended to be consumed via version requirements
not by embedding or by direct URL
pexes are entirely self-contained
so really that is what you want
1