Another question for understanding. Imagine the fo...
# general
a
Another question for understanding. Imagine the following structure:
Copy code
package_a/
  a.py
  BUILD -> python_distribution(name = "package_a")
package_b/
  b.py -> imports package_a.a
  BUILD -> python_distribution(name = "package_b")
My previous understanding was that when building wheels for
package_b
, it would detect automatically the dependency on
package_a
(via the respective file
a.py
) and I don't need to list this dependency explicityly in
python_distribution
i.e.
package_b
will just include the sources and transitive deps of the
package_a
However, some recent experiments show it is not true, so I must include
//package_a
as a
dep
of
package_b
. Is that correct? Doesn't this defeat the idea of pants taking over dep management, so that if later the actual import of
a.py
is removed, there's not dangling dependency on
package_a
remaining? Thanks! I hope I explained my issue clearly, LMK if I can refine.
1
f
can you add more information about the experiments that show the dependency is not being inferred?
(with some
./pants dependencies
or
./pants dependees
output)
This could be a bug in dep inference that needs to be fixed. But need more info to know.
a
Copy code
〉./pants dependencies --dependencies-transitive package_b/src/b.py
(prints nothing)

〉./pants dependencies --dependencies-transitive package_b/src
package_b/src/b.py

〉./pants dependencies --dependencies-transitive package_b
package_b/src/b.py
For completeness, the zip attached and the structure is (in the same repo)
Copy code
package_a
├── src
│   ├── a.py
│   └── BUILD
└── BUILD

package_b
├── src
│   ├── b.py
│   └── BUILD
└── BUILD
h
To be sure I understand, when you say "package_b will just include the sources and transitive deps of the package_a", do you mean that you would expect the wheel built from package_b to include the code of package_a as well as package_b?
What actually should be happening here is that Pants detects that package_a produces a wheel, so it places a dependency on that wheel in package_b's wheel metadata
If package_a did not produce a wheel, then package_b would embed its code in its own wheel
BUT then if there is some package_c that also depends on package_a, you end up with possibly two wheels that want to own package_a. But only one can, to avoid runtime collisions. So you have to make sure the ownership in your repo is clear.
a
Aha! Thanks a ton, this makes sense. So this means that Pants tries to figure out if a given source file
a.py
owned by some package producing a wheel. This makes sense. P.S. Yes (to your initial question), somehow I was thinking that even in presense of another owning wheel-creating package
package_a
, the individual sources like
a.py
shall be included directly into anyone who wants them, like
package_b
. P.P.S. Docs are amazing
❤️ 2
h
Glad they were helpful!
Yeah, Pants can overlay a "wheel structure" on top of the codebase, where each source file belongs to at most one wheel, and when you build wheels, it figures out which wheels need to depend on which other wheels, and sets the
Requires
metadata appropriately
Well, technically it sets the
requires
keyword in the setup.py it generates, and then it builds a wheel from that setup.py
a
May I ask an additional question? I could not find an answer in the docs: so in the above case, if I replace the dependant
package_b
with
pex_b
(i.e. pex depends on a file
a.py
owned by a distribution
package_a
) - is this expected that pex includes the whole of
package_a
? From what I see, when building
pex_b
I need no special
dependencies = [...]
clause in
pex_binary
. Actually it makes sense - just means that pex will contain the wheel of
package_b
inside and run normally.
h
A pex will contain all the code it needs, it does not have the same restriction where each source file belongs to at most one wheel
The difference is that a pex is intended to be a standalone executable
So having multiple pexes that happen to contain some of the same sources is fine - they don't interact
Whereas wheels are published and consumed, so it's important that they don't introduce collisions
a
Thank you, it's clear now!