https://pantsbuild.org/ logo
f

fresh-cat-90827

08/18/2022, 3:06 PM
I am trying to build a Python distribution (wheel) using the
python_distribution
target using a handwritten
setup.py
file. The docs say that
If you use a handwritten setup.py ... Pants will bundle whatever the script tells it to.
Running
python3 setup.py bdist_wheel
produces a wheel with expected contents, whereas
./pants package //:say-hello
produces a wheel with
dist-info
metadata only. Have I misunderstood how Pants runs the existing
setup.py
?
1
Using https://github.com/pantsbuild/example-python, root
BUILD
file:
Copy code
python_sources(
    name="setup-py",
    sources=["setup.py"],
)

python_distribution(
    name="say-hello",
    dependencies=[":setup-py"],
    provides=python_artifact(
        name="say-hello",
    ),
    generate_setup=False,
    sdist=False,
)
e

enough-analyst-54434

08/18/2022, 3:07 PM
How does that target graph know what the source file dependencies are? I think you're missing dependencies for "say-hello".
It only knows about setup.py and no other sources from what you have written down.
f

fresh-cat-90827

08/18/2022, 3:08 PM
That's my question -- I thought that because there is
setup.py
, Pants will run the
setup()
function -- which does find the packages etc
e

enough-analyst-54434

08/18/2022, 3:08 PM
You are probably forgetting Pants copies all needed files off to a /tmp sandbox before doing anything
When you run setup.py in-tree, the files are just laying there.
Pants though needs to know the full list to copy them to the sandbox.
This is true of all pants actions.
SO, you're right, Pants runs the setup() function - in the sandbox.
And the sandbox has no other files besides setup.py right now.
f

fresh-cat-90827

08/18/2022, 3:10 PM
I see, this makes sense. So if I want to piggy back on
setup.py
functionality (to find all the packages etc), then I have to make sure I get the whole tree with me as a dependency of
python_distribution
?
e

enough-analyst-54434

08/18/2022, 3:11 PM
So you need:
Copy code
dependencies=[
  ":setup-py",
  "/target/containing/actual:code",
]
Yes!
That's exactly it. If you hand-manage setup.py you have to hand-manage everything else too.
f

fresh-cat-90827

08/18/2022, 3:22 PM
gotcha! Got it working with
Copy code
python_distribution(
    name="say-hello",
    dependencies=[
        ":setup-py",
        ":MANIFEST",
        "helloworld:lib",
        "helloworld/greet:lib",
        "helloworld/greet:translations",
        "helloworld/translator:lib",
        ],
I wonder if there's a shortcut to refer to all targets located under a directory? I can't use `helloworld::`:
UnsupportedWildcard: The address
helloworld::
included a wildcard (
::
), which is not supported.
😕
e

enough-analyst-54434

08/18/2022, 3:23 PM
That was pretty intentionally never supported since it generally only leads to badness.
1
h

happy-kitchen-89482

08/18/2022, 3:23 PM
Very often you only need to depend on the entry point, because that presumably already depends on everything it needs
e

enough-analyst-54434

08/18/2022, 3:23 PM
Good and supported on CLI, generally bad for BUILD just like import * is generally bad for code.
1
f

fresh-cat-90827

08/18/2022, 3:27 PM
oh yes, thanks Benjy!
Copy code
python_distribution(
    name="say-hello",
    dependencies=[
        ":setup-py",
        ":MANIFEST",
        "helloworld:pex_binary"
is enough indeed. Reading the docs, I thought using
pex_binary
won't be enough because its transitive dependencies wouldn't be picked up:
Naively, you might think that a python_distribution publishes all the code of all the python_source targets it transitively depends on. But that could easily lead to trouble if you have multiple distributions that share common dependencies. You typically don't want the same code published in multiple distributions, as this can lead to all sorts of runtime import issues.
h

happy-kitchen-89482

08/18/2022, 4:03 PM
With a generated setup.py, Pants partitions your code across the possible dists, so that you don’t have that duplication. With your own setup.py you have to do that yourself, or suffer the duplication consequences…
🙌 1
3 Views