Hi all, If package AAA, hosted in a Pants monorepo...
# general
a
Hi all, If package AAA, hosted in a Pants monorepo under
src/AAA
, depends on BBB, which is also hosted in the monorepo under
src/BBB
, is it possible to automatically generate both AAA.whl and BBB.whl without explicitly running
pants package src/AAA src/BBB
? My original guess was
pants package src/AAA
automatically generates
dist/AAA.whl
and
dist/BBB.whl
, but the command only generates
AAA.whl
.
w
Copy code
pants package ::
? What's the problem you're trying to solve?
a
Thanks!
pants package ::
generates too many packages and PEX binaries (in my repo), which isn’t ideal for our workflow. Our team frequently modifies internal packages in our monorepo, so we’ve been using shell scripts to clean and rebuild dependencies:
Copy code
rm -rf dist  
pants package src/AAA src/BBB  
pip install dist/*.whl
However, this approach redundantly replicates Pants’ dependency resolution logic, which doesn’t seem optimal. While using PEX binaries can sometimes help, there are still cases where pip install is necessary. Is there a better way to package (and install) only what’s needed while ensuring dependencies are built automatically?
1
w
The dependencies are all build as-needed, but they might not be materialized into the dist folder, which is what you're seeing I think. Also, you might end up with something looking like
pants package ::
Is there a fixed set of packages you want built? Could you use a CLI alias to shorten it?
And if it's only a certain type you'd like materialized, there might be a way to use filter to do it: https://www.pantsbuild.org/stable/docs/using-pants/advanced-target-selection#filter-options
pants --filter-target-type=... package src/::
kinda thing
s
If you have BBB’s python_distribution target as a dependency in AAA’s python_distribution target, it will be a marked as a dependency in AAA.whl’s metadata. When you then pip install AAA.whl, that’s when BBB.whl will be pulled and installed. I think this behavior is in line with pip’s no-deps option.
But I agree with @astonishing-dentist-76763, there is definitely a case to be made for such an option to generate all the wheels at once.
a
Thanks to both of you!
s
Found this and maybe it is a bit relevant here: creating an
archive
target with the wheels that you want: https://www.pantsbuild.org/stable/docs/using-pants/assets-and-archives#archive-create-a-zip-or-tar-file
b
Are you building these wheels for "local" venvs? If so, using
pants export --py-editable-in-resolve=... --resolve=...
might be a good way to handle that: https://www.pantsbuild.org/stable/reference/goals/export#py_editable_in_resolve Also, I'm not 100% sure of this, but potentially there's pants-level dependencies between targets, in which case something like
pants dependencies src/AAA | xargs pants list --filter-target-type=python_distribution | xargs pants package
might work? That is designed to find everything AAA depends on, filter to just the
python_distributions
and then package those.
😮 1