Background: I have several

python_distribution

targets for building wheels destined for PyPI. Several of them contain one (or a few) primary entry point scripts, so those dependencies get inferred cleanly from

entry_points

and

sccripts

. One of them is a library (

st2common

) used by the rest. Though it has some utility entry points, for the most part, it contains code that is shared by the entry points in the other packages. Is there a way to avoid explicit dependencies from the

python_distribution

to the the

python_sources

in the library? In particulars, based on dependency inference, if one of the other first-party packages depends on something in the library's

python_distribution

, can that library get also depend on that code, so that it is available within the packaged wheel? In other words, I want to avoid adding explicit dependencies to satisfy many errors like this:

NoOwnerError: No python_distribution target found to own st2common/st2common/services/access.py. Note that the owner must be in or above the owned target's directory, and must depend on it (directly or indirectly). See <https://www.pantsbuild.org/v2.15/docs/python-distributions> for how python_sources targets are mapped to distributions. See <https://www.pantsbuild.org/v2.15/docs/python-distributions>.

Adding 37 more explicit dependencies quiets the error for this

python_distribution

. But, that does not feel sustainable to expect people to manually register stuff whenever they add something to

st2common

.

Several of them contain one (or a few) primary entry point scripts, so those dependencies get inferred cleanly from entry_points and scripts.

Which are "those dependencies"?

Yes. I'm using pants-generated setup.py (aside: though I look forward to replacing that with a generated pyproject.toml whenever we implement that). 👀 re-reading that section of the docs... one sec

Ah, makes sense. So,

S -> S'

is inferred, and therefore

D -> D'

would be induced, but the problem is that Pants doesn't know about

D' -> S'

yes

I can't see how that inference you're proposing would work though

yeah. I’m not sure how to implement it.

D'

needs to infer a dependency on

S'

because

D

needs

S'

I’m preparing the PR to add all of those dependencies in the

st2

repo, and I expect I’ll get questions about why we have to manually maintain such a huge list of dependencies in such a central library. I have a layout: Da/a Db/b Dc/c D’/prime where

D*/BUILD

includes a

python_distribution

. If anything in each of those directories needs to be packaged, then it goes in the parent package. So, I do not have any hierarchical `python_distribution`s defined. ie, I’m not doing this (which would be confusing): Da/some/path/Dd/d where Da has a python_distribution, and Dd has another python_distribution. Logically, D* should have a “dependency” on anything in that directory. In

setup.py

that’s like using:

from setuptools import setup, find_packages
setup(packages=find_packages(exclude=["tests"]))

So, I guess I’m looking for a replacement for

find_packages

to minimize manual maintenance.

OK, but that has nothing to do with

S

needing code from

S'

@happy-kitchen-89482, what about a special dependency field on the `python_distribution`: • depend on everything in these directories, unless it is unused by anything else. Thus, anything that is "dead code" (nothing else in the repo depends on it) just automatically drops out of the distribution. Except I still don't see how to implement that. Hmm.

Yeah, that makes more sense I think

find_packages

blindly includes everything in the given directories. pants use of inferred dependencies effectively prunes the distribution down to only files that are actually in-use, effectively dropping "dead code". So, this zsh command (

=()

substitutes the path to a temp file and saves the process output to that temp file) gives me the list of dependencies to add to the

st2common:st2common

`python_distribution`:

comm -13 =(./pants dependencies --transitive st2common:st2common | grep st2common/st2common | sort) =(./pants list --filter-target-type=python_distribution --filter-address-regex=-st2common:st2common :: | xargs ./pants dependencies --transitive | grep st2common/st2common | sort)

Once I add those as explicit deps, then this command shows me the dead python code that is not depended on by anything:

comm -13 =(./pants dependencies --transitive st2common:st2common | grep -v -e : -e __init__.py | grep st2common/st2common | sort) =(find st2common/st2common -name '*.py' -and -not -name '__init__.py' | sort)

But how would you know what files are in-use?