Hmm... I have a library that has BUILDs in all its...
# general
e
Hmm... I have a library that has BUILDs in all its submodules for fine-grained dependency resolution. But for one pex, I need the whole thing packed in because the pex will be used to evaluate other files (so the fine-grained dependency won't work as the other files may need one of the libraries). But adding just the top-level dependency didn't work (it of course just brought in the stuff from the top-level BUILD). I can add all the subtree targets manually, but is there nomenclature for bringing in "every dependency under this one" for that sort of use case? (sorry, I feel like I've asked this before). I could of course just make the top BUILD have sources like
**/*.py
but that loses the point of the fine-grained resolution when I need it.
h
So relying on dep inference to bring in what your code actually needs won't work here?
Generally the resolution of BUILD files shouldn't really matter, since we infer dependencies at the file level
you could create another top-level
python_library
that globs over
***/**.py
and then your sources would have two owners, which isn't ideal but is workable
It means things like
./pants <goal> foo/bar/baz.py
won't work because there is no unambiguous owner of that file
e
Right, hmm. So yes, dep interference didn't work because the pex is essentially packaging up a venv which a "runner" script itself will be used to evaluate scripts that rely on libraries packaged in the venv. Since the runner script doesn't use any of those libraries directly, dep interference doesn't work and I have to declare the dependencies directly. It's not that onerous to copy all the dependencies in the lib I was just doing (only one submodule) but another one has 36 submodules and more probably coming in, which creates a maintenance issue. The "umbrella library" idea you suggest is probably the way to go here, I think--understanding the ambiguity of ownership.
w
the way to think about this is roughly that all of your
import
statements include files. but anywhere where you load something “reflectively” (i.e., without an import statement, likely via a string), you’ll need to explicitly declare the dependency
for example: internally, Pants has plugins, and they are loaded reflectively by their module name. Pants’ core binary does not have an
import
statement for any plugins: instead, the
pex_binary
declares a list of plugins in its BUILD file, which causes them to be included in its PEX
and it’s really just the dependency edge from the loader to the loadee that needs to be manually declared: usually, everything beyond that occurs via
import
statements