How do I specify multiple subdirectories as a depe...
# general
f
How do I specify multiple subdirectories as a dependency of a
python_distribution
? Do I need to list all of them individually? Or is there some way to say “All
python_source()
targets which are in subdirectories of this
BUILD
file”?
h
I had to capture all my sources in one top level
python_sources
target and then add that target to my dependencies for my distribution.
I don't think the distribution tooling plays nicely with the 1:1 philosophy of putting build files at each subdirectory level. Maybe it's changed in newer releases, though.
f
I had to capture all my sources in one top level
python_sources
target
How did you do that?
h
Moved it to the top level and then changed the
sources
glob pattern to
**/*.py
f
Ah
h
And then deleted all the
BUILD
files below that
f
Did
./pants tailor --check
have a problem with that?
h
Nope, tailor just looks for sources that don't have a resolvable target
1
Since all the sources still have a target, it works fine.
f
Woop! Thanks for the help 😄
h
Thanks for the help @high-yak-85899 🙂 @future-oxygen-10553, in case you haven't yet seen it, it's often particularly a good idea to use
overrides
when you're using recursive sources globs like that. It keeps metadata as precise as possible
Copy code
python_tests(
   sources=["**/*_test.py"],
   overrides={
      "subdir/foo_test.py": {"timeout": 120},
  },
)
(You can use globs in the key, like with
*
. And you can use a tuple for the key for multiple globs / file names
🎉 1
f
Thanks @hundreds-father-404 I have used overrides for skipping linting on files before, so I’ve seen that pattern 👍
❤️ 1
w
more generally, because of dependency inference, you should only really need to mark the “root” modules that are part of the distribution
for example, Pants actually lists all of its plugins, because they are loaded “reflectively” by the pants binary (rather than via
import
statements)… https://github.com/pantsbuild/pants/blob/485d5858bbdd7f450a015d70bd6f76adb86e33ac/src/python/pants/bin/BUILD#L6-L70 but it isn’t necessary to “include all of the sources” for those plugins. they have dependencies, and those dependencies will be pulled in.
f
@witty-crayon-22786 What do you mean by “root” modules? If I have something like:
Copy code
package-name
|--- __init__.py
|--- submodule/__init__.py
|--- submodule/sububmodule/__init__.py
|--- BUILD
BUILD
pyproject.toml
w
for a library, if you’re exposing a particular public API out of some modules, then those modules would be “roots”… anything that those modules depend on needn’t be listed
f
I see
w
(although the plugin list i linked above might look large, it’s still only the “roots”… the transitive dependencies of the plugins are ~500 modules)
f
Thanks!
w
having said that, this is still a manually maintained list. that means that if something is not intended to be part of your public API, you could skip putting it in there… but you could also forget to put it in there.
h
Thanks, Stu. Great points to remember when we start making new distributions. In my case, I had a whole bunch of independent roots that would have to have been manually included, but I can see how a more traditional distribution is easier to list out.
👍 1
w
…cross referencing https://pantsbuild.slack.com/archives/C046T6T9U/p1662671864877109, which raised this again recently.