https://pantsbuild.org/ logo
#general
Title
# general
f

future-oxygen-10553

08/25/2022, 8:00 PM
How do I specify multiple subdirectories as a dependency of a
python_distribution
? Do I need to list all of them individually? Or is there some way to say “All
python_source()
targets which are in subdirectories of this
BUILD
file”?
h

high-yak-85899

08/25/2022, 8:01 PM
I had to capture all my sources in one top level
python_sources
target and then add that target to my dependencies for my distribution.
I don't think the distribution tooling plays nicely with the 1:1 philosophy of putting build files at each subdirectory level. Maybe it's changed in newer releases, though.
f

future-oxygen-10553

08/25/2022, 8:03 PM
I had to capture all my sources in one top level
python_sources
target
How did you do that?
h

high-yak-85899

08/25/2022, 8:03 PM
Moved it to the top level and then changed the
sources
glob pattern to
**/*.py
f

future-oxygen-10553

08/25/2022, 8:03 PM
Ah
h

high-yak-85899

08/25/2022, 8:03 PM
And then deleted all the
BUILD
files below that
f

future-oxygen-10553

08/25/2022, 8:03 PM
Did
./pants tailor --check
have a problem with that?
h

high-yak-85899

08/25/2022, 8:04 PM
Nope, tailor just looks for sources that don't have a resolvable target
1
Since all the sources still have a target, it works fine.
f

future-oxygen-10553

08/25/2022, 8:05 PM
Woop! Thanks for the help 😄
h

hundreds-father-404

08/25/2022, 8:11 PM
Thanks for the help @high-yak-85899 🙂 @future-oxygen-10553, in case you haven't yet seen it, it's often particularly a good idea to use
overrides
when you're using recursive sources globs like that. It keeps metadata as precise as possible
Copy code
python_tests(
   sources=["**/*_test.py"],
   overrides={
      "subdir/foo_test.py": {"timeout": 120},
  },
)
(You can use globs in the key, like with
*
. And you can use a tuple for the key for multiple globs / file names
🎉 1
f

future-oxygen-10553

08/25/2022, 8:17 PM
Thanks @hundreds-father-404 I have used overrides for skipping linting on files before, so I’ve seen that pattern 👍
❤️ 1
w

witty-crayon-22786

08/25/2022, 8:18 PM
more generally, because of dependency inference, you should only really need to mark the “root” modules that are part of the distribution
for example, Pants actually lists all of its plugins, because they are loaded “reflectively” by the pants binary (rather than via
import
statements)… https://github.com/pantsbuild/pants/blob/485d5858bbdd7f450a015d70bd6f76adb86e33ac/src/python/pants/bin/BUILD#L6-L70 but it isn’t necessary to “include all of the sources” for those plugins. they have dependencies, and those dependencies will be pulled in.
f

future-oxygen-10553

08/25/2022, 8:19 PM
@witty-crayon-22786 What do you mean by “root” modules? If I have something like:
Copy code
package-name
|--- __init__.py
|--- submodule/__init__.py
|--- submodule/sububmodule/__init__.py
|--- BUILD
BUILD
pyproject.toml
w

witty-crayon-22786

08/25/2022, 8:21 PM
for a library, if you’re exposing a particular public API out of some modules, then those modules would be “roots”… anything that those modules depend on needn’t be listed
f

future-oxygen-10553

08/25/2022, 8:21 PM
I see
w

witty-crayon-22786

08/25/2022, 8:22 PM
(although the plugin list i linked above might look large, it’s still only the “roots”… the transitive dependencies of the plugins are ~500 modules)
f

future-oxygen-10553

08/25/2022, 8:23 PM
Thanks!
w

witty-crayon-22786

08/25/2022, 8:23 PM
having said that, this is still a manually maintained list. that means that if something is not intended to be part of your public API, you could skip putting it in there… but you could also forget to put it in there.
h

high-yak-85899

08/25/2022, 8:27 PM
Thanks, Stu. Great points to remember when we start making new distributions. In my case, I had a whole bunch of independent roots that would have to have been manually included, but I can see how a more traditional distribution is easier to list out.
👍 1
w

witty-crayon-22786

09/08/2022, 9:31 PM
…cross referencing https://pantsbuild.slack.com/archives/C046T6T9U/p1662671864877109, which raised this again recently.