https://pantsbuild.org/ logo
Join Slack
Powered by
Hey :wave: One thing I've come up against today wh...
# general
w
Hey đź‘‹ One thing I've come up against today when investigating đź‘– is having to do: 
sources=["**/*.py"]
in a 
python_sources
target, so that all 
*.py
files in subdirectories are picked up. I'm wondering if I'm doing something wrong 🙂 More context in 🧵
We're planning to have a 
tools
directory to house shared code among projects in the repo. It would look something like this:
Copy code
src/tools
├── BUILD
├── __init__.py
└── log
    ├── __init__.py
    └── logger_factory.py
With packages for logging, cryptography, etc. housed under 
src/tools
.
And I wanted to check to see if there's anything significant behind the default value for 
sources
looking only in the directory housing the 
BUILD
file.
Or if it's just a sane default 🙂 Thanks
w
The idea behind the "same directory as the BUILD file" is that using 
tailor
(or in general), Pants defaults to 1 BUILD file per directory. So, there is no default recursive glob. Personally, I use 1 BUILD file per "library" so I do something similar to what you're doing
Bit of a chain there
b
Yeah the TLDR is we want to move to a world with even fewer BUILD files, but there's some issues to smooth out first
Nothing's stopping you from big globbing, just certain things get a bit more verbose
w
Aside: I thought we had same project structures in the docs - but I can't find them, so maybe I was imagining that
w
Thanks for the help. There's a few example structures linked, but none of them had sub directories.
h
You can wrap that up in a macro btw
So you can have a custom 
python_rsources()
, say, that under the hood expands to 
python_sources(sources=["**/*.py"])
To save some boilerplate
w
That's helpful đź‘Ť
h
Yeah, the fact that it doesn’t recurse is a “sensible default” for a world where you have a BUILD file in every dir.
And that is potentially desirable because when you want to tweak the metadata for a specific package you then have to start splitting targets and excluding lower down files from the higher up targets and so on
But the future we hope to move towards is that BUILD file metadata is accumulated up the filesystem hierarchy. Then you could have a handful of recursive-globbing BUILD files to start with, and as you need to add specific metadata you do so locally without having to exclude those files from the higher up targets, because the “overlap” doesn’t introduce ambiguity.
w
@happy-kitchen-89482 This feels like a question we answer a lot (111), recurse, project structure, etc May be worth elevating placement in the docs
đź‘Ť 1
h
BTW, if you have tests alongside your code then 
sources=["**/*.py"]
might be too naive, as it will capture test-related code. The full default value is here: https://github.com/pantsbuild/pants/blob/24e5435cd7840d28520a6aa205375315c0dd8191/src/python/pants/backend/python/target_types.py#L1057
And jumps through hoops to exclude test and test util files
c
Maybe it's a naive suggestion, but could one instead add a "recursive=true" argument to python_sources() and python_tests() so that the default patter is kept, but subdirectories are traversed?
h
I’m not sure how well that would work with the machinery, might be easier to just provide a recursive version (like 
python_rsources()
)
c
You can wrap that up in a macro btw
So you can have a custom 
python_rsources()
, say, that under the hood expands to 
python_sources(sources=["**/*.py"])
There’s also 
__defaults__
if you prefer to keep using the standard target names (disregard if it’s becoming too much information at once):
Copy code
__defaults__({python_sources: {"sources": ["**/*.py"]}})
https://www.pantsbuild.org/docs/targets#field-default-values
w
I think I like the macro or defaults approach better, as I think adding a recursive 
python_sources
would then lead to something like that per backend, which feels like it would really bloat the number of targets, for a small number of BUILD files and I say this as a recursive targets kinda person
âž• 1
c
if we’d like to go the recursive options route, I think it would make more sense to implement it for the sources field directly so it would work the same for any source target. But, that feels quirky any way….
5 Views
Open in Slack
PreviousNext
Powered by