https://pantsbuild.org/ logo
Join Slack
Powered by
What would be the easiest way to say that one targ...
# general
h
What would be the easiest way to say that one target should depend on every target of a certain type in another directory?
Explaining the rationale for it is probably a bit challenging/distracting
But I have a directory that has 
BUILD
files properly populated all the way down and I'd like for another target to depend on all the 
python_sources
listed there.
But I feel like I haven't seen anything to do that and it's probably easier to make one top level target that globs by 
my/path/**/*.py
or something similar to pick up what I need)
b
You might try depending on the 
name
given to a 
python_sources()
target to see if that works. Othwerise, Pants doesn't have this functionality AFAIK although future changes could allow this (can't find the google doc link...). If you're importing using strings and 
importlib
, you could try out https://www.pantsbuild.org/docs/reference-python-infer#section-string-imports
h
But then I have to spell out the entire tree that I want to depend on, right? Since I'm following the one BUILD file per directory styling.
And yeah, 
importlib
with strings is exactly what we're doing right now.
b
If you're going the 
name
route, then yeah one dependency per directory (try it out on one dir first though, I'm not sure if it'd work so don't waste time if it doesn't)
And if your strings are static E.g. 
foo.bar.baz
then 
string-imports
should work
h
They unfortunately aren't. They're provided at run time by the user.
b
Gnarly 😈
Yeah the 
name
is your only hope then, today.
@hundreds-father-404 do you have a link to the Google Doc for generated targets changes? This is exactly the use-case I think could be solved by my comments in the proposal 🤓
h
fwiw, my idea for a top level target that I refer to by name seems to work fine but I do get issues about multiple targets owning one file which I feel like I've seen before. It doesn't actually affect functionality, though. I'll check out 
python_infer
too
✅ 1
b
Yeah, my hope is you could define one top-level 
python_sources()
which globs subdirs, but can be overriden at a subdir-level 
BUILD
if needed. Today Pants sees both and doesn not attempt to select
You should be able to define metadata in the 
python_sources()
override
field for individual files if you need to override info. But you'll have to specify it relative to the 
BUILD
file dir
h
I don't think python-infer is going to change anything. Just in case I misunderstood, there's no way to depend on all targets with a certain tag or something like that? I just really want to avoid typing out all of the subfolders since that's not very extensible.
But would be fine for now as I want to substantially overhaul how we're doing things. This is just a means to get by for now without too disruptive of changes.
✅ 1
b
Yeah 
[python-infer].string_imports
would only help if you were using static strings for the module name, in-code
h
I'm assuming string imports wouldn't help me even if I did something silly like 
my.specific.folder.*
?
b
Correct 🙂
They're basically meant to help out when you're using 
__import__("foo.bar")
or 
importlib.import_module("foo.bar")
👍 1
h
Okay, well I think I have an easy enough workaround for now and this problem should be sufficiently resolved when I restructure things. I wouldn't be surprised if there are more use cases for "depend on everything with this tag or of this target type" or something like that.
h
To sum up, it sounds like dep inference won't work because there is nothing to infer off. So you need to explicitly declare deps, and you want to do so succinctly.
You might need a custom target type and a rule
I don't think macros are powerful enough for this
cc @hundreds-father-404?
h
Yeah, the current state of things for this particular case is I have (in generic terms) 
script_runner.py
in one place and then a folder of 
all/my/scripts
elsewhere (which has subfolders after that). The user specifies the path like 
all/my/scripts/folder1/this_script.py
to run and 
script_runner.py
would 
importlib
that in and execute it. So I was hoping for an extensible way to say that 
script_runner.py
depends on all of the 
python_sources
in 
all/my/scripts
.
The long term vision is that I will separately package all the things in 
all/my/scripts
and then turn 
script_runner.py
into 
pex_runner.py
But packaging them will take some time because the architecture of our repo makes them have huge transitive dependencies at the moment so they end up being pretty heavy binaries in totality.
h
Yeah so I think a custom target type for 
script_runner.py
and a rule that injects all those deps onto it is probably the way to go.
👍 1
3 Views
Open in Slack
PreviousNext
Powered by