https://pantsbuild.org/ logo
#general
Title
# general
h

high-yak-85899

02/09/2022, 4:57 PM
What would be the easiest way to say that one target should depend on every target of a certain type in another directory?
Explaining the rationale for it is probably a bit challenging/distracting
But I have a directory that has
BUILD
files properly populated all the way down and I'd like for another target to depend on all the
python_sources
listed there.
But I feel like I haven't seen anything to do that and it's probably easier to make one top level target that globs by
my/path/**/*.py
or something similar to pick up what I need)
b

bitter-ability-32190

02/09/2022, 5:04 PM
You might try depending on the
name
given to a
python_sources()
target to see if that works. Othwerise, Pants doesn't have this functionality AFAIK although future changes could allow this (can't find the google doc link...). If you're importing using strings and
importlib
, you could try out https://www.pantsbuild.org/docs/reference-python-infer#section-string-imports
h

high-yak-85899

02/09/2022, 5:05 PM
But then I have to spell out the entire tree that I want to depend on, right? Since I'm following the one BUILD file per directory styling.
And yeah,
importlib
with strings is exactly what we're doing right now.
b

bitter-ability-32190

02/09/2022, 5:06 PM
If you're going the
name
route, then yeah one dependency per directory (try it out on one dir first though, I'm not sure if it'd work so don't waste time if it doesn't)
And if your strings are static E.g.
foo.bar.baz
then
string-imports
should work
h

high-yak-85899

02/09/2022, 5:07 PM
They unfortunately aren't. They're provided at run time by the user.
b

bitter-ability-32190

02/09/2022, 5:07 PM
Gnarly 😈
Yeah the
name
is your only hope then, today.
@hundreds-father-404 do you have a link to the Google Doc for generated targets changes? This is exactly the use-case I think could be solved by my comments in the proposal 🤓
h

high-yak-85899

02/09/2022, 5:16 PM
fwiw, my idea for a top level target that I refer to by name seems to work fine but I do get issues about multiple targets owning one file which I feel like I've seen before. It doesn't actually affect functionality, though. I'll check out
python_infer
too
1
b

bitter-ability-32190

02/09/2022, 5:19 PM
Yeah, my hope is you could define one top-level
python_sources()
which globs subdirs, but can be overriden at a subdir-level
BUILD
if needed. Today Pants sees both and doesn not attempt to select
You should be able to define metadata in the
python_sources()
override
field for individual files if you need to override info. But you'll have to specify it relative to the
BUILD
file dir
h

high-yak-85899

02/09/2022, 5:49 PM
I don't think python-infer is going to change anything. Just in case I misunderstood, there's no way to depend on all targets with a certain tag or something like that? I just really want to avoid typing out all of the subfolders since that's not very extensible.
But would be fine for now as I want to substantially overhaul how we're doing things. This is just a means to get by for now without too disruptive of changes.
1
b

bitter-ability-32190

02/09/2022, 5:51 PM
Yeah
[python-infer].string_imports
would only help if you were using static strings for the module name, in-code
h

high-yak-85899

02/09/2022, 5:54 PM
I'm assuming string imports wouldn't help me even if I did something silly like
my.specific.folder.*
?
b

bitter-ability-32190

02/09/2022, 5:58 PM
Correct 🙂
They're basically meant to help out when you're using
__import__("foo.bar")
or
importlib.import_module("foo.bar")
👍 1
h

high-yak-85899

02/09/2022, 6:00 PM
Okay, well I think I have an easy enough workaround for now and this problem should be sufficiently resolved when I restructure things. I wouldn't be surprised if there are more use cases for "depend on everything with this tag or of this target type" or something like that.
h

happy-kitchen-89482

02/09/2022, 6:02 PM
To sum up, it sounds like dep inference won't work because there is nothing to infer off. So you need to explicitly declare deps, and you want to do so succinctly.
You might need a custom target type and a rule
I don't think macros are powerful enough for this
cc @hundreds-father-404?
h

high-yak-85899

02/09/2022, 6:06 PM
Yeah, the current state of things for this particular case is I have (in generic terms)
script_runner.py
in one place and then a folder of
all/my/scripts
elsewhere (which has subfolders after that). The user specifies the path like
all/my/scripts/folder1/this_script.py
to run and
script_runner.py
would
importlib
that in and execute it. So I was hoping for an extensible way to say that
script_runner.py
depends on all of the
python_sources
in
all/my/scripts
.
The long term vision is that I will separately package all the things in
all/my/scripts
and then turn
script_runner.py
into
pex_runner.py
But packaging them will take some time because the architecture of our repo makes them have huge transitive dependencies at the moment so they end up being pretty heavy binaries in totality.
h

happy-kitchen-89482

02/09/2022, 6:56 PM
Yeah so I think a custom target type for
script_runner.py
and a rule that injects all those deps onto it is probably the way to go.
👍 1
3 Views