https://pantsbuild.org/ logo
#development
Title
# development
f

fast-nail-55400

07/22/2021, 1:55 AM
are there any examples of generating targets dynamically from a plugin?
context is to generate
go_external_module
targets based on the contents of a
go.mod
file
h

hundreds-father-404

07/22/2021, 2:05 AM
The closest thing to "target gen" is
./pants tailor
. Would that work? We opted to not have dynamic target gen in Pants v2, going with the simpler codegen mechanism that only generates
Sources
. It's plausible we may want it still, and generally we've been wanting to get rid of targets in some ways. but not yet implemented
f

fast-nail-55400

07/22/2021, 2:12 AM
it probably would be more useful for Pants to transparently handle the contents of the
go.mod
file, than having to run
./pants tailor
each time
go.mod
changes. the main issue would be
go.mod
and the tailor-generated targets getting out of sync.
what does
python_requirements
do? it seems like it reads
requirements.txt
and makes
python_requirement_library
targets?
h

hundreds-father-404

07/22/2021, 2:14 AM
oh, that's the old v1 style macro. same thing that Liam added for
poetry_requirements
The API is old and has some limitations, but it works until we have a more robust mechanism!
f

fast-nail-55400

07/22/2021, 2:17 AM
and macros cannot invoke the engine
h

hundreds-father-404

07/22/2021, 2:17 AM
correct
f

fast-nail-55400

07/22/2021, 2:17 AM
thus no way to call out to the
go
tool via
Process
1
a short term solution could be to make depending on a
go_module
target means linking with all the external modules in go.mod. But my preference is to have a way of dealing with external modules on an individual basis. hence the thought of
go_external_module
.
h

hundreds-father-404

07/22/2021, 2:18 AM
it's plausible this is the impetus we need for redesigning macros for a v2 world one of the main limitations of current API is that file watching is totally broken, we won't invalidate in certain cases if your
requirements.txt
is changed (if the file name is different than we expected)
Both
python_requirements
and
poetry_requirements
would benefit from a new mechanism
f

fast-nail-55400

07/22/2021, 2:19 AM
the other solution would be for tailor to generate the
go_external_module
targets and we accept the violation of DRY principle in the short term.
frankly, targets as a concept probably map better to Go and JVM than they do for Python
given:
and generally we’ve been wanting to get rid of targets in some ways
h

hundreds-father-404

07/22/2021, 2:21 AM
a short term solution could be to make depending on a go_module target means linking with all the external modules in go.mod. But my preference is to have a way of dealing with external modules on an individual basis.
yeah, that feels pretty critical to Pants's whole "fine-grained dependencies" thing We've been realizing that dep inference is the crucial differentiator for Pants, and it's what allows for our benefits like caching and project introspection so, fine-grained 3rd party deps does seem important
targets as a concept probably map better to Go and JVM than they do for Python
Eh, even w/ Go (and Terraform) it sounds like targets aren't a great fit. Namely, that it's important the target "own" the whole dir, not less and not more. And that there's only one target doing that
f

fast-nail-55400

07/22/2021, 2:22 AM
well whatever you call it, there needs to be some concept that treats multiple files as an indivisible unit.
1
whether that is exposed to user is also something to consider. long term: I’d want Pants to infer this “target” (or whatever we call it) concept from just the existence of
go.mod
. No need to even require a BUILD file with a
go_module
target in that case. It can just be inferred.
h

hundreds-father-404

07/22/2021, 2:24 AM
Benjy wants the same thing, including w/ Python And users frequently complain about targets (rightfully so)
f

fast-nail-55400

07/22/2021, 2:25 AM
ideally we’d only require users to write stuff into a BUILD file if they need to override inference or for things that cannot be inferred for whatever reason.