Can someone ELI5 `export` vs `export-codegen`? `e...
# general
w
Can someone ELI5
export
vs
export-codegen
?
export
- Export Pants data for use in other tools, such as IDEs.
export-codegen
- Write generated files to
dist/codegen
for use outside of Pants. I'm missing some of the nuances as to what determines whether a new plugin should use one over the other, since they both appear to end up in the
dist
folder. https://www.pantsbuild.org/docs/setting-up-an-ide has some additional examples, which are
virtualenv
vs
protobuf
. The best heuristic I can come up with is that
export
is intended for tooling/support/not-part-of-release, while
export-codegen
would represent files which would be part of the compilation/packaging/binary/production release process. Specifically, I would wonder where the bash completions would go, and based on my assumption above, it's more likely an
export
since it's sourced by the terminal, and has no part in actual production code.
f
Specifically, I would wonder where the bash completions would go, and based on my assumption above, it’s more likely an
export
since it’s sourced by the terminal, and has no part in actual production code. (
The v1 bash completions were generated by a dedicated
bash-completion
goal.
That name or maybe a
completions
goal taking a
--shell=bash
argument seem fine to me.
h
The ELI5 is "this is probably not super-well designed, sorry"... but in both cases the idea is to take intermediate data that Pants normally produces and uses internally and make it available to the user.
export-codegen preceded export, and was specifically for allowing you to see generated code for debugging, or in case you had some process external to Pants that needed the generated code.
export is currently specifically for creating virtualenvs for external use
But it seemed wrong to have a special-purpose
export-venv
goal, which would have meant creating a specific and python-only goal.
So I tried to turn
export
into a generic goal, with the idea of maybe folding export-codegen into it
However now I think I was probably wrong about this
Because the generic
export
goal takes a target set as input, and so its motion is "export per-backend exportables from these targets". But , for example, the most useful input to "export a venv" is probably the name of a lockfile, not a set of targets
And exporting bash completions shouldn't need any inputs at all
Also, if I just want to export codegen, or now bash completions, I shouldn't have to wait for a resolve and venv creation, which are time-consuming
So basically `export`/`export-codegen` need rethinking. We need some way to selectively export just certain types of things, and with varying types of input. One way is "separate goal per thing you can export", but proliferating goals is not ideal.
Another might be a flag that tells
export
what to export
Anyway, some thought is needed here
w
So, my opinions on this (especially with regards to bash completion) are: • I wouldn't want a separate goal for completions. I think there are already too many goals (goal creep), most of which I don't use and don't really even understand, and since goals are generally a critical public API - I think keeping them thin provides a better developer experience, and nesting more advanced usage (like, rolling up all the Pants introspection goals into an
inspect
or something, since they're not directly related to deliverable code) ◦ The whole
fix
vs
fmt
discussion is a good example of how tricky the public API can be, especially if it ever involved a change ◦ Testing bash completions has really pushed this idea with me, seeing which goals don't have options, which do - and just generally trying to tab complete everything to see whether I can guess what can come next • It sounds like
export
and
export-codegen
could feasibly be merged, from a user-centric point-of-view (completely unrelated to technical implementation/details and targets and such) ◦ As a user, knowing that
export
takes "inside Pants" to "outside Pants", that's already good enough to provide me with direction ◦ Having options or targets seem reasonable
./pants export --venv
or
./pants export :protobuf_stuff
or
./pants export --completions
◦ Upon reading that, I'm not a huge fan of how that reads - but at least I could infer what it would do pretty easily upon reading Now, so this doesn't get bogged down as a generic API discussion, specifically with regards to completions - I'm at a bit of an impasse regarding where the completions should go in the next PR. I could put the file somewhere and use...
./pants run
?
h
You mean to make it accessible to end-users?
w
Yep, what goal/target/plugin/etc I should use to generate them for user access.
h
Good question
It does unfortunately depend on your pants.toml, so we can't simply bake it into Pants
w
Yep, my hope was that there was something that could run in the pants script, or part of the pants.toml changed discovery, and auto-export the completions file
There is also something funky that https://github.com/google/python-fire does that I need to look into. Somehow, completions just work - so I feel like they're playing around with the python aliases
In fact, I think this document summarizes (sorta) the same problem: https://docs.google.com/document/d/1PvEK0nd_k3MxRjTuyH7lYvK4y4b2iYwickn2B56WomU/edit
I guess my question is this: Let's say we somehow export a bash completion script, in the
pants
file - would it make sense to source the
./dist/pants-completion.bash
? So that it's basically there by default? In the same way we source
common.sh
or
pants_venv
? If something like that is reasonable, I think that's a good step forward Edit: Hmmm - I might be thinking of the mainline repo's
pants
script, but either way, if something like this idea is feasible.....