👋 Hi everyone! what is the recommended way of replacing text in a source file as part of the package goal? e.g. i have a template metadata.json file for my app and i have to update the contents with the output of

git describe

Hi Pablo 👋 Take a look at

experimental_shell_command

for how to run some arbitrary commands during a build. https://www.pantsbuild.org/docs/run-shell-commands

So this is actually tricky.

experimental_shell_command

or any other Pants-native target will be run in a sandbox outside of your repo (for hermeticity). What that means is you essentially aren't able to run

git

commands.

Yea, that was nagging a bit at the back of my head… although there’s ways around that particular limitation (using

GIT_DIR

), but not the prettiest. Another option is to run the

git describe

prior to the

./pants …

and inject the value as an env var for the shell command, if it’s not dynamic in nature.

hi, thank you for the replies and suggestions. I did look at

experimental_shell_command

but it seemed that i would have to end up using

sed

or

awk

which felt counterintuitive for a build system (e.g.

ant

has a

replace

task, or in gradle by creating a function that does it). I did not know that i would not have access to git which also makes it awkward. I also looked a plugins and this seemed to be the best option, but it also felt like overkill.

FWIW wanting git metadata is a common ask IMO (and it makes sense) so we should probably find a way to support it first-class

@Joshua Cannon can you share your solution?

SO we have a shell script to output the metadata , so I'm using

experimental_shell_command

to run that. Then it's a matter of: • Getting the git command to work • Ensuring the cache "does the right thing" So here's a gist you can add easily as a plugin to get the absolute path to the repo root. Note it is an abs path so remote caching is right out. And here's a gist for a plugin which will code generate a file containing the current git SHA. When used a dependency it will invalidate the relevant rules when git SHA changes. YMMV it's a hack. In your case, you might be able to just twist the latter code to generate the

metadata.json

for you (it already has the build root, see line 47)

awesome, thank you 🙂

I recently merged https://github.com/pantsbuild/pants/pull/15374 which supports running

setuptools_scm

specifically (and that calls out to git). It sounds like you want a more generic version of that?

In our case I want a collection of git info, so I think we should collect the use cases first. That'll help us decide which way to go.

BTW can you not use

experimental_shell_command

by setting GIT_DIR and GIT_WORK_TREE ?

You'd have to set it to the path to git dir, which not everyone has cloned to the same path, no?

The same relative path to the buildroot? We'd have to support interpolating

{buildroot}

the same way we support

{chroot}

You could but it really raises more questions than answers w.r.t. caching 😅

True

(which is why I have that awkward

gitstate

target)

That isn't cached

The other issue is if you get it working, depending on your dep graph, I found our build metadata deeply rooted (lots of files indirectly depend on the python file which reads the metadata). Therefore doing it the "right way" invalidates my dep graph every commit 😅

Clone the

.git

dir into the sandbox as a bare clone (no worktree) From there you can do all kinds of git-fu in read only mode.

@bitter-ability-32190 we currently use git describe to version our python package, i wonder if there is any way of curbing how much of the dep graph gets invalidated

The way we solved this is to have the file which reads the metadata use an

experimental_shell_command

which is hermetic (doesn't get invalidated). Then the dep graph is sound although the metadata is stale. Then for our docker images (or really anything we want to package) I explicitly tell pants to ignore that

experimental_shell_command

and use an identical one which depends on that

gitstate

target I provided earlier. So then it's cached behavior depends on commits, but we arne't building docker images every PR, so it's OK