Discussion of capturing files from process output ...
# development
a
Discussion of capturing files from process output (thread forthcoming)
I’m looking into this comment from @sparse-lifeguard-95737 (https://github.com/pantsbuild/pants/pull/17928#discussion_r1062928248). It looks like the digests that come out of `Process`es are relative to the workdir of the process, which means if you want to capture a file with a relative path that ascends the directory tree (and optionally descends it again), you need to specify a
workdir=
with a common parent directory, and then have the process itself
chdir
into the working directory it actually wants to work from. The manual workaround is possible for
experimental_shell_command
, because it’s possible to call
cd
as part of your command string. It’s less obvious that you can do it from
experimental_run_in_sandbox
, though most languages can let you do it. I’m wondering if there’s scope for an
output_root_directory
on
Process
? This would solve the problem here by allowing the
output_root_directory
to be the build root, and the workdir to be the
BUILD
file directory. (ping @witty-crayon-22786)
w
possibly…? many of our APIs are shaped the way they are because they need to work remotely with REAPI
that one though seems like something that could always be done post-facto, regardless of where the process ran?
if so, my primary question would just be around UX and complexity.
a
So you can definitely implement this with the primitives that are already there: you just make a
Process
with
workdir=None
and it calls
bash -c "chdir {NEW_WORKDIR} && {shlex.join(original_argv)}
(and then do whatever add/remove prefix operations are needed to get the
output_dir
correct after the process completes)
w
would all of the interactions with the
workdir
argument be clear? it seems like maybe some wouldn’t be legal (or sensical at least)…
a
So at the moment, it’s illegal for
output_files
to
..
-ascend higher than `workdir`; in this case, the
Process
implementation would notice if you
..
-ascended higher than the buildroot
(I am beginning to see that the answer here is to implement exactly what I described above, just for
esc
and
eris
yeah, that ended up being simpler than expected: https://github.com/pantsbuild/pants/pull/17938