Tom and I tried running the Protoc code in an envi...
# development
h
Tom and I tried running the Protoc code in an environment that didn’t have
unzip
installed, which resulted in downloading the tool to fail because in
core/util_rules/archive.py
, we expect for
unzip
to be discoverable on
/usr/bin:/bin:/usr/local/bin
. We also expect for
mkdir
and
tar
to both exist on those paths, along with
/bin/bash
existing. Is it reasonable to expect for
unzip
and
tar
to be installed, or should we rewrite this code to not run a
Process
? We used to do tar extraction in Rust for some JVM-related code.
f
can we just run Pants Python code in remote execution and use the built-in Python modules?
or just make sure we document the tool requirements
👍 1
I still believe a “toolchain” abstraction that maps a literal like “unzip” to a full path might be useful
although that is orthogonal to this issue
so never mind
h
I think Twitter found that Python’s modules were much slower and/or less concurrent than they wanted
f
unzip in the remote image is fine
💯 1
doing all that work remotely and storing in CAS w/o bringing down to local is good
h
Cool. Agreed it should be documented that you need it
f
although I’d rather not rely on PATH
some remote execution systems support that (and toolchain’s will) but the REAPI spec only says that absolute paths and path relative to input root are allowed
👍 1
moreover, pants providing a PATH here overwrites the PATH configured in the remote image for that request
👀 1
h
We could use the BinaryPaths mechanism to discover where the binary is installed. That didn’t exist when Benjy wrote this code a few months ago
f
let’s discuss tomorrow. I’d be fine with it passing “unzip” but not passing in a PATH
then the image’s own PATH would take effect and unzip would still be found
any way, that is still tangential to your original question about whether to use unzip
which is still 👍
💯 1