https://pantsbuild.org/ logo
#general
Title
# general
e

echoing-farmer-15630

03/10/2022, 3:26 PM
Interesting notes from experiments with pants and local buildfarms (why on earth would you do a local buildfarm? A) for testing, and B) so that pexes are built with a specified architecture without needing pre-built wheels when source dists are necessary). Having said that... I successfully got buildfarm, buildbarn, and buildgrid building pexes, generic docker containers, and dockerized pexes. Hooray for our side! Having said that, it's slower than "immediate" builds (although I should run proper benchmarks), and I did run across a curious error; not sure if it's a buildfarm error or a pants error so mentioning it here. An
ADD
and probably
COPY
instruction of the form
ADD ./my_file /tmp/install/my_file
where
my_file
is listed as part of a
files
dependency works fine on buildbarn and buildgrid but on buildfarm generates the error
Copy code
#8 ERROR: "/my_file" not found: not found
------
 > [4/5] ADD ./my_file /tmp/install/my_file:
------
failed to compute cache key: "/my_file" not found: not found
...it could be I'm doing something strange, but given that it works on 2/3 build systems but fails on the third is interesting. Wanted to at least raise it for awareness but I'm not sure if I should file an issue given that I'm not sure which product the issue is with! I'm not sure given the size of generated artifacts if remote execution or caching even makes all that much sense for a lot of pants builds since that's an awful lot of I/O, but it's certainly been a worthwhile experiment. I still may try it for our CI solution. One of the buildfarm folks raised the idea that it may be worthwhile to add something to
pants
to disable remote execution (and maybe remote caching) for only certain steps, such as building pexes via remote execution but build the docker containers locally (also useful because as @fast-nail-55400 has mentioned in another thread, docker caches are machine-specific, so it may make sense to optionally turn off remote execution for docker builds; I was doing the buildfarms locally with a mounted
/var/run/docker.sock
so it was using my local machine anyway).
f

fast-nail-55400

03/10/2022, 3:39 PM
...it could be I’m doing something strange, but given that it works on 2/3 build systems but fails on the third is interesting. Wanted to at least raise it for awareness but I’m not sure if I should file an issue given that I’m not sure which product the issue is with!
It would be helpful if you could open an issue with a reproducible example triggering the error along with a Pants debug log (via
-ldebug
). (The debug log will have the command-line/environment for the “action” being executed remotely.) Sometimes these servers have subtle differences in the remote execution environment that contribute to the failure. It’s been known to happen in other cases that works on 2/3 and fails on the other. (See https://github.com/pantsbuild/pants/issues/13170 for an example of how thorny an issue can be sometimes.)
e

echoing-farmer-15630

03/10/2022, 3:42 PM
I'll try and do so; as with all development environments time is somewhat tight and I'm not really being encouraged to experiment with build farms 🙂 But that should be doable I think.
f

fast-nail-55400

03/10/2022, 3:47 PM
just even opening the issue with an example of the failing target and the Pants invocation would be very helpful. but yeah, time constraints abound everywhere!. even with a minimal reproduction, looking into it would be lower priority for me (unless someone were actively blocked by it), given how involved debugging these are.
One of the buildfarm folks raised the idea that it may be worthwhile to add something to
pants
to disable remote execution (and maybe remote caching) for only certain steps
There have been some thoughts along that line, mostly in the context of configuring execution platform. A simpler first version would be an on-off switch like you mention.
e

echoing-farmer-15630

03/10/2022, 4:17 PM
Issue started as https://github.com/pantsbuild/pants/issues/14755; hope it's remotely useful!
f

fast-nail-55400

03/10/2022, 4:45 PM
“remotely” useful — nice 🙂
🤣 1