https://pantsbuild.org/ logo
Powered by
e

echoing-farmer-15630

03/10/2022, 3:42 PM
Are the artifacts created by a pex build available to a docker step? What I mean is that if a 
docker_image
target depends on a 
pex_binary
target, could one tag the 
pex_binary
targets with one tag and the 
docker_image
tags with another, build the 
pex_binary
targets with remote execution on and then use those generated artifacts to build the 
docker_image
targets locally without recreating the pex targets?
h

happy-kitchen-89482

03/10/2022, 3:48 PM
Generally if your 
docker_image
target depends on a 
pex_binary
then when running 
package
on the 
docker_image
, the 
pex_binary
will be packaged first, and the resulting .pex file will be available to COPY into the 
docker_image
(in fact if the Dockerfile does COPY the .pex file then Pants can even infer the dependency between the two targets for you!)
But I guess you already knew that, and are asking how this works cross-platform?
e

echoing-farmer-15630

03/10/2022, 3:50 PM
Right--and I use that inference (and it works well). What I was wondering is if I explicitly build the 
pex_binary
first (using remote execution) and then build the consuming 
docker_image
without using remote execution, does it avoid rebuilding the 
pex_binary
?
h

happy-kitchen-89482

03/10/2022, 4:02 PM
Aha, so this is uncertain territory for me, I'll go do some reading in the code.
By "explicitly" you mean in a separate Pants invocation, in order to populate the cache with the .pex file?
And then a second invocation that should use that cache entry?
e

echoing-farmer-15630

03/10/2022, 4:05 PM
Right, exactly--the idea is that you could use remote execution to build pexes but use local docker to build docker files, for example. Since the docker cache is local, this may make sense. But it may not. I'm not even clear if docker images work nicely with remote execution and properly send back the docker image and register it in the cache; I haven't gotten as far as an actually remote remote execution (buildfarm running locally)
h

happy-kitchen-89482

03/10/2022, 4:09 PM
Hmm, so if the local system is the same platform as the remote system (say linux_x86_64) then I think this would work
And will, when run on MacOS, request MacOS pexes
@curved-television-6568 do I have that right?
Unless the 
pex_binary
target explicitly specifies its platform as only linux_x86_64, I guess
e

echoing-farmer-15630

03/10/2022, 4:19 PM
That's probably fair, really. Almost darn but not quite; that's probably a reasonable and expectable behaviour, but I'm not sure what the "right answer" for pex-on-docker is really. Still need to try my horrific "local buildfarm" hack to see if it works.
h

happy-kitchen-89482

03/10/2022, 4:21 PM
I think since that code I linked to knows that the binary is going to be embedded into a docker image, and it knows the platform for that image, it might make sense for it to override the default local platform. That seems to model what the user expects...
If it did that then your idea should work, I believe
c

curved-television-6568

03/10/2022, 4:28 PM
do I have that right?
Uh, I’m not sure. Probably? 😬 I’ve not looked for a way to specify the target platform for the pex when requesting it.. that should be a good thing to do there.
h

happy-kitchen-89482

03/10/2022, 4:34 PM
Right now it looks like we'd build a MacOS pex and then blindly embed it in a Docker image, with nonsensical results... 🙂
🤔 1
c

curved-television-6568

03/10/2022, 5:28 PM
… unless the pex explicitly includes the 
platforms
field, which most likely would be required in order for it to work when building on a Mac. See: https://github.com/pantsbuild/example-python/pull/78/files#diff-5274b0bd4e38bf378e72565a83cb05861f88932be009e9e0b134a456c9adc430R28
w

witty-crayon-22786

03/10/2022, 6:32 PM
more generally, cross platform remote execution is covered here: https://github.com/pantsbuild/pants/issues/11148 … it’s very similar to cross-compiling with a docker image: https://github.com/pantsbuild/pants/issues/13682
e

echoing-farmer-15630

03/10/2022, 9:54 PM
That's the path I'm setting up now (REAPI to build pexes and dockerfiles correctly) although it's a blatant hack using a local buildfarm sharing the 
/var/run/docker.sock
file. Hilarious... but it may be working; need to get one of my OSX devs to try it.
...yeeeah, that didn't work so well (the docker on Mac system doesn't use groups the same way, so running the container as root creates files with root permissions in the user's directory, but running the container as the user creates issues getting access to the bind-mounted 
/var/run/docker.sock
... because Docker Desktop on OSX does permissions differently as it runs in a VM so matching group IDs doesn't make as much sense. Everything gotta be hard. Going to ask our DevOps OSX expects how to handle the group permissions.
w

witty-crayon-22786

03/17/2022, 4:17 PM
honestly, i think that until https://github.com/pantsbuild/pants/issues/13682 (or #11148) are implemented, your best bet is going to be pre-building and stashing any native wheels you have
because cross-building from macOS to Linux via remoting is very likely to pollute the remote environment with local environment until #11148 is implemented
Powered by