https://pantsbuild.org/ logo
Join Slack
Powered by
how are we currently using `MultiPlatformExecutePr...
# development
h
how are we currently using 
MultiPlatformExecuteProcessRequest
and the `Platform`/`PlatformConstraint` types?
h
https://github.com/pantsbuild/pants/blob/2d6001acd642c7b1ba695fdf66ba3764424f4868/src/python/pants/backend/python/rules/pex.py#L122-L135 For example, when running any rule that depends on the 
create_pex
rule, e.g. 
./pants test
, remote execution won’t work from a mac because the host platform != remote platform
h
okay, so any time we create a pex via that rule, we have access to a 
Platform
input
h
I believe. cc @early-needle-54791 , who designed this
e
Yeah. Platform is either a singleton rule or a RootRule, I forget which off the top of my head. But either way its always available.
h
this is in the contex of running a test locally iff we have a flag passed to it - we're trying to make sure the machinery to build a pytest pex locally becuase we're going to run it locally is avaiable
or add it if it doesn't exist
e
The create_pex rule should already only support the current platform using Platform.current. The MPEPR only has one field and it defaults to the current platform.
I think I understand your question.
One open issue right now is the Pex ExecuteProcessResults don’t list which platform they were built for, so we can’t prevent a pex built for a different platform from being included in the one built locally atm.
h
so maybe we need to add that
when we create a pex, we have a 
Platform
, but if we try to retrieve that 
Pex
from the cache later, nothing is stopping us from using it on a different platform than what it was built for, right?
well, except that a specific 
Platform
is an input to the rule
@hundreds-father-404 is that remote execution of a pex on a mac usecase currently broken, or was this added in order to successfully fix it?
h
The mac usecase of not being able to remote is technically the correct thing to do, but it’s frustrating in reality. It means that from my laptop that I can never remote Python tests, unless Toolchain Build farm or Google RBE start having macOS servers..
h
what's stopping a remote build from building a pex with its own platform, rather than the local platform?
h
what’s stopping a remote build from building a pex with its own platform, rather than the local platform?
sdist wheels. You can’t build a macOS-compatible wheel from a Linux machine. This is why Pants CI has both a 
Linux Build Wheels
shard and an 
OSX Build Wheels
shard
Hm one possible solution! If the 
MultiplatformEPR
can’t be resolved on the remote platform, then try the host and use local execution. Tangibly, for Python rules run on macOS, build the Pex locally, but then upload that Pex to remote execution for the later rules like invoking Pytest -- Hm, just kidding. That macOS-specific Pex would not work properly on Linux remote machines 😞
h
so right now we don't have the infrastructure to request an 
ExecuteProcessResult
for a specific platform, right? we can request a 
Pex
for a specific platform, I think, because the rule that creates it has a 
Platform
input
and 
Pex
-building will use MultiplatformEPR not EPR
with that constraint that the execution platform has to be the same as the target platform
w
unfortunately we needed to put that work down a while ago to focus on JVM remoting.
but Henry's point is important: there is some design work that needs to happen to allow for differentiating a "linux output" from an "OSX output"... the only way to do it currently is with Params
which you would need to do explicitly beforehand.
Open in Slack
PreviousNext
Powered by