https://pantsbuild.org/ logo
#development
Title
# development
h

hundreds-breakfast-49010

12/11/2019, 12:40 AM
how are we currently using
MultiPlatformExecuteProcessRequest
and the `Platform`/`PlatformConstraint` types?
h

hundreds-father-404

12/11/2019, 12:46 AM
https://github.com/pantsbuild/pants/blob/2d6001acd642c7b1ba695fdf66ba3764424f4868/src/python/pants/backend/python/rules/pex.py#L122-L135 For example, when running any rule that depends on the
create_pex
rule, e.g.
./pants test
, remote execution won’t work from a mac because the host platform != remote platform
h

hundreds-breakfast-49010

12/11/2019, 12:51 AM
okay, so any time we create a pex via that rule, we have access to a
Platform
input
h

hundreds-father-404

12/11/2019, 12:51 AM
I believe. cc @early-needle-54791 , who designed this
e

early-needle-54791

12/11/2019, 12:52 AM
Yeah. Platform is either a singleton rule or a RootRule, I forget which off the top of my head. But either way its always available.
h

hundreds-breakfast-49010

12/11/2019, 12:53 AM
this is in the contex of running a test locally iff we have a flag passed to it - we're trying to make sure the machinery to build a pytest pex locally becuase we're going to run it locally is avaiable
or add it if it doesn't exist
e

early-needle-54791

12/11/2019, 12:55 AM
The create_pex rule should already only support the current platform using Platform.current. The MPEPR only has one field and it defaults to the current platform.
I think I understand your question.
One open issue right now is the Pex ExecuteProcessResults don’t list which platform they were built for, so we can’t prevent a pex built for a different platform from being included in the one built locally atm.
h

hundreds-breakfast-49010

12/11/2019, 1:04 AM
so maybe we need to add that
when we create a pex, we have a
Platform
, but if we try to retrieve that
Pex
from the cache later, nothing is stopping us from using it on a different platform than what it was built for, right?
well, except that a specific
Platform
is an input to the rule
@hundreds-father-404 is that remote execution of a pex on a mac usecase currently broken, or was this added in order to successfully fix it?
h

hundreds-father-404

12/11/2019, 1:14 AM
The mac usecase of not being able to remote is technically the correct thing to do, but it’s frustrating in reality. It means that from my laptop that I can never remote Python tests, unless Toolchain Build farm or Google RBE start having macOS servers..
h

hundreds-breakfast-49010

12/11/2019, 1:16 AM
what's stopping a remote build from building a pex with its own platform, rather than the local platform?
h

hundreds-father-404

12/11/2019, 1:18 AM
what’s stopping a remote build from building a pex with its own platform, rather than the local platform?
sdist wheels. You can’t build a macOS-compatible wheel from a Linux machine. This is why Pants CI has both a
Linux Build Wheels
shard and an
OSX Build Wheels
shard
Hm one possible solution! If the
MultiplatformEPR
can’t be resolved on the remote platform, then try the host and use local execution. Tangibly, for Python rules run on macOS, build the Pex locally, but then upload that Pex to remote execution for the later rules like invoking Pytest -- Hm, just kidding. That macOS-specific Pex would not work properly on Linux remote machines 😞
h

hundreds-breakfast-49010

12/11/2019, 1:30 AM
so right now we don't have the infrastructure to request an
ExecuteProcessResult
for a specific platform, right? we can request a
Pex
for a specific platform, I think, because the rule that creates it has a
Platform
input
and
Pex
-building will use MultiplatformEPR not EPR
with that constraint that the execution platform has to be the same as the target platform
w

witty-crayon-22786

12/11/2019, 4:47 PM
unfortunately we needed to put that work down a while ago to focus on JVM remoting.
but Henry's point is important: there is some design work that needs to happen to allow for differentiating a "linux output" from an "OSX output"... the only way to do it currently is with Params
which you would need to do explicitly beforehand.