https://pantsbuild.org/ logo
#pex
Title
# pex
w

witty-crayon-22786

04/10/2020, 5:26 PM
what might cause the exact same pex command targeting explicit platforms to fail on one machine vs another (full disclosure: both machines have an /etc/pexrc file, but they're identical):
Copy code
pex --no-build --no-pypi --disable-cache -f /tmp/tmp.lZIeuxe9PT/dist/deploy/wheels/pantsbuild.pants/e789654e53bd346f7d3604402f13aca03f53429d/1.26.0rc1+gite789654e -f /tmp/tmp.lZIeuxe9PT/dist/deploy/wheels/3rdparty/e789654e53bd346f7d3604402f13aca03f53429d/1.26.0rc1+gite789654e -o /tmp/tmp.lZIeuxe9PT/dist/pants.1.26.0rc1+gite789654e.py36.pex -vvvvvvvvv --no-emit-warnings --no-strip-pex-env --script=pants '--interpreter-constraint=CPython==3.6.*' --platform=linux_x86_64-cp-36-m --platform=macosx_10.11_x86_64-cp-36-m pantsbuild.pants==1.26.0rc1+gite789654e pantsbuild.pants.contrib.confluence==1.26.0rc1+gite789654e pantsbuild.pants.contrib.go==1.26.0rc1+gite789654e pantsbuild.pants.contrib.mypy==1.26.0rc1+gite789654e pantsbuild.pants.contrib.node==1.26.0rc1+gite789654e pantsbuild.pants.contrib.python.checks==1.26.0rc1+gite789654e pantsbuild.pants.contrib.scalajs==1.26.0rc1+gite789654e pantsbuild.pants.contrib.scrooge==1.26.0rc1+gite789654e pantsbuild.pants.contrib.avro==1.26.0rc1+gite789654e pantsbuild.pants.contrib.googlejavaformat==1.26.0rc1+gite789654e pantsbuild.pants.testutil==1.26.0rc1+gite789654e
from the release script on
0697d3a29a9
using pex 2.1.7... succeeds on my OSX laptop, with/without
PEX_IGNORE_RCFILES=true
fails on a centos6 box
on the centos6 box,
-vvvvvvvvv
indicates
Copy code
Skipping link: none of the wheel's tags match: cp36-abi3-linux_x86_64: <a href='file:///tmp/tmp.NDLBCVNB4R/dist/deploy/wheels/pantsbuild.pants/e789654e53bd346f7d3604402f13aca03f53429d/1.26.0rc1%2Bgite789654e/pantsbuild.pants-1.26.0rc1%2Bgite789654e-cp36-abi3-linux_x86_64.whl'>file:///tmp/tmp.NDLBCVNB4R/dist/deploy/wheels/pantsbuild.pants/e789654e53bd346f7d3604402f13aca03f53429d/1.26.0rc1%2Bgite789654e/pantsbuild.pants-1.26.0rc1%2Bgite789654e-cp36-abi3-linux_x86_64.whl</a>
Skipping link: none of the wheel's tags match: cp36-abi3-macosx_10_11_x86_64: <a href='file:///tmp/tmp.NDLBCVNB4R/dist/deploy/wheels/pantsbuild.pants/e789654e53bd346f7d3604402f13aca03f53429d/1.26.0rc1%2Bgite789654e/pantsbuild.pants-1.2`6.0rc1%2Bgite789654e-cp36-abi3-macosx_10_11_x86_64.whl'>file:///tmp/tmp.NDLBCVNB4R/dist/deploy/wheels/pantsbuild.pants/e789654e53bd346f7d3604402f13aca03f53429d/1.26.0rc1%2Bgite789654e/pantsbuild.pants-1.26.0rc1%2Bgite789654e-cp36-abi3-macosx_10_11_x86_64.whl</a>
which results in not finding the root wheels
underlying failing pip command was:
Copy code
/usr/bin/python3.6 /tmp/tmpivdin432 --disable-pip-version-check --isolated --no-python-version-warning -vvv --cache-dir /tmp/tmpitoelml7 download --dest /tmp/tmp4053f_su/resolved_dists/cp36-cp36m --no-index --find-links /tmp/tmp.lZIeuxe9PT/dist/deploy/wheels/pantsbuild.pants/e789654e53bd346f7d3604402f13aca03f53429d/1.26.0rc1+gite789654e --find-links /tmp/tmp.lZIeuxe9PT/dist/deploy/wheels/3rdparty/e789654e53bd346f7d3604402f13aca03f53429d/1.26.0rc1+gite789654e --only-binary :all: pantsbuild.pants==1.26.0rc1+gite789654e pantsbuild.pants.contrib.confluence==1.26.0rc1+gite789654e pantsbuild.pants.contrib.go==1.26.0rc1+gite789654e pantsbuild.pants.contrib.mypy==1.26.0rc1+gite789654e pantsbuild.pants.contrib.node==1.26.0rc1+gite789654e pantsbuild.pants.contrib.python.checks==1.26.0rc1+gite789654e pantsbuild.pants.contrib.scalajs==1.26.0rc1+gite789654e pantsbuild.pants.contrib.scrooge==1.26.0rc1+gite789654e pantsbuild.pants.contrib.avro==1.26.0rc1+gite789654e pantsbuild.pants.contrib.googlejavaformat==1.26.0rc1+gite789654e pantsbuild.pants.testutil==1.26.0rc1+gite789654e
... i have a suspicion that it is still somehow the
/etc/pexrc
other important data: this seems to have stopped working between 2.1.6 and 2.1.7
is the python that is used to subprocess pip relevant?
e

enough-analyst-54434

04/10/2020, 5:45 PM
OK - few things off the top to try and clear up basic confusion: Your command shows:
... --interpreter-constraint=CPython==3.6.* --platform=linux_x86_64-cp-36-m ...
That's saying resolve for all local interpreters Pex can find (subject to pexrc if any) that are
CPython==3.6.*
(for me this would be 3) and also resolve for 1 foreign platform, so 4 independent resolves for me, any of which could fail independently. Was this already understood?
w

witty-crayon-22786

04/10/2020, 5:47 PM
...
no, not at all.
e

enough-analyst-54434

04/10/2020, 5:47 PM
Barging ahead though, given above, for local resolves, the correct local interpreter is used. For foreign resolves, some interpreter is used, but its not important which, since the resolve is foreign (ie building sdists is turned off and foreign resolves are binary wheel download only).
OK. I'll stop there. Let me know if I should explain more about something.
oy. so that's been wrong a while.
thanks!
e

enough-analyst-54434

04/10/2020, 5:49 PM
Well, it looks like its wrong for release mode. I think there is a mode where trying to build for the local python is correct?
Yeah:
Copy code
-p  Build a pex from prebuilt wheels for this release.
 -q  Build a pex which only works on the host platform, using the code as exists on disk.
w

witty-crayon-22786

04/10/2020, 5:49 PM
yea, possible.
build
vs
fetch
in that bash function
ok, thank you... i can follow up on that!
e

enough-analyst-54434

04/10/2020, 5:51 PM
Excellent.
w

witty-crayon-22786

04/10/2020, 7:13 PM
so, one thing that is clear is that the
fetch
mode should not be using the
interpreter-constraints
at all.
but should
build
? the default would be to build for the
current
platform, but that's not going to constrain to python3
what you said about "building for the three python interpreters" is... not strictly what we really want. just one, presumably
e

enough-analyst-54434

04/10/2020, 7:15 PM
Using interpreter constraints with Pex is almost never what you want. If we already have a proper python picked out by the script, we should just use it. That can be done with
--python=[path to that python]
The --interpreter-constraints feature was added to Pex to support Pants only and seems like a general misfeature.
w

witty-crayon-22786

04/10/2020, 7:17 PM
and we do. ok.
e

enough-analyst-54434

04/10/2020, 7:17 PM
Excellent.
w

witty-crayon-22786

04/10/2020, 7:19 PM
thanks a lot!