https://pantsbuild.org/ logo
Join Slack
Powered by
<@U04S45AHA> there's a CI failure in my PR <https:...
# pex
a
@enough-analyst-54434 there's a CI failure in my PR https://github.com/pantsbuild/pants/pull/6022 in the osx shard where pex is failing to resolve locally-built dists in 
PythonRun
only, with `Failed to execute PEX file, missing macosx_10_12_x86_64-cp-27-cp27m compatible dependencies for: my local dist`: see https://travis-ci.org/pantsbuild/pants/jobs/400569783. this does not repro on my osx laptop. in 
SetupPyRunner
we're tagging the locally-built dist with 
--plat-name
set to 
get_local_platform()
if there are native sources. 
get_local_platform()
has a TODO linking to this issue: https://github.com/pantsbuild/pex/issues/511, and it seems reasonable to think these are related, but it's not clear to me what would cause this resolution failure.
afaict 
SetupPyRunner
is being invoked with the 
interpreter
from 
self.context.products.get_data(PythonInterpreter)
, and that is also used to create the pex that is consumed in 
PythonRun
. i tried setting 
--pre
in the 
WrappedPex#cmdline()
method, which did not seem to resolve the issue (nor did blasting caches), but especially since this is only on the osx shard i would suspect the platform is different somehow.
any thoughts just on how the platforms might be out of sync is all i'm looking for when you have the time, thanks
also, I wrote the SetupPyRunner part about a week ago, so that may be suspect
the catch is that this failure doesn't occur when the 
--tag-build
is not used for local dists though (as is done on master)
e
I'll ignore platform differences for a moment. So the resolution for the PythonRun pex requirements happens way back in pyprep.requirements`. That is where allow_prereleases would be needed.
But just always 
allow_prereleases
whenever there are 
python_dists
in play is incorrect.
a
ok, i'm looking through 
ResolveRequirementsTaskBase
now. why would this resolution error only show up when invoking the pex (as opposed to when building it)?
and i understand the point about not having 
allow_prereleases
on all the time, will figure out how to ensure it's only to resolve just the local dists
e
why would this resolution error only show up when invoking the pex (as opposed to when building it)?
Agreed. Thinking more.
a
locally on osx, we are detecting correctly whether the target set has any native sources in 
resolve_requirements()
(and setting the wheel platform), and i don't think that would be different in travis
e
I do not know off the top of my head but can check: are you sure this is a valid version?: 1.0.0+dcffd59dfb65eb2abd72f11322448606a7c76dc6.defaultfingerprintstrategy.da39a3ee5e6b.897870053801
In python setuptools eyes?
I know +[a-za-Z]+ is ok
Not sure about dots
a
there is a page i have open explaining legal tagged builds -- that's what the raw fingerprint passed to 
--tag-build
is converted to when making the dist by distutils. i will find that page
e
looks ok
a
i doubt case sensitivity would affect this, not in this way
e
The thing to do here then is to 1. edit .travis.yml on your branch down to just the 1 shard and 2. add PEX_VERBOSE=5 into the mix. Easiiest way would be to modify the .travis.yml script line and prepend that I think.
a
got it
e
The pex verbose will show us what is scanned (and rejected) for that dep
a
doing that now, will post a link when ci is up. i'll ping you again when i've figured it out or figured out that i can't
grr, PEX_VERBOSE wasn't passed down to the subprocess. i think i know how to do that though dw
e
OK, slowing down. Its an intg that is failing - some of those run 
hermetic
which scrubs env
a
yeah
e
Perhaps target the failing test and edit it
a
right there with you
also removed the rust testing so it's faster this time lol
so if you search 
test_pants_run
in this log you'll see the pex verbose output: https://api.travis-ci.org/v3/job/400607859/log.txt. one thing i think is fine but want to verify is that the wheel filename ends in 
-cp27-cp27m-macosx_10_12_x86_64.whl
, but pex is looking for the platform 
macosx_10_12_x86_64-cp-27-cp27m
-- i'm assuming that's just a matter of display and not identity, but maybe not?
it does show that the dist we want is contained in the pex, but it doesn't satisfy the dep (for that exact dist, but without a version tag)
lots of output that i'm going through now so need to context switch yet
there is 
pex: Please build pex with the subprocess32 module for more reliable requirement installation and interpreter execution.
-- i'm not seeing this locally
...i am, actually, but not on all locally-built dists with native code, just the one that's failing on travis
PEX_VERBOSE=5 ./pants -ldebug run examples/src/python/example/python_distribution/hello/fasthello:main
that's a lie, it's on all locally-built dists, this is not different from ci, sorry
the test in ci is using 2.7.10 instead of 2.7.14, otherwise the fingerprints/paths are the same
everything else is using 2.7.10 as well though it seems (declared as such in the pytest output). still looking
the timings are cut off for some weird reason at the "searching dependency cache" line in which it should have found the dist: 
pex: Activating PEX virtual environment from /Users/travis/build/pantsbuild/pants/.pants.d/tmp/tmp75lcmR.pants.d/pyprep/requirements/CPython-2.7.10/24a44c0071b9e805fe7b2172be7b3e498b6f69e1-DefaultFingerprintStrategy_8c9b6e40089d :: Searching dependency cache: /Users/travis/build/pantsbuild/pants/.pants.d/tmp/tmp75lcmR.pants.d/pyprep/requirements/CPython-2.7.10/24a44c0071b9e805fe7b2172be7b3e498b6f69e1-DefaultFingerprintStrategy_8c9b6e40089d/.deps
and it's all on one line, it should be two and have 
: 2.0ms
or so at the end of both
"should have found the dist" meaning "it does that on my laptop":
Copy code
pex: Activating PEX virtual environment from /Users/dmcclanahan/tools/pants/.pants.d/pyprep/requirements/CPython-2.7.14/f14bea5cea130765649dd5c2465d9be7af894484-DefaultFingerprintStrategy_8c9b6e40089d: 4.6ms                                                                                                                                       
pex:   Searching dependency cache: /Users/dmcclanahan/tools/pants/.pants.d/pyprep/requirements/CPython-2.7.14/f14bea5cea130765649dd5c2465d9be7af894484-DefaultFingerprintStrategy_8c9b6e40089d/.deps: 2.0ms
pex:     Adding fasthello 1.0.0+b2746123820ccd47afb2e520476673e3aea8a39a.defaultfingerprintstrategy.da39a3ee5e6b.897870053801: 0.2ms
pex:   Resolving fasthello==1.0.0+b2746123820ccd47afb2e520476673e3aea8a39a.defaultfingerprintstrategy.da39a3ee5e6b.897870053801: 0.7ms
pex:   Activating fasthello 1.0.0+b2746123820ccd47afb2e520476673e3aea8a39a.defaultfingerprintstrategy.da39a3ee5e6b.897870053801: 0.9ms
pex:     Adding sitedir: 0.5ms
pex: Activating PEX virtual environment from /Users/dmcclanahan/tools/pants/.pants.d/pyprep/sources/b416569e036d849d23748fcd2cd6aab6409ccc01-DefaultFingerprintStrategy_4645322e1ea7 pex: Activating PEX virtual environment from /Users/dmcclanahan/tools/pants/.pants.d/pyprep/sources/b416569e036d849d23748fcd2cd6aab6409ccc01-DefaultFingerprintStrategy_4645322e1ea7: 0.6ms
and the interpreter it's using at 
/System/Library/Frameworks/Python.framework/Versions/2.7/bin/python2.7
is 
2.7.10
on this machine, which lines up (everything else has 2.7.10 in the path names, so that's correct)
thinking that what 
--plat-name
wants may not be the exact platform name (???), looking at https://stackoverflow.com/questions/35112511/pip-setup-py-bdist-wheel-no-longer-builds-forced-non-pure-wheels#36886459. looking for release notes between 2.7.10 vs 2.7.14 to see if there's a difference in behavior (or i could just try invoking 2.7.10 locally...)
looking in the 
wheel
source, the 
--plat-name
argument is definitely incorrect, so i'll try fixing that and seeing what happens
https://github.com/pypa/wheel/blob/master/wheel/bdist_wheel.py (it uses a 
get_platform()
method defined in an adjacent module)
e
Yeah pex_uitil.get_local_platform is definitely not what you want
The release.sh script does this right
a
i'll check it
that's great to hear because i was just diving into cpython source again
e
I included the link in a comment when I figured out the right way.
This explains why osx and not linux
This should fail on all OSX
a
that was my concern as well, i'm going to blast all of the caches i know of and try again (after making the platform change)
thanks a lot
pex's platform method describes a nuance with MACOSX_DEPLOYMENT_TARGET which i will address after we can figure out how to build anything at all
it's definitely different (it ends with 
_intel.whl
now) so this may be the right track
tossing it up into ci in a sec
e
From all the side reading on this I'm pretty positive local version identifiers don't require allow_prereleases
pre and local are different beasts
So that whole potential problem is not a problem
a
that makes sense, 
--pre
was 100% a shot in the dark
yes
come to think of it i should have realized 
--plat-name
was almost definitely the first thing to look into, lesson learned. ci up at https://travis-ci.org/pantsbuild/pants/builds/400643859?utm_source=github_status&amp;utm_medium=notification
i also used the name 
Platform
for platform-specific native code stuff, was going to change that anyway and realizing the current naming ambiguity that i wasted time on gives me more impetus to change that name too
e
pex.environment.Environment
- which is relevant erroring at runtime bit here, logs platform tags it thinks it supports at level 9
So maybe try againt with PEX_VERBOSE=9
a
i will try that -- i was seeing a wheel ending in 
macosx_10_13_intel.whl
locally, but the remote version was still showing the old 
macosx_10_12_x86_64.whl
, so i thought blasting the cache might work (haven't rebuilt since then). pushing the verbosity increase now
e
Remote fixes osx version down
Which should explain 10.12 vs 10.13
a
yeah sorry that was not the concern
e
oh, intel vs x86_64
gotcha
a
but locally the wheel was named 
_x86_64
before the platform change, now it's intel, yeah
e
May be worth the next shot - using that util.
a
hmmmmm thanks, i will try that
(still fails using the distutils platform, looking through the output now)
and haven't used the pex util you just linked yet
i actually just restarted that build and realized that would clear it for you too oops
travis for the PR on pantsbuild/pants isn't running yet, which is fine
speak of the devil
also going to add 
-ldebug
to the pants invocation in the test, should have done that earlier
ok so it still fails, but i also noticed that there's this "Translating <the whl we care about> into dist" section in travis, but not locally (i grepped for 
trans
in the stderr), so that seems highly relevant. going to push a commit with debug on to see what the produced wheel file's name is (because it's still looking for 
_x86_64
in ci)
i built python 2.7.10 so i could see if that was involved with the whl name changing but i can't figure out how to make pants accept that
sorry, haven't looked, not can't figure out -- i meant that i am looking
yep found
using 
pex.pep425tags.get_platform()
returns it to being 
_x86_64.whl
, but the same error persists. i'm clearing the caches and rebuilding, i don't expect that to help.
Open in Slack
PreviousNext
Powered by