https://pantsbuild.org/ logo
Join Slack
Powered by
Hi maintainers, I fail to build the universal pex...
# development
c
Hi maintainers, I fail to build the universal pex for 2.14.x and 2.15.x 🧵
It worked just fine for 2.17.0.dev and 2.16.x, so I’m not sure what’s a miss here.. rather than getting bogged down, I’d prefer it if this works for some of you to run and upload the pex to GH please? 🙂
Copy code
# 2.14.x 71a4a4527839d710594b39d49f4758c9493e70c6
# 2.15.x 420f842454dd753e019b6ae74b674ff5457358a3
PANTS_PEX_RELEASE=STABLE ./build-support/bin/release.sh build-universal-pex
I would also need someone to update the docs at readme.com for each of 
2.14.x
, 
2.15.x
, 
2.16.x
and 
main
(at 
a39c1b970be2cb4421b20728c21c3222aeb4b015
).
w
so… there was a fix for PEX building that might not have been backported all the way to 2.14.x, but the PEX should actually still get generated
check whether a file is actually generated at the mentioned path, and if so, ignore the error
if not: what’s the error?
c
there was no file in 
dist/deploy/pex
as it was for the other versions that worked error:
Copy code
Fetching <https://binaries.pantsbuild.org/wheels%2F3rdparty%2F71a4a4527839d710594b39d49f4758c9493e70c6%2F2.14.2rc2%2Bgit71a4a452%2Furllib3-1.26.15-py2.py3-none-any.whl>

Fetching <https://binaries.pantsbuild.org/wheels%2F3rdparty%2F71a4a4527839d710594b39d49f4758c9493e70c6%2F2.14.2rc2%2Bgit71a4a452%2Fzipp-3.15.0-py3-none-any.whl>

[=== 00:27 Checking prebuilt wheels for 2.14.2rc2+git71a4a452 ===]
All 2 pantsbuild.pants packages were fetched and are valid.
Wrote whl with version 2.14.2rc2 to /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2/pantsbuild.pants-2.14.2rc2-cp39-cp39-manylinux2014_x86_64.whl.

Wrote whl with version 2.14.2rc2 to /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2/pantsbuild.pants-2.14.2rc2-cp38-cp38-manylinux2014_x86_64.whl.

Wrote whl with version 2.14.2rc2 to /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2/pantsbuild.pants-2.14.2rc2-cp37-cp37m-macosx_10_15_x86_64.whl.

Wrote whl with version 2.14.2rc2 to /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2/pantsbuild.pants.testutil-2.14.2rc2-py3-none-any.whl.

Wrote whl with version 2.14.2rc2 to /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2/pantsbuild.pants-2.14.2rc2-cp37-cp37m-manylinux2014_x86_64.whl.

Wrote whl with version 2.14.2rc2 to /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2/pantsbuild.pants-2.14.2rc2-cp38-cp38-macosx_10_15_x86_64.whl.

Wrote whl with version 2.14.2rc2 to /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2/pantsbuild.pants-2.14.2rc2-cp39-cp39-macosx_11_0_arm64.whl.

Wrote whl with version 2.14.2rc2 to /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2/pantsbuild.pants-2.14.2rc2-cp39-cp39-macosx_10_15_x86_64.whl.

pid 7082 -> /private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/tmph2hfhxjp/venvs/cac1718c056bb509f51fcdcc0c376b33deaaa8ec/26206f86a43095c89a46831693c798eb92a902ac/bin/python -sE /private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/tmph2hfhxjp/venvs/cac1718c056bb509f51fcdcc0c376b33deaaa8ec/26206f86a43095c89a46831693c798eb92a902ac/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --use-deprecated legacy-resolver --isolated -q --cache-dir /private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/tmph2hfhxjp/pip_cache download --dest /private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/tmph2hfhxjp/downloads/resolver_download.1jpofpw_/Users.andreas.stenius..pyenv.versions.3.7.13.bin.python3.7 --only-binary :all: pantsbuild.pants==2.14.2rc2+git71a4a452 --no-index --find-links /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2+git71a4a452 --find-links /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/3rdparty/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2+git71a4a452 --retries 5 --timeout 15 exited with 1 and STDERR:
ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==2.14.2rc2+git71a4a452 (from versions: none)
ERROR: No matching distribution found for pantsbuild.pants==2.14.2rc2+git71a4a452

Traceback (most recent call last):
  File "/Users/andreas.stenius/.cache/pants/named_caches/pex_root/venvs/0a326a02477142002846fbf7d79fe20cd05bfd62/bdfcee4cd90a613a01dad3eee8a4b0670645276b/pex", line 236, in <module>
    runpy.run_module(module_name, run_name="__main__", alter_sys=True)
  File "/Users/andreas.stenius/.pyenv/versions/3.7.13/lib/python3.7/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/Users/andreas.stenius/.pyenv/versions/3.7.13/lib/python3.7/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/Users/andreas.stenius/.pyenv/versions/3.7.13/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/pants-sandbox-gFSMVq/build-support/bin/_release_helper.py", line 1250, in <module>
    main()
  File "/private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/pants-sandbox-gFSMVq/build-support/bin/_release_helper.py", line 1236, in main
    build_pex(fetch=True)
  File "/private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/pants-sandbox-gFSMVq/build-support/bin/_release_helper.py", line 856, in build_pex
    check=True,
  File "/Users/andreas.stenius/.pyenv/versions/3.7.13/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/Users/andreas.stenius/.cache/pants/named_caches/pex_root/venvs/0a326a02477142002846fbf7d79fe20cd05bfd62/bdfcee4cd90a613a01dad3eee8a4b0670645276b/bin/python', '/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/tmpmhrks77y/pex', '-o', 'dist/pants.2.14.2rc2+git71a4a452.pex', '--no-build', '--no-pypi', '--disable-cache', '-f', '/Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2+git71a4a452', '-f', '/Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/3rdparty/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2+git71a4a452', '--no-strip-pex-env', '--console-script=pants', '--python-shebang', '/usr/bin/env python', '--interpreter-constraint', 'CPython>=3.7,<3.10', '--platform=linux_x86_64-cp-37-m', '--platform=linux_x86_64-cp-38-cp38', '--platform=linux_x86_64-cp-39-cp39', '--platform=macosx_11.0_x86_64-cp-37-m', '--platform=macosx_11.0_x86_64-cp-38-cp38', '--platform=macosx_11.0_x86_64-cp-39-cp39', 'pantsbuild.pants==2.14.2rc2+git71a4a452', '--venv']' returned non-zero exit status 1.
similar error for the 2.15.x branch
w
…hm. so it sounds like perhaps one of the wheels expected by
Copy code
'--interpreter-constraint', 'CPython>=3.7,<3.10', '--platform=linux_x86_64-cp-37-m', '--platform=linux_x86_64-cp-38-cp38', '--platform=linux_x86_64-cp-39-cp39', '--platform=macosx_11.0_x86_64-cp-37-m', '--platform=macosx_11.0_x86_64-cp-38-cp38', '--platform=macosx_11.0_x86_64-cp-39-cp39'
…wasn’t generated by the CI job?
c
ah could be, I did notice there was indeed fewer wheels published to pypi during the publish step for the older versions.
w
did the CI job wheel build finish?
c
yes
I didn’t look at the logs, but all tasks where green for the Version Bump job
oh.. wait there were jobs also for the pushed release tag, didn’t see them. so that could’ve failed if those are the ones that count
ah, that would’ve affected the tag to commit mapping.. and was failed for 2.14 but ok for 2.15
re-run the publish-tag-to-commit-mapping for 2.14 still fails, so 🤷 ? https://github.com/pantsbuild/pants/actions/runs/4782211544
(but unrelated, I think)
w
hm. were there backports to the branch that might have caused something to fail?
c
nothing that stands out to me.. you can see the PRs in the changelog
e
This is old and well known and we keep forgetting. Mac M1 is a no-go for this until Benjy fixed it in 2.16.x - right?:
Copy code
/private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/tmph2hfhxjp/downloads/resolver_download.1jpofpw_/Users.andreas.stenius..pyenv.versions.3.7.13.bin.python3.7 --only-binary :all: pantsbuild.pants==2.14.2rc2+git71a4a452 --no-index --find-links /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2+git71a4a452 --find-links /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/3rdparty/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2+git71a4a452 --retries 5 --timeout 15 exited with 1 and STDERR:
ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==2.14.2rc2+git71a4a452 (from versions: none)
ERROR: No matching distribution found for pantsbuild.pants==2.14.2rc2+git71a4a452
That's trying to use Python 3.7 
Users.andreas.stenius..pyenv.versions.3.7.13.bin.python3.7
to download a wheel for 
pantsbuild.pants==2.14.2rc2+git71a4a452
on an M1. Good luck with that since we only build 3.9 for M1.
@curved-television-6568 I'll do the docs publishing and PEX building / uploading here shortly for the releases missing these things.
👍 1
c
Dang. I knew it blew on something last time around. Obviously forgot on what..
Thanks @enough-analyst-54434
e
You're welcome. I'll accept payment in annoying you all by pointing out its been how many years since Apple Silicon and still no non-janky affordable CI for it.
💸 1
Ditch your damn macs!
c
it’s the one I’ve got (from $ work).. my private alternative is a Win box 😮 (haven’t had one in years, so I feel lost on that one 😅 )
could of course run *nix on it tho…
e
Exactly. That's what I did.
c
but… I’d say it’s a small win to have me ditch it as long as there are users out there using it.. we might as well have some early feedback if we break for that platform somehow..
e
Well, yes - there is the enabler problem. You have to stick your neck out and hope alot of people follow you; otherwise there's always "but other people" - which is sadly valid. Its Molochian.
Hopefully other corps step up and start pitching into the Pantsbuild non-profit bank account to pay for this stuff.
💯 1
w
but… we do build M1 wheels in CI, don’t we? it shouldn’t matter what the client machine looks like?
this is the M1 wheel build for the head commit of the 2.14.x branch https://github.com/pantsbuild/pants/actions/runs/4780507287/jobs/8500427358
e
Sure - someone needs to fix 2.14 and 2.15 then
Not worth it. Benjy did it for 2.16 +
This had been punted for ~2 years until he did.
Now we're all just confused and forgetting history.
w
most definitely. but… i’m not understanding how the client machine matters here
all of these are downloaded wheels, right?
e
We have a buggy script.
Benjy fixed the bugs.
w
got it. and the bug only impacts M1 clients?
e
Your curiosity is too late!
Yes.
w
got it.
e
Eric was the primary affected releaser for a long time.
There used to be a nag in the script for M1 users.
Benjy was the one to step up and really fix finally.
w
i still don’t have an M1, so wouldn’t have seen it.
e
Exactly.
I actually have that wrong - there were 2 sets of M1 issues. Building the wheel fixed by Macstadium bucks and then issues with the pants.pex. But at any rate.
w
e
Hey @curved-television-6568 what's up with naimg the releases 
s/real tag name/v.../
?Is this a new thing I should be doing too? It seems confusing to have a release named after not any tag or any version Pants uses.
@witty-crayon-22786 looks like it.
w
k: picking it
e
I think an M1 user should do that to actually test its worth doing so.
w
e
All the picking we do is - insane IMO.
Too much
w
it would be less sane to leave releasing broken for two of our releasers from branches which are actively being released?
e
Yes
You have no clue your pick fix is the fix.
w
@curved-television-6568: can you confirm that the edit in #18817 fixes PEX building for you?
e
It should be done if done at all by an M1 user and verified to work.
Sometimes automation is not the answer. Post it note - just ask for help for these 2 old and dying brnaches. Seems simple to me.
@curved-television-6568 I left the 2.14 amd 2.15 releases with their native release tag. Feel free to edit if that's the new thing or edit yours back if it's not.
Ok @curved-television-6568 the reference re-gen + doc sync is complete for all 4 release tags.
c
Thanks! It was the release title only, so for display purposes and did not affect download paths or other. But regardless, I removed it to be consistent with previous releases.
@witty-crayon-22786 confirmed fix works for me. I get another issue later, but the universal pex is in the dist/deploy/pex folder.
e
Yeah - the test failure - right?
I still contend it is much much more sane to just add a note to the docs for 2.14.x and 2.15.x. There are many more processes than this that require asking for help for some releasers and these branches are dead.
c
yea, smoke test running 
pants.pex --version
w
@enough-analyst-54434: 2.15.x is not dead… this is only it’s second stable release
c
well, with this I can navigate most of it 😉
e
Ok - I'll desist. I cannot grok Pantsbuild well any longer.
c
I consider the amount of bespoke scripts in build-support as a metric for missing pants features, fwiw
e
@curved-television-6568 no. I needed to do docs and pypi deletes
The process is only anywhere near smooth for Stu Benjy and I
IIUC
c
@enough-analyst-54434 yes, but those are tied to permissions rather than build issues
e
Everyone else has plenty of roadbumps.
c
yup, will want to look at streamlining this one piece at a time.
👍 1
w
have landed those picks.
c
I’ll add picks for 18477 too
2 Views
Open in Slack
PreviousNext
Powered by