https://pantsbuild.org/ logo
c

curved-television-6568

04/24/2023, 1:05 PM
Hi maintainers, I fail to build the universal pex for 2.14.x and 2.15.x 🧵
It worked just fine for 2.17.0.dev and 2.16.x, so I’m not sure what’s a miss here.. rather than getting bogged down, I’d prefer it if this works for some of you to run and upload the pex to GH please? 🙂
Copy code
# 2.14.x 71a4a4527839d710594b39d49f4758c9493e70c6
# 2.15.x 420f842454dd753e019b6ae74b674ff5457358a3
PANTS_PEX_RELEASE=STABLE ./build-support/bin/release.sh build-universal-pex
I would also need someone to update the docs at readme.com for each of
2.14.x
,
2.15.x
,
2.16.x
and
main
(at
a39c1b970be2cb4421b20728c21c3222aeb4b015
).
w

witty-crayon-22786

04/24/2023, 2:59 PM
so… there was a fix for PEX building that might not have been backported all the way to 2.14.x, but the PEX should actually still get generated
check whether a file is actually generated at the mentioned path, and if so, ignore the error
if not: what’s the error?
c

curved-television-6568

04/24/2023, 3:23 PM
there was no file in
dist/deploy/pex
as it was for the other versions that worked error:
Copy code
Fetching <https://binaries.pantsbuild.org/wheels%2F3rdparty%2F71a4a4527839d710594b39d49f4758c9493e70c6%2F2.14.2rc2%2Bgit71a4a452%2Furllib3-1.26.15-py2.py3-none-any.whl>

Fetching <https://binaries.pantsbuild.org/wheels%2F3rdparty%2F71a4a4527839d710594b39d49f4758c9493e70c6%2F2.14.2rc2%2Bgit71a4a452%2Fzipp-3.15.0-py3-none-any.whl>

[=== 00:27 Checking prebuilt wheels for 2.14.2rc2+git71a4a452 ===]
All 2 pantsbuild.pants packages were fetched and are valid.
Wrote whl with version 2.14.2rc2 to /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2/pantsbuild.pants-2.14.2rc2-cp39-cp39-manylinux2014_x86_64.whl.

Wrote whl with version 2.14.2rc2 to /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2/pantsbuild.pants-2.14.2rc2-cp38-cp38-manylinux2014_x86_64.whl.

Wrote whl with version 2.14.2rc2 to /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2/pantsbuild.pants-2.14.2rc2-cp37-cp37m-macosx_10_15_x86_64.whl.

Wrote whl with version 2.14.2rc2 to /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2/pantsbuild.pants.testutil-2.14.2rc2-py3-none-any.whl.

Wrote whl with version 2.14.2rc2 to /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2/pantsbuild.pants-2.14.2rc2-cp37-cp37m-manylinux2014_x86_64.whl.

Wrote whl with version 2.14.2rc2 to /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2/pantsbuild.pants-2.14.2rc2-cp38-cp38-macosx_10_15_x86_64.whl.

Wrote whl with version 2.14.2rc2 to /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2/pantsbuild.pants-2.14.2rc2-cp39-cp39-macosx_11_0_arm64.whl.

Wrote whl with version 2.14.2rc2 to /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2/pantsbuild.pants-2.14.2rc2-cp39-cp39-macosx_10_15_x86_64.whl.

pid 7082 -> /private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/tmph2hfhxjp/venvs/cac1718c056bb509f51fcdcc0c376b33deaaa8ec/26206f86a43095c89a46831693c798eb92a902ac/bin/python -sE /private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/tmph2hfhxjp/venvs/cac1718c056bb509f51fcdcc0c376b33deaaa8ec/26206f86a43095c89a46831693c798eb92a902ac/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --use-deprecated legacy-resolver --isolated -q --cache-dir /private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/tmph2hfhxjp/pip_cache download --dest /private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/tmph2hfhxjp/downloads/resolver_download.1jpofpw_/Users.andreas.stenius..pyenv.versions.3.7.13.bin.python3.7 --only-binary :all: pantsbuild.pants==2.14.2rc2+git71a4a452 --no-index --find-links /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2+git71a4a452 --find-links /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/3rdparty/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2+git71a4a452 --retries 5 --timeout 15 exited with 1 and STDERR:
ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==2.14.2rc2+git71a4a452 (from versions: none)
ERROR: No matching distribution found for pantsbuild.pants==2.14.2rc2+git71a4a452

Traceback (most recent call last):
  File "/Users/andreas.stenius/.cache/pants/named_caches/pex_root/venvs/0a326a02477142002846fbf7d79fe20cd05bfd62/bdfcee4cd90a613a01dad3eee8a4b0670645276b/pex", line 236, in <module>
    runpy.run_module(module_name, run_name="__main__", alter_sys=True)
  File "/Users/andreas.stenius/.pyenv/versions/3.7.13/lib/python3.7/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/Users/andreas.stenius/.pyenv/versions/3.7.13/lib/python3.7/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/Users/andreas.stenius/.pyenv/versions/3.7.13/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/pants-sandbox-gFSMVq/build-support/bin/_release_helper.py", line 1250, in <module>
    main()
  File "/private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/pants-sandbox-gFSMVq/build-support/bin/_release_helper.py", line 1236, in main
    build_pex(fetch=True)
  File "/private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/pants-sandbox-gFSMVq/build-support/bin/_release_helper.py", line 856, in build_pex
    check=True,
  File "/Users/andreas.stenius/.pyenv/versions/3.7.13/lib/python3.7/subprocess.py", line 512, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/Users/andreas.stenius/.cache/pants/named_caches/pex_root/venvs/0a326a02477142002846fbf7d79fe20cd05bfd62/bdfcee4cd90a613a01dad3eee8a4b0670645276b/bin/python', '/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/tmpmhrks77y/pex', '-o', 'dist/pants.2.14.2rc2+git71a4a452.pex', '--no-build', '--no-pypi', '--disable-cache', '-f', '/Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2+git71a4a452', '-f', '/Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/3rdparty/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2+git71a4a452', '--no-strip-pex-env', '--console-script=pants', '--python-shebang', '/usr/bin/env python', '--interpreter-constraint', 'CPython>=3.7,<3.10', '--platform=linux_x86_64-cp-37-m', '--platform=linux_x86_64-cp-38-cp38', '--platform=linux_x86_64-cp-39-cp39', '--platform=macosx_11.0_x86_64-cp-37-m', '--platform=macosx_11.0_x86_64-cp-38-cp38', '--platform=macosx_11.0_x86_64-cp-39-cp39', 'pantsbuild.pants==2.14.2rc2+git71a4a452', '--venv']' returned non-zero exit status 1.
similar error for the 2.15.x branch
w

witty-crayon-22786

04/24/2023, 3:29 PM
…hm. so it sounds like perhaps one of the wheels expected by
Copy code
'--interpreter-constraint', 'CPython>=3.7,<3.10', '--platform=linux_x86_64-cp-37-m', '--platform=linux_x86_64-cp-38-cp38', '--platform=linux_x86_64-cp-39-cp39', '--platform=macosx_11.0_x86_64-cp-37-m', '--platform=macosx_11.0_x86_64-cp-38-cp38', '--platform=macosx_11.0_x86_64-cp-39-cp39'
…wasn’t generated by the CI job?
c

curved-television-6568

04/24/2023, 3:30 PM
ah could be, I did notice there was indeed fewer wheels published to pypi during the publish step for the older versions.
w

witty-crayon-22786

04/24/2023, 3:30 PM
did the CI job wheel build finish?
c

curved-television-6568

04/24/2023, 3:30 PM
yes
I didn’t look at the logs, but all tasks where green for the Version Bump job
oh.. wait there were jobs also for the pushed release tag, didn’t see them. so that could’ve failed if those are the ones that count
ah, that would’ve affected the tag to commit mapping.. and was failed for 2.14 but ok for 2.15
re-run the publish-tag-to-commit-mapping for 2.14 still fails, so 🤷 ? https://github.com/pantsbuild/pants/actions/runs/4782211544
(but unrelated, I think)
w

witty-crayon-22786

04/24/2023, 3:40 PM
hm. were there backports to the branch that might have caused something to fail?
c

curved-television-6568

04/24/2023, 3:57 PM
nothing that stands out to me.. you can see the PRs in the changelog
e

enough-analyst-54434

04/24/2023, 4:26 PM
This is old and well known and we keep forgetting. Mac M1 is a no-go for this until Benjy fixed it in 2.16.x - right?:
Copy code
/private/var/folders/y2/6t5bbk554wv5h6w_qlgpcpnm0000gr/T/tmph2hfhxjp/downloads/resolver_download.1jpofpw_/Users.andreas.stenius..pyenv.versions.3.7.13.bin.python3.7 --only-binary :all: pantsbuild.pants==2.14.2rc2+git71a4a452 --no-index --find-links /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/pantsbuild.pants/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2+git71a4a452 --find-links /Users/andreas.stenius/src/github/kaos/pants/dist/deploy/wheels/3rdparty/71a4a4527839d710594b39d49f4758c9493e70c6/2.14.2rc2+git71a4a452 --retries 5 --timeout 15 exited with 1 and STDERR:
ERROR: Could not find a version that satisfies the requirement pantsbuild.pants==2.14.2rc2+git71a4a452 (from versions: none)
ERROR: No matching distribution found for pantsbuild.pants==2.14.2rc2+git71a4a452
That's trying to use Python 3.7
Users.andreas.stenius..pyenv.versions.3.7.13.bin.python3.7
to download a wheel for
pantsbuild.pants==2.14.2rc2+git71a4a452
on an M1. Good luck with that since we only build 3.9 for M1.
@curved-television-6568 I'll do the docs publishing and PEX building / uploading here shortly for the releases missing these things.
👍 1
c

curved-television-6568

04/24/2023, 4:28 PM
Dang. I knew it blew on something last time around. Obviously forgot on what..
Thanks @enough-analyst-54434
e

enough-analyst-54434

04/24/2023, 4:30 PM
You're welcome. I'll accept payment in annoying you all by pointing out its been how many years since Apple Silicon and still no non-janky affordable CI for it.
💸 1
Ditch your damn macs!
c

curved-television-6568

04/24/2023, 4:31 PM
it’s the one I’ve got (from $ work).. my private alternative is a Win box 😮 (haven’t had one in years, so I feel lost on that one 😅 )
could of course run *nix on it tho…
e

enough-analyst-54434

04/24/2023, 4:33 PM
Exactly. That's what I did.
c

curved-television-6568

04/24/2023, 4:34 PM
but… I’d say it’s a small win to have me ditch it as long as there are users out there using it.. we might as well have some early feedback if we break for that platform somehow..
e

enough-analyst-54434

04/24/2023, 4:35 PM
Well, yes - there is the enabler problem. You have to stick your neck out and hope alot of people follow you; otherwise there's always "but other people" - which is sadly valid. Its Molochian.
Hopefully other corps step up and start pitching into the Pantsbuild non-profit bank account to pay for this stuff.
💯 1
w

witty-crayon-22786

04/24/2023, 4:43 PM
but… we do build M1 wheels in CI, don’t we? it shouldn’t matter what the client machine looks like?
this is the M1 wheel build for the head commit of the 2.14.x branch https://github.com/pantsbuild/pants/actions/runs/4780507287/jobs/8500427358
e

enough-analyst-54434

04/24/2023, 4:43 PM
Sure - someone needs to fix 2.14 and 2.15 then
Not worth it. Benjy did it for 2.16 +
This had been punted for ~2 years until he did.
Now we're all just confused and forgetting history.
w

witty-crayon-22786

04/24/2023, 4:44 PM
most definitely. but… i’m not understanding how the client machine matters here
all of these are downloaded wheels, right?
e

enough-analyst-54434

04/24/2023, 4:44 PM
We have a buggy script.
Benjy fixed the bugs.
w

witty-crayon-22786

04/24/2023, 4:45 PM
got it. and the bug only impacts M1 clients?
e

enough-analyst-54434

04/24/2023, 4:45 PM
Your curiosity is too late!
Yes.
w

witty-crayon-22786

04/24/2023, 4:45 PM
got it.
e

enough-analyst-54434

04/24/2023, 4:45 PM
Eric was the primary affected releaser for a long time.
There used to be a nag in the script for M1 users.
Benjy was the one to step up and really fix finally.
w

witty-crayon-22786

04/24/2023, 4:46 PM
i still don’t have an M1, so wouldn’t have seen it.
e

enough-analyst-54434

04/24/2023, 4:46 PM
Exactly.
I actually have that wrong - there were 2 sets of M1 issues. Building the wheel fixed by Macstadium bucks and then issues with the pants.pex. But at any rate.
w

witty-crayon-22786

04/24/2023, 4:48 PM
e

enough-analyst-54434

04/24/2023, 4:49 PM
Hey @curved-television-6568 what's up with naimg the releases
s/real tag name/v.../
?Is this a new thing I should be doing too? It seems confusing to have a release named after not any tag or any version Pants uses.
@witty-crayon-22786 looks like it.
w

witty-crayon-22786

04/24/2023, 4:50 PM
k: picking it
e

enough-analyst-54434

04/24/2023, 4:51 PM
I think an M1 user should do that to actually test its worth doing so.
e

enough-analyst-54434

04/24/2023, 4:51 PM
All the picking we do is - insane IMO.
Too much
w

witty-crayon-22786

04/24/2023, 4:52 PM
it would be less sane to leave releasing broken for two of our releasers from branches which are actively being released?
e

enough-analyst-54434

04/24/2023, 4:52 PM
Yes
You have no clue your pick fix is the fix.
w

witty-crayon-22786

04/24/2023, 4:52 PM
@curved-television-6568: can you confirm that the edit in #18817 fixes PEX building for you?
e

enough-analyst-54434

04/24/2023, 4:52 PM
It should be done if done at all by an M1 user and verified to work.
Sometimes automation is not the answer. Post it note - just ask for help for these 2 old and dying brnaches. Seems simple to me.
@curved-television-6568 I left the 2.14 amd 2.15 releases with their native release tag. Feel free to edit if that's the new thing or edit yours back if it's not.
Ok @curved-television-6568 the reference re-gen + doc sync is complete for all 4 release tags.
c

curved-television-6568

04/24/2023, 5:46 PM
Thanks! It was the release title only, so for display purposes and did not affect download paths or other. But regardless, I removed it to be consistent with previous releases.
@witty-crayon-22786 confirmed fix works for me. I get another issue later, but the universal pex is in the dist/deploy/pex folder.
e

enough-analyst-54434

04/24/2023, 5:52 PM
Yeah - the test failure - right?
I still contend it is much much more sane to just add a note to the docs for 2.14.x and 2.15.x. There are many more processes than this that require asking for help for some releasers and these branches are dead.
c

curved-television-6568

04/24/2023, 5:53 PM
yea, smoke test running
pants.pex --version
w

witty-crayon-22786

04/24/2023, 5:53 PM
@enough-analyst-54434: 2.15.x is not dead… this is only it’s second stable release
c

curved-television-6568

04/24/2023, 5:54 PM
well, with this I can navigate most of it 😉
e

enough-analyst-54434

04/24/2023, 5:54 PM
Ok - I'll desist. I cannot grok Pantsbuild well any longer.
c

curved-television-6568

04/24/2023, 5:54 PM
I consider the amount of bespoke scripts in build-support as a metric for missing pants features, fwiw
e

enough-analyst-54434

04/24/2023, 5:54 PM
@curved-television-6568 no. I needed to do docs and pypi deletes
The process is only anywhere near smooth for Stu Benjy and I
IIUC
c

curved-television-6568

04/24/2023, 5:55 PM
@enough-analyst-54434 yes, but those are tied to permissions rather than build issues
e

enough-analyst-54434

04/24/2023, 5:55 PM
Everyone else has plenty of roadbumps.
c

curved-television-6568

04/24/2023, 5:56 PM
yup, will want to look at streamlining this one piece at a time.
👍 1
w

witty-crayon-22786

04/24/2023, 6:05 PM
have landed those picks.
c

curved-television-6568

04/24/2023, 6:15 PM
I’ll add picks for 18477 too
2 Views