https://pantsbuild.org/ logo
Join Slack
Powered by
In the ST2 repo, regenerating the lockfile with 2....
# development
p
In the ST2 repo, regenerating the lockfile with 2.15.0 works fine (the lockfile shows pex_version 2.1.111). But, it is failing on 2.16.0a0 (which 
./pants help tools
shows is using pex 2.1.126). Any hints on debugging would be appreciated. error message in the🧵
✅ 1
Copy code
ProcessExecutionFailure: Process 'Generate lockfile for st2' failed with exit code 1.
stdout:

stderr:
Failed to import pip: No module named 'pip'

Download pip:
curl <https://bootstrap.pypa.io/get-pip.py> | python

<snip 2 more pip download errors>
<snip "running dist_info" and a bunch of "creating" and "writing" lines>

Could not gather lock metadata for 3 projects with source artifacts:
1. /tmp/pants-sandbox-9iomkx/.tmp/tmppu9xjjqs/usr.bin.python3.6m/st2-auth-backend-flat-file-0.1.1.zip: Executing /home/cognifloyd/.cache/pants/named_caches/pex_root/venvs/52748ab20d11c253b0b94c5711d08220c8d6e353/5985ed09b49a653d6596b0e14d134c5456cf1a9f/bin/python -sE /home/cognifloyd/.cache/pants/named_caches/pex_root/venvs/52748ab20d11c253b0b94c5711d08220c8d6e353/5985ed09b49a653d6596b0e14d134c5456cf1a9f/pex -c import sys

import setuptools.build_meta


if not hasattr(setuptools.build_meta.__legacy__, 'prepare_metadata_for_build_wheel'):
    sys.exit(75)

result = setuptools.build_meta.__legacy__.prepare_metadata_for_build_wheel(*('/tmp/pants-sandbox-9iomkx/.tmp/tmp5fg3n042/build',), **{})
with open('/tmp/pants-sandbox-9iomkx/.tmp/tmpgryhk1b5', "w") as fp:
    fp.write(result)
 failed with 1
<snip 2 other equivalent error messages but for different zips>


Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.
The 3 projects pex seems to be choking on are vcs requirements that, in this case, contain first party code. So I can change those other repos if needed (for now they need to remain vcs requirements as changing that is more than I want to tackle with the st2 infra in its current state).
Here are those requirements:
Copy code
st2-auth-backend-flat-file @ git+<https://github.com/StackStorm/st2-auth-backend-flat-file.git@master>
st2-auth-ldap @ git+<https://github.com/StackStorm/st2-auth-ldap.git@master>
st2-rbac-backend @ git+<https://github.com/StackStorm/st2-rbac-backend.git@master>
Oh crap. The initial error comes from a stupid thing that those git repos call in setup.py https://github.com/StackStorm/st2-auth-ldap/blob/master/dist_utils.py#L64-L70 @enough-analyst-54434 assuming that pip is installed is a bad assumption, right?
e
Correct. That setup.py is frankly doing a lot of crazy looking things. Are you 100% sure your needs are that far off brand to need all that?
p
I consider that stuff legacy crap… I want to delete all of these hand-crafted setup.py files. Eventually, I’ll bring pants to tame the other st2 repos, for now, they need to limp along until pants can bless them with sanity.
e
Ok, well if you don't want to change those repos much, adding a pep-518 pyproject.toml with your build requirements (Pip) probably makes sense.
p
I was able to rip out some of the screwy stuff in those setup.py files. Now regenerating the lockfile works just fine.
e
Excellent.
p
I hate non-declarative setup.py files!
t
I am facing a very similiar issue suddenly :
Copy code
Could not gather lock metadata for 1 project with source artifacts:
1. /tmp/pants-sandbox-Fusvjh/.tmp/tmp6341hm_u/myorg.dist.export.python.virtualenvs.default.3.8.16.bin.python3.8/cairocffi-1.5.1.tar.gz: Executing /tmp/named_caches/pex_root/venvs/b9549377f92ee3eff1e9e1fa14bb6cb3d0147571/5fd7049af63e03f347278c89401424cd9731df9a/bin/python -sE /tmp/named_caches/pex_root/venvs/b9549377f92ee3eff1e9e1fa14bb6cb3d0147571/5fd7049af63e03f347278c89401424cd9731df9a/pex -c import sys

import build


if not hasattr(build, 'prepare_metadata_for_build_wheel'):
    sys.exit(75)

result = build.prepare_metadata_for_build_wheel(*('/tmp/pants-sandbox-Fusvjh/.tmp/tmpjtfl64d3/build',), **{})
with open('/tmp/pants-sandbox-Fusvjh/.tmp/tmptd9v7we7', "w") as fp:
    fp.write(result)
 failed with 1



Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.
Looks like it is due to the a new version of the package with only this commit: https://github.com/Kozea/cairocffi/commit/e70945dd27ec11d931b604da03c88adcb4de9440 Can you explain why so I can file an issue there?
When upgrading from 2.15 to 2.16.rc0 I am facing this issue even with 1.5.0 of that package
e
@thousands-plumber-33255 that looks like a bug and a different one at that. I'll try to repro here in the next hour and report back.
👍 1
Ok. I repro. The missing reason is just above the fold in your snip - Pants probably hides it:
Copy code
discovered packages -- ['cairocffi']
running dist_info
creating /tmp/tmpk_oljxf2/build/cairocffi.egg-info
writing /tmp/tmpk_oljxf2/build/cairocffi.egg-info/PKG-INFO
writing dependency_links to /tmp/tmpk_oljxf2/build/cairocffi.egg-info/dependency_links.txt
writing requirements to /tmp/tmpk_oljxf2/build/cairocffi.egg-info/requires.txt
writing top-level names to /tmp/tmpk_oljxf2/build/cairocffi.egg-info/top_level.txt
writing manifest file '/tmp/tmpk_oljxf2/build/cairocffi.egg-info/SOURCES.txt'
reading manifest file '/tmp/tmpk_oljxf2/build/cairocffi.egg-info/SOURCES.txt'
reading manifest template '<http://MANIFEST.in|MANIFEST.in>'
adding license file 'LICENSE'
writing manifest file '/tmp/tmpk_oljxf2/build/cairocffi.egg-info/SOURCES.txt'
creating '/tmp/tmpk_oljxf2/build/cairocffi-1.5.1.dist-info'
error: invalid command 'bdist_wheel'
Could not gather lock metadata for 1 project with source artifacts:
So, that's 
invalid command 'bdist_wheel'
and that's because the build system requires omits wheel: https://github.com/Kozea/cairocffi/blob/b48a5b3776490ded8931dd8e0e92db91178ead5e/pyproject.toml#L1-L2 I have to read some specs to see if that is a bug on their part or on Pex's part. A short term workaround though is to fork the project and add 
wheel
to those requirements and change your cairocffi requirement to a vcs requirement on your fork.
I'll report back once more with the result of my reading up on whether Pex should automagically always be supplying wheel as a dep to all PEP-517 builders out of band.
Ok, standards aside, Pip does silently provide wheel out of band; so Pex should too. I have some more reading to do but I'll file an issue and link that here to track this.
Ah, ok - this is a Pex bug. I'm not consulting 
get_requires_for_build_wheel
when its present before calling `prepare_metadata_for_build_wheel`(https://peps.python.org/pep-0517/#get-requires-for-build-wheel).
Ok, @thousands-plumber-33255 Pex 2.1.134 is released with the fix. You should be able to fix your Pants setup by adding this to your `pants.toml`:
Copy code
[pex-cli]
version = "v2.1.134"
known_versions = [
    "v2.1.134|macos_arm64|82b24645769c19483c1306c1ba7a888471a5e1df3a2b538788bc7e0d1b20dbf0|4085867",
    "v2.1.134|macos_x86_64|82b24645769c19483c1306c1ba7a888471a5e1df3a2b538788bc7e0d1b20dbf0|4085867",
    "v2.1.134|linux_x86_64|82b24645769c19483c1306c1ba7a888471a5e1df3a2b538788bc7e0d1b20dbf0|4085867",
    "v2.1.134|linux_arm64|82b24645769c19483c1306c1ba7a888471a5e1df3a2b538788bc7e0d1b20dbf0|4085867"
]
👀 1
t
Thank you!
So this can be removed once Pants is including this PEX version=
?
e
@thousands-plumber-33255 yup. That's any Pants >= 2.16.0rc1 and 2.17.0.dev4
👍 1
Open in Slack
PreviousNext
Powered by