Hi all! I have the following in a pants/pex lockfi...
# general
f
Hi all! I have the following in a pants/pex lockfile:
Copy code
{
          "artifacts": [
            {
              "algorithm": "sha256",
              "hash": "72264f1594e547c6ac986c694a0c7617b62bd45fb6e806d81ddb5c46b538dfec",
              "url": "git+<https://github.com/photocrowd/django-cursor-pagination@master>"
            }
          ],
          "project_name": "django-cursor-pagination",
          "requires_dists": [],
          "requires_python": null,
          "version": "0.2.1"
        },
For reasons, I’m using
pex3 lock export
to turn this into a pip-style requirements.txt with hashes. The corresponding output in the export file is:
Copy code
django-cursor-pagination==0.2.1 \
  --hash=sha256:72264f1594e547c6ac986c694a0c7617b62bd45fb6e806d81ddb5c46b538dfec
which then pulls the file from our Artifactory index (which has a different hash) than the branch from the repo. Is this intentional on the part of pex to omit the URL? How can I tell Pex to include that part?
c
Friendly FYI, as our primary pex expert @enough-analyst-54434 is stepping down from pants support, we have a (revived) #C087V4P1T channel for questions specific for pex. I don’t believe this has been widely communicated yet.
😞 1
f
Thanks! Should I paste this over there?
c
you’re of course still welcome to post pex questions here and we’ll do our best to answer them 🙂
f
Or I guess since you’ve thrown up the bat signal he might show up here?
coke 1
c
I don’t think cross posting will be necessary as John could just as well reply in this thread here.. 🤷
(or someone else in the know, it’s just not me 😉 )
e
It's not intentional @future-oxygen-10553, you'll need to file an issue. Even with that though, I think things still won't work. IIRC Pip won't work in hash checking mode for VCS, or it does, but with an inscrutable hash value (what to hash for a VCS url? There are many ways to go about it, Pex uses one approach).
Do you mean
@master
?
Not awesome
f
Yeah, this is a transitive dependency for my project, fortunately the direct dep is an internal package, so I can ask them to update this 👍
You are correct though, I edited the exported file manually and got:
Copy code
Collecting git+<https://github.com/photocrowd/django-cursor-pagination@master> (from -r python-reqs.txt (line 756))
ERROR: Can't verify hashes for these requirements because we don't have a way to hash version control repositories:
    git+<https://github.com/photocrowd/django-cursor-pagination@master> (from -r python-reqs.txt (line 756))
Nonetheless, other URLs would work, right? Just not VCS URLs? So this is worth fixing?
e
URLs are weirder. Normally you'd run Pip with the hashed requirements file and the same --indexes you used when you created it. For a point of comparison, does pip-compile use wheel URLs or requirements.
What exactly the heck are you trying to do / work around / accomplish?
f
I’m not sure what pip-compile does
We have a pants managed repo, but we don’t want to have a pants-managed local venv. So the solution we came up with is to have pex export the pants lockfile to a requirements file and have pip install that
We’re not actually installing directly with URL requirements, just curious if that should be a supported feature and whether I should file an issue to fix this. If it’s just not possible, it’s not worth fixing anyways
e
Ok, just a sec. You're using the wrong tool here.
f
the more you know
e
@future-oxygen-10553 you want
pex3 venv create --lock my-lock [reqs] -d right-here.venv
,
If you leave out reqs it installs the whole lock file.
You'll need Pex 2.1.135 and newer for that: https://github.com/pantsbuild/pex/releases/tag/v2.1.135
We’re not actually installing directly with URL requirements, just curious if that should be a supported feature and whether I should file an issue to fix this.
It might be possible, but I think its not an appropriate feature. I used pip-compile as a point of comparison since it's ~industry standard for Pip workflows that want to lock with hashed requirements files.
f
Thank you so much! I should learn to just ask if there’s a way to do what I want before hacking something. The more I learn about pex, the more impressed I become ❤️
❤️ 1
We’re already on 137, to be able to support the newest
pip
versions
So that won’t be a problem 🙂
e
I'd say whenever you're facing a Rube Goldberg machine, step back and question alot.
f
Words to live by
Can I get pex to read a pants-created lock file directly, or do I have to sed the comments added by pants out of the file first? Pex is unhappy because comments aren’t standard JSON
Also, how does pex pick the Python to link in the venv it creates? Can pex install dependencies into an existing venv?
e
Pants is a bad citizen here, you have to strip the comments. In terms of which Python, you should really read
--help
, but try
--python
.
I'm not sure on the last question. Try it.
f
👍