https://pantsbuild.org/ logo
#general
Title
# general
a

ancient-breakfast-45303

08/19/2022, 8:02 PM
Is it intentional that when using
pex --lock ...
that pants only considers the urls in the lock file and not the
--find-links
and
--index
arguments? I have a case where a lockfile references pypi but I need to build a pex in a constrained environment where the artifacts need to be downloaded from an internal registry.
e

enough-analyst-54434

08/19/2022, 8:05 PM
It is intentional. You would need to run
pex3 lock update --index ...
1st to flip URLs.
h

hundreds-father-404

08/19/2022, 8:06 PM
@enough-analyst-54434 why do we pass the indexes and find links args to Pex when installing a Pex lock, then? https://github.com/pantsbuild/pants/blob/4ef5b6a5b3a6b304e7a780c63fe62edc4cd3c154/src/python/pants/backend/python/util_rules/pex.py#L419 You answered this in https://github.com/pantsbuild/pants/pull/16458, but I don't think I understand. Do we grab the indexes recorded in the lock, or what we pass to pex on the CLI, or both?
e

enough-analyst-54434

08/19/2022, 8:07 PM
My answer is actually from the Pex end. It's intentional that Pex ignores this data (it should probably fail).
h

hundreds-father-404

08/19/2022, 8:07 PM
Pex ignores which data? The CLI args, or the lock values?
e

enough-analyst-54434

08/19/2022, 8:07 PM
The we's kill me. I may have badly reviewed but I'm almost positive I did not wire this stuff over in Pants.
Pex should ignore indexes and find links if you ask it to use --lock. It fetches the URLs in the lock directly
h

hundreds-father-404

08/19/2022, 8:09 PM
So to clarify, could we safely land https://github.com/pantsbuild/pants/pull/16458?
e

enough-analyst-54434

08/19/2022, 8:09 PM
Ah yes, sdists.
Pants should really not support those!
But it does, so you need to :/
a

ancient-breakfast-45303

08/19/2022, 8:10 PM
I have a workaround where I assemble a pex_root on another machine, and bring it over to the host
because if the artifacts are cached everything works
e

enough-analyst-54434

08/19/2022, 8:11 PM
The lock update is really the way to go here.
a

ancient-breakfast-45303

08/19/2022, 8:11 PM
yeah it does, not sure now I did not know this, thanks!
e

enough-analyst-54434

08/19/2022, 8:12 PM
Pex could make this easier by filling in all the -p you'll need to pass if you pass none though.
Maybe it does this already? I forget!
There is a test that demonstrates flipping from PyPI to find links but changing nothing else IIRC.
a

ancient-breakfast-45303

08/19/2022, 8:14 PM
got it, I'll try it out and report back
e

enough-analyst-54434

08/19/2022, 8:14 PM
Thanks Zameer.
a

ancient-breakfast-45303

08/19/2022, 8:14 PM
no thank you!
e

enough-analyst-54434

08/19/2022, 8:17 PM
Eric, no indexes or find-links are stored in the lock, they are just needed for PEP-517 / PEP-518 building of locked sdists.
👍 1
I just scanned back up and saw that question.
Technically indexes can be needed for wheels, but only as a back door to pass in creds if they are not in .netrc.
PEX scrapes them out of --indexes config.
a

ancient-breakfast-45303

08/19/2022, 8:24 PM
I guess the problem with
pex3 lock update
is that it can re-resolve the dependencies, where as I just want the urls to be updated. I would need to some how specify an empty list for
-p
?
e

enough-analyst-54434

08/19/2022, 8:29 PM
You need -p <pin> for each project pinned in the lock.
That's where Pex could help out, I think you have to manually concoct that list as things stand with a
jq
command or somesuch.
So, can you explain more the whole public lock, private use thing? Is this ~open source a thing from a company and then use it back in the company behind a firewall?
a

ancient-breakfast-45303

08/20/2022, 12:19 AM
it's not opensource but accessing the private registry from laptops is a PITA
I can also do the jq thing with
-p
for now, it's not a big ask because I can work around it by just caching the pex root and pulling it down
e

enough-analyst-54434

08/20/2022, 12:21 AM
Ok. I do think switching indexes find-links should be easier to do. I added support for it and tests; so worth a few cycles when I have free time.
I don't exactly have free time, but this bugged me. You should be able to switch repos or pick up newly published artifacts after-the-fact: https://github.com/pantsbuild/pex/pull/1884
4 Views