Can I get some details on what all is happening wh...
# general
h
Can I get some details on what all is happening when generating constraints and exporting a resolve? I would like to proxy pypi dependencies through our Artifactory server using their remote repository capabilities. I'm having trouble with two issues • Generating lockfiles seems to error out finding distributions. I think this is a chicken/egg problem because the distributions don't readily exist in the proxied cache until they've been `pip install`ed. • Retrieving artifacts specified in the lockfile seems to be dependent on the url in the artifacts section of a package. This doesn't trigger any caching in the remote repository because it doesn't seem to be using
pip install
. Do those observations match what the underlying tooling would be doing? I need to ask JFrog (Artifactory developers) how they expect remote repositories to be used for lockfile generation. One idea I had was to do lockfile generation without going through the proxy and then introduce the proxy when it came time to install. I don't think that works because of what I'm seeing with the second bullet above.
My goal is to engage better with JFrog about how to leverage their capability, but I don't think it will be very productive if I leave so much unknown about what Pants is attempting to do.
h
Pants is using Pex to generate the lockfile, which it does here
Pex delegates much of that work to Pip
You can set up a custom package index to point at custom URLs, has that not worked for you?
h
Yep, I've set up the custom index, but it doesn't find distributions. And I think that's because it hasn't formally requested them and triggered the proxy to cache them.
I know my cache layer is working because if I do
pip install <some package> --index <my index>
, I pull the package I expect and see it in my cache layer.
So it's intended to look and fail like a mirror, but it only caches the things you've actually requested. Don't want to be in the business of hosting all the artifacts that have ever been pushed there.
That's why I had the idea to separate package exploration and package installation. But it seems I am not able to modify the paths specified in the lockfile for where artifacts are served from. I think at the export layer, pants is grabbing wheels and source distributions directly rather than getting them through pip based on what I've seen.