Can I get some details on what all is happening wh...
# general
Can I get some details on what all is happening when generating constraints and exporting a resolve? I would like to proxy pypi dependencies through our Artifactory server using their remote repository capabilities. I'm having trouble with two issues • Generating lockfiles seems to error out finding distributions. I think this is a chicken/egg problem because the distributions don't readily exist in the proxied cache until they've been `pip install`ed. • Retrieving artifacts specified in the lockfile seems to be dependent on the url in the artifacts section of a package. This doesn't trigger any caching in the remote repository because it doesn't seem to be using
pip install
. Do those observations match what the underlying tooling would be doing? I need to ask JFrog (Artifactory developers) how they expect remote repositories to be used for lockfile generation. One idea I had was to do lockfile generation without going through the proxy and then introduce the proxy when it came time to install. I don't think that works because of what I'm seeing with the second bullet above.
My goal is to engage better with JFrog about how to leverage their capability, but I don't think it will be very productive if I leave so much unknown about what Pants is attempting to do.
Pants is using Pex to generate the lockfile, which it does here
Pex delegates much of that work to Pip
You can set up a custom package index to point at custom URLs, has that not worked for you?
Yep, I've set up the custom index, but it doesn't find distributions. And I think that's because it hasn't formally requested them and triggered the proxy to cache them.
I know my cache layer is working because if I do
pip install <some package> --index <my index>
, I pull the package I expect and see it in my cache layer.
So it's intended to look and fail like a mirror, but it only caches the things you've actually requested. Don't want to be in the business of hosting all the artifacts that have ever been pushed there.
That's why I had the idea to separate package exploration and package installation. But it seems I am not able to modify the paths specified in the lockfile for where artifacts are served from. I think at the export layer, pants is grabbing wheels and source distributions directly rather than getting them through pip based on what I've seen.