#20808 [v2.20.0] Lockfile generation with multiple indexes fails due to mismatching hashes Issue created by seifertm Describe the bug Upgrading Pants from v2.19.1 (PEX pinned to v2.1.137) to v2.20.0 breaks lockfile generation for PyTorch when multiple indexes are involved:

cat << EOF >pants.toml
[GLOBAL]
pants_version = "2.20.0"
backend_packages = [
  "pants.backend.python",
]
[python]
enable_resolves = true
interpreter_constraints = ["==3.11.*"]

[python.resolves]
python-default = "3rdparty/python/lock.txt"

[python-repos]
indexes = [
    "<https://pypi.org/simple/>",
    "<https://download.pytorch.org/whl/cpu>",
]
EOF

cat << EOF >BUILD
python_requirement(
    name="torch_cuda",
    requirements=[
        "torch==2.2.1,!=2.2.1+cpu",
    ],
)
EOF

pants generate-lockfiles

Actual output:

14:36:40.10 [ERROR] 1 Exception encountered:

Engine traceback:
  in `generate-lockfiles` goal

ProcessExecutionFailure: Process 'Generate lockfile for python-default' failed with exit code 1.
stdout:

stderr:
Expected sha256 hash of 26bd2272ec46fc62dcf7d24b2fb284d44fcb7be9d529ebf336b9860350d674ed when downloading torch but hashed to b90669b162984e302fbd05c9c270ef796e467903944ecefa7457babe9611607e.



Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.

Apparently,

torch-2.2.1-cp39-none-macosx_11_0_arm64.whl

has different content on both indexes.

$ sha256sum torch-2.2.1-cp39-none-macosx_11_0_arm64.whl  # PyPI version
26bd2272ec46fc62dcf7d24b2fb284d44fcb7be9d529ebf336b9860350d674ed  torch-2.2.1-cp39-none-macosx_11_0_arm64.whl
$ sha256sum torch-2.2.1-cp39-none-macosx_11_0_arm64.whl  # PyTorch repository version
b90669b162984e302fbd05c9c270ef796e467903944ecefa7457babe9611607e  torch-2.2.1-cp39-none-macosx_11_0_arm64.whl

Pants version v2.20.0 OS Linux Additional information I suspected the PEX upgrade as part of the Pants upgrade to be the culprit, so I started testing different PEX versions. I found that PEX v2.1.148 creates the lockfile as expected, whereas v2.1.149, v2.1.150, and v2.3.1 don't. The diff look unsuspicious to me, though. Related to #18965. pantsbuild/pants