Hey All, running into an issue related to a VCS de...
# general
s
Hey All, running into an issue related to a VCS dependency in a lockfile, hoping someone can help me figure it out
Copy code
# pyproject.toml
[project]
...
dependencies = [
    ...
    nautobot @ git+<https://github.com/utsc-networking/nautobot@utsc-custom>
]
...
Copy code
# pyproject.lock (my pants resolve lockfile)
...
{
  "locked_resolves": [
    {
      "artifacts": [
        {
          "algorithm": "sha256",
          "commit_id": "b8ec0a65737f3d61615ccdeb4a1aad7a85795cd7",
          "hash": "814aa5468450b3a9d7a6d7c5e217b915ef8ae541e259c76f9204d136d88150e5",
          "url": "git+<https://github.com/utsc-networking/nautobot@utsc-custom>"
        }
      ],
      "project_name": "nautobot",
      ...
    },
    ...
  ]
}
Copy code
$ pants export --resolve=python-default
...
10:58:11.28 [ERROR] 1 Exception encountered:

Engine traceback:
  in `export` goal

ProcessExecutionFailure: Process 'Build pex for resolve `python-default`' failed with exit code 1.
stdout:

stderr:
There was 1 error downloading required artifacts:
1. nautobot 2.4.2 from git+<https://github.com/utsc-networking/nautobot@utsc-custom>
    Expected sha256 hash of 814aa5468450b3a9d7a6d7c5e217b915ef8ae541e259c76f9204d136d88150e5 when downloading nautobot but hashed to 4ea93d4a93006bd293a618c24f4c77c1cfb669f80f2a3596be3778660bcb6e06.
I don't know how this git repo is being hashed, but something's not right here. The first time this happened, i chalked it up to a wierd transient glitch, manually updated the hash in my pants lockfile but then a couple days later it happened again, and then again this morning. the pinned repo commit hasn't changed in all this time, and the repo itself also hasn't changed
and to make things even weirder, i have another VCS dep in my monorepo,
pexpect @ git+<https://github.com/pexpect/pexpect@eb2820cec514c3ed5482e80ad3438cd31f2fa1ef>
, and that one doesn't exhibit this behaviour
c
Is this "flaky" (like if you run it in a loop it will trigger), or something that mysteriously happens after some period of time? Is
utsc-custom
a mutable branch, or a tag?
s
it's not flaky.... i can re-export a dozen times in a day and it won't happen again. only seems to happen after some amount of time, at least a day
and
utsc-custom
is a branch, but the branch (and the commit it points to) haven't changed in all this time
b
@stale-twilight-79248 in short:
Copy code
:; diff -x .git -r /tmp/tmponz65mq9/nautobot-2.4.2/nautobot/ /tmp/tmpv5kck6z7/nautobot_effc2cf1981a49d0ac9fb3e976f9a2dd/
Only in /tmp/tmpv5kck6z7/nautobot_effc2cf1981a49d0ac9fb3e976f9a2dd/docs: apps
Only in /tmp/tmpv5kck6z7/nautobot_effc2cf1981a49d0ac9fb3e976f9a2dd/docs: assets
Only in /tmp/tmpv5kck6z7/nautobot_effc2cf1981a49d0ac9fb3e976f9a2dd/docs: development
Only in /tmp/tmpv5kck6z7/nautobot_effc2cf1981a49d0ac9fb3e976f9a2dd/docs: generate_code_reference_pages.py
Only in /tmp/tmpv5kck6z7/nautobot_effc2cf1981a49d0ac9fb3e976f9a2dd/docs: img
Only in /tmp/tmpv5kck6z7/nautobot_effc2cf1981a49d0ac9fb3e976f9a2dd/docs: index.md
Only in /tmp/tmpv5kck6z7/nautobot_effc2cf1981a49d0ac9fb3e976f9a2dd/docs: macros.py
Only in /tmp/tmpv5kck6z7/nautobot_effc2cf1981a49d0ac9fb3e976f9a2dd/docs: media
Only in /tmp/tmpv5kck6z7/nautobot_effc2cf1981a49d0ac9fb3e976f9a2dd/docs: nautobot_logo.png
Only in /tmp/tmpv5kck6z7/nautobot_effc2cf1981a49d0ac9fb3e976f9a2dd/docs: nautobot_logo.svg
Only in /tmp/tmpv5kck6z7/nautobot_effc2cf1981a49d0ac9fb3e976f9a2dd/docs: overview
Only in /tmp/tmpv5kck6z7/nautobot_effc2cf1981a49d0ac9fb3e976f9a2dd/docs: release-notes
Only in /tmp/tmpv5kck6z7/nautobot_effc2cf1981a49d0ac9fb3e976f9a2dd/docs: requirements.txt
Only in /tmp/tmpv5kck6z7/nautobot_effc2cf1981a49d0ac9fb3e976f9a2dd/docs: user-guide
So the docs symlink is the issue. Looking into that.
Ok, this is only an issue when using
--avoid-downloads
, which is the default when Pex is using modern Pip. For
--no-avoid-downloads
, Pip zips up the VCS archive and double-includes docs from the symlink - that's the 4ea93d4a93006bd293a618c24f4c77c1cfb669f80f2a3596be3778660bcb6e06 hash. For
--avoid-downloads
, it presents a cloned directory (no zip) and that hash is 814aa5468450b3a9d7a6d7c5e217b915ef8ae541e259c76f9204d136d88150e5.
Ok, easy fix. I should have a release out this afternoon @stale-twilight-79248 - thanks for the report.
s
John, you are a wonderful, incredible person. In the time it took me to go pick up my kids from school, you managed to troubleshoot, identify, diagnose, and fix one of the most esoteric issues I've come across in all my time using pants and pex Thank you! Seriously, thank you!
b
You're welcome. The fix is released here: https://github.com/pex-tool/pex/releases/tag/v2.92.2
🎉 1
s
can confirm it works! newly generate lockfile contains the correct hash, and export works without error Thanks again!
b
Thanks for confirming, although this is not surprising. The fix PR included a direct repro: https://github.com/pex-tool/pex/pull/3149/changes#r3095793336