Is it a known bug that Pants fails to detect mismatch b/t a

version-control style python_requirement

(reference) and the lockfile? Here's what I'm running into: • we have pants version 2.28.0 • we have a

python_requirement

defined in 3rdparty/BUILD:

MY_DEP_VERSION = "v1.xx.xx"
MY_DEP_GIT_URL = f"my-dep@ git+<https://github.com/my-repo/my-dep.git@{MY_DEP_VERSION}>"
python_requirement(
    name="my_dep",
    requirements=[MY_DEP_GIT_URL],
    modules=["mydep"],
)

• after generating the lockfile, I bumped `MY_DEP_VERSION`to a newer version. Surprisingly, running "pants check" or "pants package" on targets than depend on "my_dep" didn't error out. • When I manually run "pants generate-lockfiles", the summary does show the upgraded dependencies for "my_dep" from v1.xx.xx to the newer version "v1.yy.yy" I wonder if this is a bug? Is it known or I should file a new one? Or I'm doing something wrong with version control requirement? Thank you!

Pants doesn't automatically regenerate lockfiles when the inputs change (it should though, and hopefully will in the future), so this, unfortunately, is expected behavior. You would see the same thing if you changed any requirement, not just a version control style one.

hmm, for other requirements which pin a specific pip/pypi version, I tried to modify the pinned version of a requirement in BUILD, then

pants check ::

did error out, saying the lockfile is incompatible. BTW, I do already have a custom workflow to automate regeneration of lockfiles when this happens.

Ah yes, that is not specifically checking for that per se, but rather it is trying to use the requirement (by subsetting the lockfile) and not finding it in the lockfile

Why on earth does Pants store a lock header if not to fail fast on an input change?

To provide more details on the VCS requirement, in the lockfile, resolved dependency looks like:

{
  "artifacts": [
    {
      "algorithm": "sha256",
      "commit_id": "abcdef1234567890deadbeefcafebabefeedface",
      "hash": "1234567890abcdef1234567890abcdef1234567890abcdef1234567890abcd",
      "url": "git+<https://example.com/org/sample-repo.git@vX.Y.Z>"
    }
  ],
  "project_name": "sample-repo",
  "requires_dists": [...],
  "requires_python": ">=3.10",
  "version": "X.Y.Z"
}

And this requirement is being depended on throughout our python project in many places. After modifying version from "X.Y.Z" to a different(newer) version, both

pants check ::

and

pants package ::

would still succeed... If I manually run

pants generate-lockfiles

after modifying version, I can see in the diff that

commit_id

,

hash

,

url

and

version

all have changed as expected

Pants does validate that a given set of requirements is in the metadata before it tries to consume those requirements from a lockfile, but that is lazy validation, and I think not really necessary (presumably Pex would give a legible error in this case anyway). AFAICT the metadata tracks which requirements were used to generate the lockfile, but not where they came from (which requirements.txt, or pyproject.toml, say). So that seems to be to be of limited use. I'm not sure that any of that metadata is worth its weight. I would like to replace it with just "changes to this file (and/or these option values) should trigger a lockfile regeneration", a la Cargo.toml and Cargo.lock.

Great, thanks. I don't mind working the issue if you want to turn to new Python backend stuff. I'd personally be very happy to see that progress and people thrashing being left off the hook. The VCS will be tricky since Pex does not lock based on commit, but based on the content hash of the clone after built into an sdist.

Ah, sounds like a fix is not easy so I will let you at it: https://github.com/pex-tool/pex/issues/3032

@happy-kitchen-89482 re:

Ah yes, this comment may as well say "why are we even bothering with this"...

The code looks misguided all around. If you're going to have a lock header you definitely should validate the input requirements, but you should be validating the input requirements to the lock, not the subset requirements. So, the comment is right I'd say, but it misses the forest for the trees since the whole concept is botched. Perhaps its too expensive to calculate the full root requirement set and check it has not changed wrt lock header? If so though, that would be the more appropriate comment.

I'll upgrade to this for the next Pants dev release, but you can update in your own config, without upgrading Pants, by adding this to `pants.toml`:

[pex-cli]
version = "v2.1.143"
known_versions = [
"v2.73.1|macos_arm64|e6907e079a3f7c917dc88b41d892f732d4b8dbe388abfefc064d19c4a9f3c7e8|4939987",
"v2.73.1|macos_x86_64|e6907e079a3f7c917dc88b41d892f732d4b8dbe388abfefc064d19c4a9f3c7e8|4939987",
"v2.73.1|linux_x86_64|e6907e079a3f7c917dc88b41d892f732d4b8dbe388abfefc064d19c4a9f3c7e8|4939987",
"v2.73.1|linux_arm64|e6907e079a3f7c917dc88b41d892f732d4b8dbe388abfefc064d19c4a9f3c7e8|4939987"
]

Yeah it works! @happy-kitchen-89482 Thanks so much for prompt fix!

The thanks are due to @brief-scientist-13682. Glad it works!