With the new pants-native lockfile generation, how...
# general
h
With the new pants-native lockfile generation, how is it expected for pants to stay aware if lockfiles are properly updated. As an example, I generated my lockfile, ran a test, changed a relevant line in
requirements.txt
that would invalidate the lockfile (for a transitive dependency of the test), but I did not get prompted at all when I reran my test.
And it doesn't seem like the
generate-lockfiles
behavior is cached. With no changes between calls, it seems like the entire lockfile generation is triggered each time.
h
Hi! Do you see the header at the top of the file? That is how we validate whether the lock file is stale. Note that it only tracks with the direct inputs to the lock are That is the exact same mechanism whether you use poetry or Pex
With no changes between calls, it seems like the entire lockfile generation is triggered each time.
That is correct. Generating a lock file is inherently side effecty and you may have kept the same loose requirement range but see if a new version gets pulled in
h
It didn't detect it, though, it seems. I added
cryptography<37
which both changed the direct inputs and invalidated the existing lockfile, but I didn't see any kind of warning. How is that staleness expected to be reported? Is that something like
./pants tailor --check
?
Makes sense on pulling new updates in. Though, I don't think that's how other lock tools work. They'll respect what's in the lockfile unless you specifically call out a package to upgrade. I believe that's how poetry works (and it's definitely how
pip-tools
works).
h
It is only reported when installing the lock file, so should happen when doing some thing like
test
h
interesting. I'll try to reproduce. In this scenario, I generated the lockfile, ran the test, added
cryptography<37
because
asyncssh
has a warning (which we elevate as test errors through pytest.ini), reran that test that uses
asyncssh
and still got the same test execution. Maybe I just missed the note about lockfile (though I'd expect that to be a show stopper and not a passive log).
h
It should be! It defaults to error. To be honest, it is totally plausible this code path is not working properly. We made a lot of changes to one of the most complex parts of the code base - and we cannot remove that complexity until we are fully confident pex lock files are working great. We do not want to break existing users prematurely I'm gonna go to lunch, but can try to reproduce if you do not beat me to it. Feel free to open an issue if this is indeed not working
h
Great. Thanks for the confirmation and helpful doc links!
❤️ 1
w
it will only error if a dependency is used by some code, but different in the lockfile: do you actually have an (inferred) dep on
cryptography
?
h
From what I could tell about the test I was running, it was a transitive dependency. So the first party code used
asyncssh
which used
cryptography
.
w
mm, got it. so no direct dependency.
h
I believe so
w
the reasoning behind lockfile invalidation only looking at dependencies which are directly* used by a target is slightly stale now… would you mind opening a ticket about this?
h
can do
w
thank you!
h
Doesn't sound like a bug from your description so I made it a feature request: https://github.com/pantsbuild/pants/issues/15318