https://pantsbuild.org/ logo
Join Slack
Powered by
I think I've asked this before, but when (if at al...
# general
h
I think I've asked this before, but when (if at all) does pants determine a lockfile needs to be updated?
h
If any of the inputs to the lockfile generator have changed. Typically this means either your requirements.txt (or equivalent), or your interpreter constraints
h
hmm, that's what I thought it was. I have a scenario right now where I generated my lockfile with 
pyscard<2.0.4
, then changed my requirements file to 
pyscard>2.0.4
but I'm not getting any notice about my lockfile needing an update.
Is there some behavior I need to turn on to have it validate the lockfile?
I do see 
pyscard<2.0.4
in the lockfile metadata
h
Hmm, which version of Pants?
h
2.13.0
The only command I've run after changing 
requirements.txt
is 
./pants test
in case that's relevant.
Did a 
./pants package
on something else and did not see different behavior just to make sure.
If I add a completely new entry to 
requirements.txt
, I still don't see any invalidation happen (double checked I didn't change 
[python].invalid_lockfile_behavior
)
Is it possible that the lockfile validation is only working with the subset of the lockfile/requirements needed for the goals I am running?
I think that's what's happening
h
Is it possible that the lockfile validation is only working with the subset of the lockfile/requirements needed for the goals I am running?
Ahhhh, yeah that is it!
h
I'm kind of torn on what I'd like to happen. I guess at some point, you should be running something (in CI or otherwise) that teases out that behavior.
h
Oh, yeah, makes sense.
Interesting question what the right thing to do is
h
I think we wanted to avoid the performance implications of needing to consult 
AllTargets
to discover what goes into the lockfile. But honestly, that concern may be overblown: we already use 
AllTargets
for dep inference
h
It also seems like this validation only worries about direct dependencies potentially? In this scenario, we're using 
yubikey-manager
. 
yubikey-manager
needs 
pyscard>=1.9,<3
. 
pyscard==2.0.4
has installation issues so we add an additional requirement of 
pyscard>2.0.4
. Running my test that uses 
yubikey-manager
does not trigger an invalid lockfile error
h
yep, only tracks direct requirements. we want to avoid making network calls so don't know what the transitive deps of your third-party requirements are
h
Don't you have to know at some point if I am to build/run/test things?
h
those third-party deps get resolved when building 
requirements.pex
, but that gets cached. and they're agnostic to Pants the whole time
h
What would you recommend for maintaining visibility on something like that? We have lots of cases where packages we directly depend on specify something incorrectly for their transitive dependencies and we have to fix that by adding more entries to our 
requirements.txt
.
My only thought is to supplement with a unit test that makes sure the entries in 
requirements.txt
exactly match the metadata of my 
constraints.txt
h
The transitive third-party deps are the output of the lockfile generation, and we should invalidate based on inputs, so I’m not sure I understand the question?
h
So scenario is • I use package A • package A depends on some range of package B • that range of B actually has issues • I add a constraint in requirements.txt for package B (even though I don't import it directly)
This is, without a doubt, unavoidable.
There are multiple dependencies in our system that leave their transitive dependencies either too flexible or just verifiably wrong that I am left to either 1) find a different package to use or 2) add my own constraints in requirements.txt to force package resolution to choose a valid transitive dependency.
I think my scenario is a little easier than the general behavior because I keep all dependency needs in 
requirements.txt
. There's no searching through all targets to find places where other 
python_requirement
targets have been called out. So I can ask my system the simple question like "does the metadata in the lockfile exactly match requirements.txt"
h
So this is what pip’s constraints.txt files are for. @hundreds-father-404, do we plan to continue to support those as inputs to a lockfile (separate from our former janky use of them as sorta-kinda lockfiles, before we had proper lockfiles)?
👍 1
h
6 Views
Open in Slack
PreviousNext
Powered by