I ran into a frustrating issue with the new "uploa...
# general
c
I ran into a frustrating issue with the new "uploaded prior to" functionality. I'm intending to set a default of 7 days, however when generating a lockfile I need to be flexible and allow people to override that when necessary to pick up various package fixes. In this situation pants gives an error because the "uploaded prior to" value in pants.toml was different to the value in the lockfile metadata. I don't think this should produce the error.
h
How are you overriding this when generating a lockfile?
c
Just the command line argument in the documentation. Apologies I don't have access to it right now. This feels like a property of the particular lockfile generation invocation and doesn't speak to the compatibility of the lockfile itself (unlike interpreter constraints, for example)
f
The users can override the option on the Pants command line.
I assume you are using
--python-resolves-to-uploaded-prior-to="{'RESOLVE_NAME': 'TIMESTAMP'}"
?
What's the error?
c
Yes that's the command line argument that I used. The issue was caused when running "pants package", the error stated that there was a mismatch in the "uploaded prior to" value between the lockfile metadata and pants.toml.
f
Please post the exact error log including any stack trace (but redact any confidential information).
c
Apologies it's not an exact error, I actually finished up at the company yesterday and no longer have access to the machine
f
I actually finished up at the company yesterday and no longer have access to the machine
ack
So the issue arose when trying to consume the existing lock file with a subsequent
pants package
where the default uploaded prior to was in effect but the lockfile metadata had the shorter time value from when it was specified on the CLI during lockfile generation?
c
It was close to that, I didn't have the pants.toml config at all, there is a program that we call that orchestrates lockfile generation in a container with snyk vulnerability scanning, was relying on the default value always being passed into pants when calling generate-lockfiles.
f
so when
pants package
was called, was
--python-resolves-to-uploaded-prior-to=
set at all (whether in
pants.toml
or on CLI etc.)?
trying to understand if the config was unset or had a different value from the lockfile metatdata
c
Apologies, the config wasn't set when
pants package
was called
Wasn't set via cli arg or config
h
I think there's an argument that lockfile metadata validation should not care about this field. Honestly, there's an argument that almost all lockfile validation is overengineered and unnecessary. However, imagine if Pants automatically regenerated the lockfile (as many users expect it to, and as
uv
does, for example): In that case a change in that field would cause a new lockfile to be generated at use time. So this validation would be reflecting the expected reality. My point being that your use case is actually also expressing that you don't ever want to autogenerate lockfiles, and only want them to be generated when explicitly asked for?
So I could go either way... Anyone else have thoughts?
c
Since python dependency resolution can execute arbitrary code I personally feel uncomfortable with doing it automatically (even if it's in a reasonable sandbox)
h
good point.
uv
will regen lockfiles before
uv run
, so I guess that is less of a concern if you're about to run code anyway. So maybe on balance removing that over-strict validation is the way to go.
c
Is automatically regenerating lockfiles a pants design decision or is it something from uv? Maybe I'm being too conservative but supply chain attacks increasingly scare the shit out of me and the higher the frequency of lockfile generation the higher the risk.
h
Pants doesn't do it at all.
uv
does, so it seems like what users want (and we might make such a change in the future). But I should clarify that
uv
only does this if inputs to the lockfile have changed. The assumption is that if you have made such a change then you want the lockfile to be updated to match.
uv
will regenerate if the thing you changed is the "uploaded prior to" value. And you're effectively saying that you would not want that to happen. So I think the solution would be to be able to opt in or out of auto-regeneration (again, this is in a hypothetical future where we offer such a feature). To be clear, this has no immediate practical implications. I'm trying to tease out the semantic implications of not invalidating the lockfile based on this specific input value.
This is a thought exercise to better understand the question
c
I feel like anything where time is an input is really hard to fit into a deterministic caching system. Like in principle the SHA of the index is an input and we should invalidate whenever that changes, but that is clearly unworkable. I sometimes wonder if it would be better if Pants knew nothing about generating lock files and it was closer to a regular input, but all the use that got us to the current muddle are still reasonable.