https://pantsbuild.org/ logo
#development
Title
# development
h

hundreds-father-404

08/18/2022, 8:40 PM
Oof. Now that we track
indexes
and
find_links
in lockfile header, we will possibly be writing secrets to the file. We have a whole section in docs "Authenticating to custom repos" that says how to use env var interpolation to keep your secrets safe; but
lockfile_metadata.py
will only see the final value
the simplest hack is to revert the PR and punt on this...it means we won't eagerly tell users they need to regenerate lockfiles when they may need to. Otherwise, I think we need some way to sanitize the lockfile headers
conveniently, no longer storing the values in the header will make my life easier with local requirements support (--path-mappings). Otherwise, I need to add sanitation of
find_links
given I've far exceeded my timebox, I'm leaning towards reverting the metadata tracking. @witty-crayon-22786 thoughts?
w

witty-crayon-22786

08/18/2022, 8:45 PM
given the discussion in the other thread re: using versions to differentiate, it definitely seems ok for it not to be in the lockfile.
h

hundreds-father-404

08/18/2022, 8:46 PM
like, fundamentally okay to not ever store in the lockfile? The idea with storing it is it may impact the result of the lock, so the only safe thing to do is regenerate
b

bitter-ability-32190

08/18/2022, 8:46 PM
Out of curisoity, why does the header have to be plaintext JSON? Can we hash the JSON and store that?
w

witty-crayon-22786

08/18/2022, 8:47 PM
no. i could imagine it being a good idea. but since attempting to have “the same artifact with different content in two repos/indexes” is A Very Bad Idea, the consequences of not regenerating the lockfile are pretty slim.
👍 1
you’ll fail to consume it, rather than getting surprisingly invalid data
Out of curisoity, why does the header have to be plaintext JSON?
Can we hash the JSON and store that?
at least some of the fields do need to be readable. so it would be partial.
b

bitter-ability-32190

08/18/2022, 8:48 PM
Got an example?
h

hundreds-father-404

08/18/2022, 8:48 PM
Can we hash the JSON and store that?
This is actually how I was doing constraints files originally. But a user suggested we change to plaintext because it's much more convenient for git diffs and merge conflicts https://github.com/pantsbuild/pants/pull/16469
b

bitter-ability-32190

08/18/2022, 8:48 PM
Oh like
version
?
w

witty-crayon-22786

08/18/2022, 8:48 PM
for lockfile updating vs from-scratch creation: we need to read the previous requirements to delta
and yea, human readable is good.
h

hundreds-father-404

08/18/2022, 8:49 PM
okay, I think I should turn off
indexes
and
find_links
for now, given not tracking it is lower risk, exceeding my timebox, and simplifies local requirements support
👍 1
w

witty-crayon-22786

08/18/2022, 8:52 PM
attempting to have “the same artifact with different content in two repos/indexes” is A Very Bad Idea
one unfortunate exception to this is local source/git requirements… i’m very not familiar with how John went about supporting that, but frequently folks will try to live-edit sources and then have them consumed without bumping the version
not sure if relevant here.
h

hundreds-father-404

08/18/2022, 9:06 PM
Hm, actually how do we feel about @bitter-ability-32190’s recommendation to use a hash of the indexes and
find_links
? That isn't very hard to do. Alternatives: • no tracking at all • some sanitization scheme, which probably means some new mechanism for interpolation. complicated and duplicative of current interpolation support
That isn't very hard to do.
eh, but I guess it still complicates my
--path-mappings
work. So I'll continue with removing this. (It's not an API change, we haven't released this yet.)
b

bitter-ability-32190

08/19/2022, 12:52 PM
for lockfile updating vs from-scratch creation: we need to read the previous requirements to delta
I'm thinking that ought to be the responsibility of the underlying tool, but I'm very likely to be wrong 😅