https://pantsbuild.org/ logo
#development
Title
# development
h

hundreds-father-404

08/24/2021, 5:03 AM
Any UX thoughts on lockfiles here? For tool lockfiles, you set
[black].lockfile
to either
<default>
(provided by Pants),
<none>
, or a path to your own file. It defaults to
<default>
For supporting multiple user lockfiles, you can set
lockfile=path/to/lock.txt
on targets like
pex_binary
, otherwise it defaults to
[python-setup].lockfile
. You can also set
lockfile="<none>"
to opt out. I can't decide if it should be
lockfile=None
or
lockfile="<default>"
to signal to use the global default...I think the latter for consistency and because it seems like
lockfile=None
would be confusing that people might think it meant "don't use a lockfile"
p

proud-dentist-22844

08/24/2021, 3:02 PM
I would prefer using
lockfile=None
over
lockfile="<none>"
Also, maybe treat it as a bool
lockfile=True
use the global default
lockfile=False
or
lockfile=None
opt out of lockfile
lockfile=some/path/file.lock
for an explicit lock file I avoid magic strings wherever I can
🤔 1
👍 1
h

hundreds-father-404

08/24/2021, 3:06 PM
Interesting, thanks Jacob. I was considering the bool approach for the internal factoring, and didn't consider it for the public API Thoughts on
bool | str
for tool lockfiles? If False, no lockfile. If True, default lockfile provided by Pants. If str, path to custom lockfile
p

proud-dentist-22844

08/24/2021, 3:34 PM
Yup. That sounds good.
👍 1
h

hundreds-father-404

08/24/2021, 3:35 PM
Cc @ancient-vegetable-10556 thoughts as someone who's been touching this code a lot?
a

ancient-vegetable-10556

08/24/2021, 3:38 PM
If we can use
--no-lockfile
or
--lockfile=blah.lock
then sure, but
lockfile=True
/`lockfile=False` /`--lockfile=blah.txt` feels pretty awkward
Using `True`/`False` still feels a lot like magic strings to me
I’m leaning towards different flags •
--[namespace-]no-lockfile
for no lockfile •
--[namespace-]default-lockfile
for default lockfile (where applicable) •
--[namespace-]lockfile
for user-specified lockfile
p

proud-dentist-22844

08/24/2021, 3:45 PM
hmm. Is there a way to have exclusive options groups so that only one of no-lockfile, default-lockfile, or lockfile can be included on the commandline or in the config file?
a

ancient-vegetable-10556

08/24/2021, 3:45 PM
Yeah, it’s manual-ish in code, but it’s doable
let me find chapter and verse
p

proud-dentist-22844

08/24/2021, 3:46 PM
And does that mean that in the config file, you’d have to do
no-lockfile=True
?
h

hundreds-father-404

08/24/2021, 3:56 PM
Multiple mutually exclusive options is supported in the options system, but not the target API. I think we want consistency here, so would bias away from multiple options / fields
If we can use
--no-lockfile
or
--lockfile=blah.lock
then sure, but
lockfile=True
/`lockfile=False` /`--lockfile=blah.txt` feels pretty awkward Yes, you could on the CLI use --lockfile, --no-lockfile, and --lockfile=blah.txt
a

ancient-vegetable-10556

08/24/2021, 3:56 PM
That doesn’t feel terrible.
w

witty-crayon-22786

08/24/2021, 4:20 PM
i feel pretty strongly that resolves should have names other than the filename of their lockfile, and that they should be declared in
pants.toml
with an implicit default resolve
1. it’s less redundant on a file-by-file basis 2. it allows us to detect the universe of resolves using only config
h

hundreds-father-404

08/24/2021, 4:39 PM
Ah okay. But even with that, the modeling of "default vs no lockfile vs custom resolve" still is the same. Special strings vs bool | str
w

witty-crayon-22786

08/24/2021, 4:41 PM
i don’t think that we need per-target “opting out of using lockfiles”… that’s a repository level decision i think
so per-target, i think it should be
resolve: str | None
with
None
giving you the default resolve.
you might need to choose a name for the “default” resolve in
pants.toml
, depending on how it’s modeled.
h

hundreds-father-404

08/24/2021, 4:46 PM
I think we probably do want to allow you to opt out of lockfiles for individual targets. Kind of like the lockfile_ignore tag used in generate_constraints.sh. It's an escape hatch and also could facilitate debugging/rapid iteration
w

witty-crayon-22786

08/24/2021, 4:46 PM
and, fwiw: the reason i feel strongly is mostly the latter bit: it feels like it will be important for optimizations (á la https://pantsbuild.slack.com/archives/C0D7TNJHL/p1629402899237900)… if it turns out we don’t need that, then i don’t feel as strongly
@hundreds-father-404: you could do that by putting yourself in a new lockfile, or globally disabling lockfiles
no need to do it on the target itself
h

hundreds-father-404

08/24/2021, 4:53 PM
You feel strongly about the name of the resolve, or not allowing individual targets to disable lockfiles for its build, or both?
h

hundreds-father-404

08/24/2021, 4:55 PM
With globally disabling lockfiles, true, but easily could be overly coarse. If you only need an escape hatch for a single pex_platform, that's a bummer to have to disable everything I can't fully anticipate why you'd need that escape hatch, but that's why I think it's valuable. An escape hatch for unknown unknowns
h

happy-kitchen-89482

08/24/2021, 5:00 PM
I agree with Stu that lockfiles should have logical names distinct from their path.
1
h

hundreds-father-404

08/24/2021, 5:00 PM
Cool, that makes sense to me. I'll plan on that
h

happy-kitchen-89482

08/24/2021, 5:01 PM
The options system won't support
bool | str
as an option type
h

hundreds-father-404

08/24/2021, 5:01 PM
Thoughts @happy-kitchen-89482 on whether individual targets (
pex_binary
,
python_tests
) have an escape hatch to opt out of lockfiles, even if you use them for everything else?
w

witty-crayon-22786

08/24/2021, 5:06 PM
the semantics of https://github.com/pantsbuild/pants/issues/12591 will be important there… if we reach the point where the lockfile auto-updates on use, there isn’t really a difference between opting out and using a throwaway un-committed lockfile
h

happy-kitchen-89482

08/24/2021, 8:03 PM
Hmmm I'm not sure re the escape hatch
Trying to figure out how likely users are to hit a problem
Probably?
h

hundreds-father-404

08/24/2021, 8:11 PM
If our past is any predictor of the future, I do think it's likely there will be unknown unknowns. Good example: in the past, people had issues with
platforms
and
pex_binary
we didn't anticipate, where they needed to disable the constraints file for those
pex_binary
targets. We had no escape hatch except for a global opt out, so we had to cherry-pick a hotfix that always opts you out in that case At least while multiple user lockfiles are in flux (e.g. w/ Pex generation upcoming), a more granular escape hatch seems like great insurance
h

happy-kitchen-89482

08/24/2021, 9:09 PM
Yeah, that's probably right
w

witty-crayon-22786

08/24/2021, 9:10 PM
except that in the case of multiple resolves, they will have the escape hatch of using a custom resolve for a problematic binary
h

hundreds-father-404

08/24/2021, 9:10 PM
Yes, but you still must use the lockfile mechanism. Because of the way lockfiles work w/ being
--no-transitive
, your lockfile must be valid - you can't just use an empty lockfile I think "disable lockfiles everywhere" is too coarse if you only want to disable lockfiles for a particularly problematic target
Some updates: • I still think we want to allow individual targets to opt out of lockfiles as an escape hatch • Now that we have "named resolves", the field becomes
resolve="data-science"
for example • The field defaults to
default
, without any
<>
characters. The resolve's name is simply
default
, which the user will have set in
[python-setup].resolves_to_lockfiles
This leaves open the question of how to model disabling a lockfile, with two options: 1.
resolve="<none>"
2.
resolve=None
First gives parity with tool lockfiles. Wdyt?
Maybe parity is less important because the data formats of the options system (e.g. TOML) != BUILD files (Python)? TOML doesn't have a
None
type Do what's idiomatic over conformity? -- (update) Yeah, I think
resolve=None
is the way to go. It's the simplest modeling code-wise
w

witty-crayon-22786

08/31/2021, 12:52 AM
One thing that would probably be useful is to sketch out the entire user facing API. I had some more thoughts about it the other day that I should probably write down too.
h

hundreds-father-404

08/31/2021, 1:14 AM
Yeah, although I am pretty close to finishing up the UX via https://github.com/pantsbuild/pants/pull/12703 and a followup to add
resolve
field to
pex_binary
The main open question for me is how to handle MyPy and Pylint knowing which resolve to use when running on
python_library
targets. Otherwise, named resolves has solved most of my UX concerns