https://pantsbuild.org/ logo
#development
Title
# development
b

bitter-ability-32190

07/15/2022, 12:29 AM
I'm picking up renaming `fmt` to `fix`. Speak now or forever hold your peace
πŸ‘ 3
πŸ‘ŽπŸ» 1
πŸ‘Ž 3
πŸ˜’ 1
😐 2
βœ… 1
@polite-garden-50641 I don't see a comment on the ticket, can you elaborate on πŸ‘Ž ?
a

ambitious-actor-36781

07/15/2022, 1:01 AM
It's just going to be annoying telling my team whos just getting used to
fmt
to use fix.
p

polite-garden-50641

07/15/2022, 1:06 AM
tbh.. feels like busy work… I don't see the actual user benefit. I didn't (and won't ) bother reading the ticket (there is so much comments on it…) feels like a total waste of brain cycles πŸ€·β€β™‚οΈ There is so much other stuff that could be done to improve usability and perf.. that I don't see the point of doing this kind of work or even considering it.
πŸ’― 1
b

bitter-ability-32190

07/15/2022, 1:07 AM
@ambitious-actor-36781 I can sympathize for sure. My own dayjob's users will likely have a few bricks to lob my direction. Rest assured we'll be following our deprecation policy (and possibly extending the normal cycle), where your code and commands are still valid after an upgrade (save for some warnings or forced settings). So even after upgrading you can still use
fmt
for at least a version (if not more, this is a large change). I also want to highlight that this policy exists so Pants can evolve over time. As we decide we want to do more interesting and creative things, or find the err in our past ways we have two options: β€’ Bend: in this case that means making these breaking changes slowly and methodically with some heartburn along the way β€’ Break: Slap a new version (Pants v3) or a new name because we broke too much I'm glad we're dedicated to bending so that, over time, we can continue to grow internally and with the community.
@polite-garden-50641 ouch 😞 I hope you're just having a bad day, because your message is very disheartening. If you aren't going to read past the surface-level title and try to understand the reasoning, I'd appreciate if you didn't use such negative language and followed the code of conduct.
❀️ 3
That being said, in hopes you are willing to try and understand the change: There's an existing proposal written by @hundreds-father-404 which has been open for comment for a while. The TL;DR on user impact is that having both fixers and formatters in one goal means I can't use
--loop
or
fmt-on-save
because fixers make semantic changes that undoes dev work. This was seen as the best way to fix that without proliferating goals.
c

careful-address-89803

07/15/2022, 4:19 AM
I was concerned that this would be a pain point for teams which recently onboarded to Pants (like mine), since they'd have to change the command they enter most often. But if we have
fix --style-only
then we can keep
fmt
around as an alias for low (no?) cost. I think if we keep it around for long enough (maybe even until 3.0), this won't be a problem.
πŸ‘ 1
b

bitter-ability-32190

07/15/2022, 10:51 AM
So part of the motivation as well is to avoid a proliferation of goals. So eventually
fmt
will be removed. However you'll love the fact that you can define your own CLI aliases: https://www.pantsbuild.org/docs/reference-cli So
fmt
could always live on in your repo or on your machine as an alias ☺️ (FYI @ambitious-actor-36781. I forgot to mention this as well)
πŸ‘ 1
w

wide-midnight-78598

07/15/2022, 1:09 PM
Having been pretty involved in commenting on this one - did any "final" proposal actually land? I feel like Eric's gdoc and the Github issue all had a lot of well thought-out pros/cons, but it wasn't as simple as a re-naming, it was basically revisiting the entire `check`/`lint`/`fmt`/`tailor`/`update-build-files` API to see what should be merged, what needed options, what should be re-named. For example, at the end of the proposal, it says something with an "if", giving it a sense of not being a final decision:
If we do want to add non-safe "fixers", we should consider using a dedicated goal like tailor or something new.
@bitter-ability-32190 Could you maybe enumerate what changes that ticket would cover, so we have an idea of the scope of API change? Anything that reduces the number of goals and improves goal semantics gets a big πŸ‘ from me. Conversely, anything that changes the external API but leads to basically "the same thing" gets a huge πŸ‘Ž ... Hence averaging out to strongly neutral...
p

proud-dentist-22844

07/15/2022, 1:34 PM
I need to regenerate some API spec files, but the files have to be checked in because people consume them via GitHub raw content. Codegen seems like it would be perfect, but nothing in the repo depends on those files. I ended up adding a
fmt
plugin for it to ensure it happens regularly. But I cringe because this is very much not formatting.
fix
would not cause the cringe. So a big πŸ‘ from me.
b

bitter-ability-32190

07/15/2022, 1:45 PM
In the end I believe update-build-files will be folded into
fix
, but we need to have
fix
for that to be allowable. Additionally I would expect the formatting aspect v the fixing aspect to be split.
tailor
I believe was determined to be special enough to remain. I don't think we discussed non-safe fixers, however I wouldn't want us to make design decisions based on "maybe" future state (after all, as said above we have ways of changing over time) So this is the first step to a few of those possible additional tasks, which is why I'm picking it up.
βž• 1
I do plan on picking up some of those additional tasks, like merging update-build-files. My end goal is to upstream some in-repo changes, but these are precursors
f

flat-zoo-31952

07/15/2022, 2:11 PM
I understand the reasoning here, but churn on public APIs has a strong impact. As someone who's fighting an uphill battle to even get adoption in his org, this kinda design churn hurts. It makes arguing cases for Pants much harder, because this is the kinda stuff that our users are expected to use.
βž• 1
h

hundreds-father-404

07/15/2022, 2:18 PM
@flat-zoo-31952 what about setting up a
[cli].alias
so that
fmt
==
fix
in your repo, meaning people don't have to change if you don't want them to?
f

flat-zoo-31952

07/15/2022, 2:21 PM
It's one thing to churn in the more internal parts of Pants: plugin API, more sophisticated options, even target field namesβ€”because these things are mostly irrelevant to a typical developer's workflow, and can be managed by a team of specialists who follows Pants changes more closely. But goals are the most visible part of the Pants public API that would be used by a broad subset of users. And changing this stuff too often sends a subtle message that this is beta software. I'm of the opinion that if a goal is added, it should be kept for quite a bit longer than the current deprecation policy suggests. Perhaps deprecated goals should remain available as pluggable backends for several version cycles? It's some maintenance and some extra tests, but I think that's part of the cost of building software that people actually use
I'd prefer a backend over an alias since the options aren't the same... the bulk of the logic could probably be factored into a non-goal rule
b

bitter-ability-32190

07/15/2022, 2:33 PM
Actually the options will be the same, plus a new one. All of the options on
fmt
are valid for
fix
. So unless you're using the subsystem name (e.g.
--fmt-only=...
) You shouldn't see the difference
Re churn: if we determined that the proposal is the direction we want to go, it'll hurt less at 100 adopters than 1000.
βž• 1
FWIW I'm feeling a lot of intensity behind the rename, but unfortunately not a lot of suggestions for how to fix the underlying problem a different way, or concrete ideas on how to manage the pain. That's admittedly very disheartening to have the discussion be surrounded by the cover of the book 😞
If instead of renaming we deprecated fmt, and made it a subset of fix, would that alleviate some concern? I'm not sure how feasible it is, but I can try
h

hundreds-father-404

07/15/2022, 2:44 PM
That could be an option. And then I like that both
fmt
and
fix
can show up in
./pants help goals
, and we can explain
fmt
is a (deprecated) subset of the
fix
goal, which does not include semantic "fixers" like PyUpgrade and Scalafix
we intentionally marked all of our "semantic fixers" as experimental backends, so there's more flexibility to remove those from the
fmt
goal and only run them with
fix
f

flat-zoo-31952

07/15/2022, 2:57 PM
Re churn: if we determined that the proposal is the direction we want to go, it'll hurt less at 100 adopters than 1000.
That's fine, but that is very much a beta software approach. And at some level I can't argue with it because I get the need to move forward, and I get that design is an iterative process. And it works within your depreciation policy. But I can say as a user it's frustrating. I don't think about Pants every day. I can't afford to. I don't think I'm alone in that. So when I come back and I see that goals are moving again, it takes mental energy to figure out what's going on, even if there are workarounds available. This change sounds okay I guess, and it does show that you think about your users in this. But decisions like this clearly get made by people who get to think about Pants all the time. So it still feels like you're missing a "person who uses Pants but has other duties" UX persona from your design process. So I don't have a great logical argument here. I just have the observation that churn is deeply frustrating as a user, especially for things that for all intents and purposes felt fine.
πŸ‘ 2
Deprecating fmt as opposed to renaming sounds like a good plan
b

bitter-ability-32190

07/15/2022, 3:10 PM
Goals are moving again
Can you elaborate? I'm getting the impression there's a past experience that bit you as well, and I'd love to know what it was. And can you help me understand what would make this change easier to digest if you were reading the release announcement or however you consume an upgrade?
πŸ‘ 1
FWIW
fmt
was always going to be deprecated, we don't believe in rugpulling. However we can move the slider on overall UX with that deprecation
w

wide-midnight-78598

07/15/2022, 3:14 PM
I'm feeling a lot of intensity behind the rename
The consequence of touching a public API, unfortunately - broad-reaching implications, with potentially minimal benefits for most/all of the user base
concrete ideas on how to manage the pain
I think as mentioned in either the GH issue, or GDoc, or maybe even this thread - if there is/are built-in aliases, or automation ways to solve this - those are decent pain mitigation strategies. When there are BUILD file target name changes (which is also a public API change), we have a dedicated command that fixes them and it's approximately zero mental exertion other than reading a deprecation note after updating pants. No big deal. If
fmt
stops existing (or any goal, actually), then there is some amount of mental exertion, and there might be build scripts, CI tools, etc that all need to be updated in any repo using Pants. I have a bunch of Pants repos, and that's not a big deal for me - but for some people, that's much harder due to larger scripting facilities, or just wayyyyyy more people involved in the process. My assumption/hope would be that on any public API change, there is a "lengthy" deprecation time, automated script updates, and documentation notes the deprecated version but auto-redirects to the newer version.
πŸ‘ 1
πŸ‘πŸ» 1
h

hundreds-father-404

07/15/2022, 3:19 PM
@bitter-ability-32190 I wonder if we could keep
fmt
in like a "hidden" state for a long long time, even after a long normal deprecation. Like, it doesn't show up in
./pants help goals
, but if you type it, we treat it as
fix --style-only
, and print a short deprecation saying to use
fix
instead. Seems very technically feasible
a short deprecation
To reduce CLI noise. And Pants users can use the
ignore_warnings
option if they really don't want that warning
πŸ‘ 1
w

wide-midnight-78598

07/15/2022, 3:19 PM
I only sorta get that "API churn means beta software" feeling, but like - that might be a language or industry thing too. Any web developer knows how severe API churn can get, while I have C libraries from 15 years ago I can still plug and play. So long as the API change leads to a tangible improvement (knowing that "improvement" is subjective), I think they're fair game, even in build tools. BUT! I will say that when it comes to API changes, I, personally, would like to suffer pain all at once, rather than every few releases, seeing a goal change or break.
h

hundreds-father-404

07/15/2022, 3:22 PM
with potentially minimal benefits for most/all of the user base
For clarity: the intended benefit of this change is so that we can move forward adding semantic fixers to what is now the
fmt
goal, including merging
update-build-files
. Technically, we could have
./pants fmt
do things like BUILD file deprecation fixers...but that semantically is wrong
w

wide-midnight-78598

07/15/2022, 3:22 PM
"hidden" state for a long long time, even after a long normal deprecation
I'm struggling to remember the tool or language I saw this in, but one pattern I've seen is that public API changes are never officially breaking until a major version change. I know that just sounds like semantic versioning, but it was pretty awesome. The deprecated API was basically invisible unless you had already used it in the past. So, new users wouldn't really even know it existed, while existing users were completely unaffected until a 3.0 release.
πŸ‘ 1
πŸ‘πŸ» 1
f

flat-zoo-31952

07/15/2022, 3:22 PM
Can you elaborate? I'm getting the impression there's a past experience that bit you as well, and I'd love to know what it was.
I think I just value API stability a lot. It's come up before, although I can't recall the specifics now, and the Slack history is probably gone. IIRC I think it was some changes that were made in prep for target generators, which ended up being a really good idea, but it was frustrating to deal with then. There's a lot of smart people working on this project, and often things are being done to make way for some really cool ideas in the future, but the benefit to users in the meantime is often very unclear in the short term. This is normal with any kind of upgrade process.
h

hundreds-father-404

07/15/2022, 3:23 PM
It's come up before
python_library -> python_sources
, iirc. It was SUPER disruptive, even though it was automated
It was SUPER disruptive
Not only BUILD file churn, but you had to update things like
./pants filter --target-type
to differentiate between
python_source
vs
python_sources
, in some cases
w

wide-midnight-78598

07/15/2022, 3:24 PM
Technically, we could have
./pants fmt
do things like BUILD file deprecation fixers...but that semantically is wrong
@hundreds-father-404 Exactly. The semantics would matter to someone like me... but, to 90% of the userbase? Who knows if they're as pedantic as me πŸ™‚
h

hundreds-father-404

07/15/2022, 3:25 PM
I think I feel strongly against
fmt
running "semantics-changing fixers" because a major design goal for Pants is to be _intuitive_: you can figure it out without reading docs prior knowledge. Which is key because we try to assume most of our non-Slack users barely read our docs
f

flat-zoo-31952

07/15/2022, 3:26 PM
Yeah, but I don't see a problem with goal proliferation, especially if you can find a way to make some of the less desirable goals "hidden" somehow
h

hundreds-father-404

07/15/2022, 3:27 PM
yeah, I am fine with keeping
fmt
around ~indefinitely, as long as new users only encounter
fix
. it's not much code for us to support that
f

flat-zoo-31952

07/15/2022, 3:27 PM
But I could be wrong, and I know that doing this universally causes confusion and non-trivial maintenance burden
Any web developer knows how severe API churn can get, while I have C libraries from 15 years ago I can still plug and play.
A lot of the reaction to this is going to come down to where you expect Pants to be on this spectrum. I don't expect 15 years stability, but maybe 1 year is a better target than 2-3 months (which is where the current deprecation policy tends to land with 2 minor versions)
I'd also like to know the end-goal of some of this... is there expected to be a point at which the API churn slows? When we find the right set of goals that do what people want in general? I think that's what I mean by the "beta software" comment. It's fine that we churn to make changes that drive the project forward. But we've gotta be pushing towards a point at which some of this becomes more stable. Right? πŸ€”
b

bitter-ability-32190

07/15/2022, 3:39 PM
Is that not a product of learning from our mistakes? Eventually the pendulum stops swinging because we found the right steady-state
To round this out, when I pick up the folding of update-build-files we can do the same kind of dance
b

busy-vase-39202

07/15/2022, 5:22 PM
@flat-zoo-31952 @ambitious-actor-36781 @wide-midnight-78598 Thank you so much for bringing up this perspective. You're absolutely right that the majority of the maintainers team aren't in your position as a Pants admin supporting a lot of new and potentially resistant end-users, and underrepresentation of that perspective is a known weakness (which we've been actively working to remedy by recruiting more people like y'all, including @bitter-ability-32190, onto the maintainers team). Please do continue to advocate here for your end-users and keep helping us understand their usage and concerns as well as your own as a Pants admin. Feedback like yours is so valuable and necessary. I hope we ultimately get to have each of you on the maintainers team. And please do reach out to me anytime if you'd be interested in pursuing that as a goal!
❀️ 3
h

hundreds-father-404

07/15/2022, 5:39 PM
But we've gotta be pushing towards a point at which some of this becomes more stable. Right?
My two cents is that certain parts of the system should be stable stable, whereas others we will want to have time to discover the best representation. Specifically: experimental backends. Whereas ideally we stop changing fundamental things like CLI specs, the core goals, targets in general
πŸ‘πŸ» 1
b

bitter-ability-32190

07/15/2022, 10:27 PM
@flat-zoo-31952 @ambitious-actor-36781 @polite-garden-50641 @wide-midnight-78598 What's y'all's feelings on just adding a
fix
goal separate from
fmt
? We'd still dissolve
update-build-files
into both goals. But we've introduced another goal. I would say some of the pain of a new goal like this might be alleviated with fixing https://github.com/pantsbuild/pants/issues/10542. Then we can revisit the merging separate from the above
πŸ‘ 1
(personally this is my favorite option and @fast-nail-55400 's, but of course, the community is what helps us decide)
w

wide-midnight-78598

07/16/2022, 12:25 AM
I read @fast-nail-55400’s latest issue comment, and I think it was an excellent synopsis of the practical issues with changing the public API. As I stated early in the Github thread, I dislike API-creep, and I'm a bigger fan of
./pants lint --fix
in lieu of a dedicated
fix
command, but that might just speak to what my day-to-day languages/tools are (not Rust with
cargo fix
and not Go with
go fix
). I'm not sure I understand what the technical scope of
fix
would be (as in, what identifies that which is `fix`able? Moreover, would we have the idea of a
fix --check-only
for CI? that just seems strange) If
fmt
was relegated to whitespace, unused imports/vars, braces, indents, and strictly visual presentation - I'm a huge +1 on that (no semantic or code-flow changes, and nothing that would change compiled byte-code). I think
fmt
is currently like 95% this, other than like 1 or 2 plugins. Would
update-build-files
have its duties split into two?
fmt
handles the visual presentation, while
fix
would handle any semantic tweaks (ie. target name changed)? I think this is the most practical, community-centric approach. Nothing severe breaks, additive API, and then deprecating and removing an API which becomes redundant.
πŸ‘ 1
b

bitter-ability-32190

07/16/2022, 12:27 AM
Yup you got the gist. Lint would also run fixers to check code, just as formatters. So your CI still only needs lint and check.
f

flat-zoo-31952

07/16/2022, 12:32 AM
Adding makes more sense to me in this case. I didn't fully get why these needed to be grouped together in the first place.
πŸ‘ 1
w

wide-midnight-78598

07/16/2022, 12:36 AM
Go - "Fix finds Go programs that use old APIs and rewrites them to use newer ones. After you update to a new Go release, fix helps make the necessary changes to your programs." Rust - "cargo-fix - Automatically fix lint warnings reported by rustc" Which one is closer to what we would do?
πŸ‘ 1
f

flat-zoo-31952

07/16/2022, 3:25 PM
After going over this again, i'd argue that
fix
is too general a term for these actions, at least as a goal. And it would be confusing to have it present along with
lint --fix
if that was adopted as well
update
has the right ring to it for the sense of
go fix
but it's an overloaded term as well, not sure there are terms that aren't overloaded in this sense
p

proud-dentist-22844

07/16/2022, 3:27 PM
I think I would expect
./pants update
to update itself
f

flat-zoo-31952

07/16/2022, 3:27 PM
It's kinda what's nice about a term like
tailor
, even if it comes from a silly pun, is that it has no overloaded expectations of behavior
yeah not really suggesting that
./pants update
be used
modernize
?
semantic-fix
? something about naming things being hard...
b

bitter-ability-32190

07/16/2022, 3:30 PM
If other ecosystems already have the notion of fix, why wouldn't we borrow that schema?
I don't think lint --fix will stick. Lint doesnt change the workspace and I think that's here to stay
βž• 1
f

flat-zoo-31952

07/16/2022, 3:31 PM
Because their notions are different
Go uses fix to mean "update API calls" where as rust uses it to mean "fix compiler lint warnings"
Go has
go fmt
for fixing lint issues
I don't think there's a way to group the verbs these tools use without being a bit reductive
b

bitter-ability-32190

07/16/2022, 3:58 PM
It doesn't have to be perfect, just good enough. Also I think brevity is important here, as we want it to fit in with fmt and lint
βž• 1
f

flat-zoo-31952

07/16/2022, 4:03 PM
Idk about "good enough", and when creating a new API it's good to be careful. Otherwise you have a tendency to back yourself into corners it's hard to get out of once people start binding themselves to the workflows you've allowed them to create
Even the razor proposed early in the issue is a bit wobbly to me:
That might be the right razor to use: do I need to run it as part of CI?
Different teams are going to have different standards for this. Some teams will auto-commit certain fixes in CI. Do we need to make a separate
fix
command for that? Idk, in my mind, a notion as broad as "fix" can't be given the precise contextual definition it needs in a tool as general as Pants. I think getting this right is far more important than brevity
b

bitter-ability-32190

07/16/2022, 6:07 PM
Seems like at this point we're not discussing if there should be a new goal, but what it should be named. I'll make a new thread so we can discuss without this threads history.
πŸ‘ 1
w

witty-crayon-22786

07/18/2022, 5:25 PM
just caught up: thank you everyone for the thoughts in this thread, and to @bitter-ability-32190 for your patience. my takeaways are: 1. we should probably consider committing to a longer deprecation cycle for β€œthe most visible” user APIs like goals. or at the very least start a convention about choosing appropriate deprecation periods for changes, rather than always going for the minimum period. 2. the pivot to a discussion on naming over here is great. if discussion starts to slow down, consider writing the choices up as a contentious decision vote (not necessarily because it’s contentious, per-se, but mostly to ensure that the choices are clearly enumerated, and everyone interested gets heard)
βœ… 2
πŸ™Œ 2
b

bitter-ability-32190

07/18/2022, 7:50 PM
Did you have an opinion on new goal name for the other thread?
w

witty-crayon-22786

07/18/2022, 9:19 PM
not yet, but if you’re trying to decide today i can take a look
b

bitter-ability-32190

07/18/2022, 9:24 PM
No, today is mostly for resting πŸ€’
My end goal is dissolving update-build-files
πŸ‘ 2
w

witty-crayon-22786

07/18/2022, 9:27 PM
will try to before tomorrow.
b

bitter-ability-32190

07/18/2022, 9:31 PM
My actual end goal is to dissolve that goal so I don't have to run it, and to allow for in-repo build file formatters, dissolving a chunk of my in-repo plugins πŸ˜… One PR I can make today is a "<none>" formatter option for build files (required if I'm gonna plug my own)
6 Views