🚲 🏚️ "Experimental" name bikeshed time 🥼 🧫 In follow up from the last contributor meeting, it's time to talk about the "experimental" prefix we're attaching to some goals, options, and targets to suggest that they will change. While it's good that we have a way to get feedback while explicitly saying that we're not pushing a final version, I don't think just prefixing things with "experimental" is the way forward. Especially if it's included as part of "normal" output: it makes Pants as a whole look a bit more experimental than it needs to be. But there would need to be alternatives to this if we changed it. More in 🧵

For point 3, I'd much prefer

beta_

to

experimental

🙂

I actually think it would be neat if we could leave things in a namespaced form until we've decided to commit to an API

We already to that today with experimental plugins: https://github.com/pantsbuild/pants/tree/main/src/python/pants/backend/experimental Not sure the right sauce for things like options though...

and my preference for name is "labs" because it connotes experimentation but like in a positive way

which probably muddies the waters about the distinction between experimental and non-experimental features

they do, but always as prefixed with `experimental-`… which makes it very, very clear what they are. so yea, it requires a deprecation cycle to rename the goal, but that can easily be a very, very long deprecation. note also though that goals won’t show up if they don’t have any implementations. so

experimental-bsp

and

experimental-deploy

(for example) won’t show up unless the

java

/

scala

and

helm

backends are enabled (respectively). i.e. you already need to have opted into an experimental thing

(FWIW I've never encountered

labs

in the wild, so newbie me would've had to go googling WTF

labs-bsp

is)

I think that having

alpha

or

labs

or whatever in the registration string is fine, but not in end-user facing things like goal names or target names?

I think splitting the registration string from the goal/target name is very interesting. I'm a big +1 for

experimental

FWIW. I think if it's something that's actually in an experimental state, it should be unconditionally clear that is the state. I've quite liked Kotlin's handling of this (https://kotlinlang.org/api/latest/jvm/stdlib/kotlin.experimental/) - because you know what you're getting into when using those namespaces. alpha/beta/gamma/omega or whatever implies that in a release or two, those should have gone away.

So for example, I wish we had not named it

experimental_shell_command

To be more precise, if I saw

beta_target

I probably wouldn't hesitate to use it, assuming it's just a polishing away from being perma.

@happy-kitchen-89482: the issue with

experimental_shell_command

was exactly that we don’t know what the final name should be (…not to mention the rest of the API)

-1 to

contrib

- that is such a vague and uninformative thing. To me it means "this thing is contributed by someone who is not a core dev, so their stuff is second or third class and probably not very good, so YMMV. we are only distributing it, and not supporting it"

Hmm maybe the issue here is that they're not actually all hidden behind registration switches

Hm... that's a bug then.

(having said that: if it had been hidden behind an experimental backend, maybe it would not itself have needed to be marked experimental… but it wouldn’t have dodged the need to rename it when stabilizing)

Exactly, I think the backend should be marked experimental, because changing that name affects one config file line. But the target name should not have been, because it affects potentially many BUILD files.

To me "experimental" connotes that the thing could disappear at any time. That it's under evaluation as to whether it's worthwhile. An experiment can conclude at any time, resulting in either adoption or disposal. Whereas alpha/beta connote that something is intended to be permanent but considered unfinished in some way. These are very different messages. An overabundance of the former suggests we don't have a roadmap and are kinda flailing around trying various stuff out. Which is quite an incorrect impression.

I view

experimental

more as being "volatile APIs" - so odds are if it's a language, sdk, API, or something that even made it to getting published or docs - the concept is planned to exist, but with zero promises on what the API will look like. If I was using a

beta

(or sometimes even an

alpha

) that should come with a reasonable-ish timeframe, and some reasonable API stability

https://pantsbuild.slack.com/archives/C046T6T9U/p1660764205591099?thread_ts=1660758047.365839&cid=C046T6T9U yea, possibly. But it depends whether the audience for the fact that it is experimental is the administrator of the repo, or the end user of a repo. Enabling the backend is something you do once.

I can see a role for all 3 labels. For instance, C support right now is an experiment, yes? Exploring whether and how that might work. Whereas scala support is clearly here to stay. It's been under development for a year, is stable, trusted in production use. Calling it`pants.backend.experimental.scala` leaves the reasonable superficial impression that it's roughly as far along as c support.

that might get to using a catchall name like

unstable

, but then requiring a tracking issue (á la rust, as mentioned in the meeting) which explains “how unstable” it is.

+1

unstable

I like

unstable

more than

experimental

! (Although we should not rename existing things to use that)

If the issue is word connotation, and we wanted to swing it to positive, then maybe what python does with

future

- but I don't think that describes stability

Right now the docs return 80 hits for "experimental", and none appear to define what we mean by that term. I don't blame anyone for making assumptions. How about as an interim measure we start by publishing a definition of what "experimental" status means and doesn't mean within the scope of Pants project, and where to go to learn how close something is to being declared stable?

__future__

in python denotes an API in a future release that is available as an opt-in in the current release. So no wiggle room on how the API will look. So with my python glasses on, I'd rather not use

future

.

API in a future release that is available as an opt-in in the current release

Fair point.

(this is going to be an awesome rainbow-colored bikeshed... I wonder how the nuclear reactor is coming along... 😉 )

@flat-zoo-31952 wanna make a strawman poll, or whats next?

Just to be clear, is the desired goal to keep whatever this magic word is embedded in the registration string only? Or is there still a discussion of whether magic word can leak into targets/goals?

Yeah let's finish having that part of the discussion before moving on to voting.

^ let's clarify this point, I thought Stu had a a decent point about the diff between admin and user views on this: https://pantsbuild.slack.com/archives/C046T6T9U/p1660764522655699?thread_ts=1660758047.365839&cid=C046T6T9U

Is the concern in the event of a rug pull? Or an API change? In the best case, someone could go from

magicword

to stable, and end user would never know. In the non-best cases, prefixing targets with

magicword

guarantees that it'll affect everyone, with ... some ... benefit?

(re Josh's point) True. However reading the docs is also a thing that happens when people are evaluating Pants, and I've heard from more than one company that there were concerns raised about how much Pants appeared to be [experimental|unstable|alpha|beta|unfinished]. So while it doesn't affect admins in practice, every time it's in a backend name it's another iteration in the docs that feeds into spurious internal debates as to whether Pants is even mature enough to adopt.

Using my experiences switching from

kotlin.experimental

or switching over to

androidx

- if

fix

takes care of the boilerplate within a reasonable degree, I typically don't care whether namespaces change.

Keep in mind that there's FUD about Pants that doesn't exist about VSCode. We're constantly dealing with the false belief that Pants EOL'd when Twitter moved on. Most recent time I had to debunk that: this month.

Well, that's interesting to learn

It was brought up in my original internal pitch to migrate (re: Twitter, not

experimental

)

Yeah, it's silly. Oh well!

https://pantsbuild.slack.com/archives/C046T6T9U/p1660765876132949?thread_ts=1660758047.365839&cid=C046T6T9U OTOH, I don't think we want people reading the docs for "shell_command" and thinking "oh great, there's an easy way to port over whatever makefile commands I have and it will just work", only to find out that 1) no it won't, and 2) the target is still in development and might change

I love novel names 🙂

muslin

? https://en.wikipedia.org/wiki/Muslin#Dressmaking_and_sewing

These test-models are sometimes referred to as "muslins" in the United States

There's no association so: 1. People reading it have no assumptions 2. We have to document it Not saying this is my vote to replace

experimental

, just saying if we have to choose one this one is unique and clever 😉

just saying if we have to choose one this one is unique and clever

Or, the EXACT OPPOSITE... ... ...

next

Draw a line in the sand, those that like puns on this side, and those that don't on that side... I like puns 😛

next

implies to me it'll be in the next release

Exactly.

The number of releases with experimental shell support votes against

next

Either way, I'd suggest if there is some

magicword

it tries to be something already used in existing ecosystem of languages and tools, and is not too clever to hinder/confuse non-native-English speakers by default The amount of French punny code I've had to deal with in my life... sigh

Possible sewing terms: • "basting" is putting in temporary stitches to hold things together until you've put in the final ones. • A muslin can be a test garment that you're making out of cheap fabric to test the fit. (it is also a type of fabric) • Tacks are loose stitches to transfer pattern markings onto the fabric prior to sewing. • A Block is a super simple pattern made to your measurements that can be customized into all kinds of things (eg: I have a dress block and a skirt block and when I want to make something I can use those to get my basic fit right and then modify necklines, length, skirt swing, shoulder seam placement, etc however I want.) I think "muslin" is maybe the closest fit for an "experimental" replacement.

I love that we have an actual seamstress on the team that we can consult. (Aside: Tansy's dressmaking skills are epic, btw.)

I have never (yet) made pants though.

"Test the fit" seems like a really accurate connotation, I like that "test" is also right in the mix, which seems apt for a build system. I can see

muslin

making a good blog post and short definition box for the docs. It's not a common term among programmers, but it's not a joke. It's an apt sewing allusion, like

tailor

.

I bow to you Tansy, all my sewing projects have ended in misery, rage, and very... unique designs

I've had a few of those myself :) I took some free online classes that really helped, and there's a book called “the perfect fit” which was amazing! Most of the time now my issues are patterns which look stunning on the pattern packet, but don't suit me at all 😂 Also, every time I gain or loose weight I stop seeing for a while because I don't want to have to redo all my patterns and blocks in a new size 😂😭

omg I opened the floodgates for names

You may have never made pants, but you did help make Pants!

So to summarize here and find consensus so we can move the discussion forward: 1. We need to clearly define 1 or more levels of unstable release states, give them names, and write documents stating what they mean 2. FUD around common-practice names for these unstable release states is real, so using a novel name is warranted in this case to avoid hand-wavy dismissals of Pants as "not production-ready" 3. The names used for these unstable release states should appear in the backend registration names (i.e., python namespaces in which they are defined), but not in actual target, goal, subsystem, or option names Does everyone roughly agree with these points?

I'm personally not really a fan of novel names -- they're not intuitive which is imo a key goal for Pants. They require learning custom jargon. I continue to prefer

unstable

or

experimental

. Although I won't block I agree with 3 where possible. Sometimes it is not, like experimental options for a stable backend (Python) For 1, I encourage limiting ourselves to one level for things that users have to type in config files, so they don't have to migrate from

.alpha

to

.beta

. It's worked nicely to call the code

experimental

, then in our docs clarify whether it means Alpha, Beta, etc

I'm +1. Reaching a little into further discussion, we can always augment a subsystem's options with these by adding a new subsystem which mirrors the existing one. E.g.

python

will have

--bikeshed-python

to "namespace" the experimental options. Public goals can be augmented through a similar subsystem

experimental

->

fudless

🧌

Yeah we should take some time to address FUD vs intuition. I think @hundreds-father-404’s desire to make things as intuitive as possible makes sense but it's not an unalloyed good. Intuition is necessarily bound to preconceived notions. This is both good and bad. It's good in that you can use metaphor to leverage your existing understanding of something to help quickly get a mental model of what's happening here. It's bad in that you will bring whatever baggage you have associated with your existing understanding, which can muddy the waters or cause knee-jerk reactions. On the good side, using a word like

target

probably works well for anyone familiar with

make

, even if Pants' use of it is a bit different. While there's some danger that people may not get, say, all source files can be targets in Pants' model of the world, it's close enough and can be explained. Worse is something like

BUILD

, which a large, large number of people seem to think is intrinsically related to compilation, or Jenkins, or something like that. To me "build" can mean practically any set of actions I take on source code, but that's arguably a very jargon-y definition. Not everyone has read Build Systems à la Carte. I've learned to be careful about this particular word in CI talks because it can lead to so much confusion for people who wonder what I'm doing to compile Python code all the time. It's possible we could have called this process "sewing" and the files

SEAM

files, and while it would be silly, at least using the novel names with little context would signal to listeners that it is a novel concept and you should suspend your judgement for a second to figure out what we mean. My point is that intuition is a neat goal, but it's not an unqualified win all the time, because you take the good with the bad. But I think the razor may be just this: when you want someone to suspend judgement, use novel terms. When you want them to bring their experiences to bear, use familiar ones. I think we need to judicious in how many novel terms we introduce, for fear of learning fatigue, but it's not wrong to introduce some things like that sometimes.

Great point, thanks Josh for taking the time to explain that. I feel like this should be memorialized somehow -- maybe we would benefit from documenting more of the project's philosophy

+1 to Josh and Eric. Really well articulated.

It's possible we could have called this process "sewing" and the files

SEAM

files

In thinking about it, it's a shame they werent called

PANTSFILES

or

PANTSBUILD

files - seems to be de rigueur for each tool to have its own named configurations

Bazel's

WORKSPACE

and

BUILD

would like a word 🙄

You can rename them, we're actually using

.BUILD

internally for the time being

We use

BUILD.pants

, but generally speaking...

I don't actually want to open that can of worms, using the word

BUILD

is fine, it was just an example of the tradeoff that happens when naming things in a way that appeals to intuition: some people may have the wrong intuition

oh yes, that ship has sailed 🙂

I'm a

BUILD.pants

person too - ambiguity significantly reduced

I don't know where this fits into the puzzle, but one thing that worries me is we don't cross-pollinate that much. Therefore naturally different corners of the codebase (mostly split by backend) might do the same thing in slightly different ways. How do we prevent that? And how can we (if at all) remedy it for code already out there?

Good topic, but sounds like a new thread. For here, let's stay focused on getting to a resolution re the original question of

experimental

.

So to go back to our summary points: 1. We need 1 or more levels that are documented. 2. We need a name for it that doesn't inspire FUD 3. Experimental marker name should only be used in backend names Item (3) probably warrants peeling off to a separate thread, as it's mostly orthogonal now. As for (1), I think @hundreds-father-404’s idea that it should be just 1 marker with a tracker issue attached is good, and has precedent in places like Rust. It also means we only need to come up with (and agree on) one name. Any objections here? And of course for (2) we need to duke it out on the name. We've had some good suggestions here, might put it to poll, and a bit more discussion in another thread.

Cross-post from another thread: Aside: I think Kotlin/Jetbrains really nails stability levels, both in description and the end result https://kotlinlang.org/docs/components-stability.html

I would very much like permission to copy and adapt this page.