https://pantsbuild.org/ logo
Join Slack
Powered by
2.11 release timeline: we've already done `dev3`, ...
# development
h
2.11 release timeline: we've already done 
dev3
, which means we're supposed to start 
rc0
today/this weekend per our release strategy. We reflected in Pants 2.8 it was bad we violated this. Anyone trying to get things into 2.11 still?
Our docs:
Usually, we release 3-4 dev releases before switching to the first release candidate rc0. This means we usually release dev0, dev1, dev2, sometimes dev3, and then rc0.
We try to limit the number of changes in each stable release to make it easier for users to upgrade. If the dev releases have been particularly disruptive, such as making major deprecations, we may start a release candidate sooner, such as after dev1.
lockfile blockers: • adding Pex lockfile generation https://github.com/pantsbuild/pants/pull/14740 parametrization blockers: • https://github.com/pantsbuild/pants/issues/14521 • https://github.com/pantsbuild/pants/issues/14430 • https://github.com/pantsbuild/pants/issues/14519 @bitter-ability-32190 I think you might have two at least: • asset inference https://github.com/pantsbuild/pants/pull/14049 • deprecating old Options API @curved-television-6568 anything from you?
b
I would 😭
👍 1
h
@witty-crayon-22786 *some of those parametrization things are big, and https://github.com/pantsbuild/pants/issues/14519 could be quite involved in that it changes the semantics of explicit dependencies. What do you think between: 1) Blocking cutting rc0 on them, vs 2) Cherry-picking them, vs 3) Landing in Pants 2.12 and 
parametrize
is a soft launch in 2.11
w
am working on the parametrization blockers, but i think that most of them are likely fine to pick.
changes the semantics of explicit dependencies
it shouldn’t. it makes something that fails currently succeed. i’m working on it right now, but the Specs parsing is the thing that is probably largest.
imo, with this much outstanding, let’s wait till early/mid next week so we don’t get too bogged down in cherry-picking. once we get one or two more in can cut.
h
but the Specs parsing is the thing that is probably largest.
This seems totally fine to me to cherry-pick
w
maybe. but it will change address orders.
👀 1
(you wanted params after generators, but it’s the other way around right now.)
👍 1
h
Right, which I still agree with. Would it make sense to do that as precursor PR to unblock the release if specs doesn't make it in time?
let’s wait till early/mid next week
sgtm
w
yea, maybe.
f
when are we thinking compiled codgen will land?
2.11 or 2.12?
w
2.11
Chris just landed for the JVM
f
and go protobuf is ready for review
👍 1
h
(I still have to review) Thoughts for both Go and JVM on dependency inference of those languages importing the compiled code? That blocks publicizing the new feature. But we could cherry-pick it like we did with Thrift and Pants 2.10
f
The Go code already has a rule to parse the import path out of protobuf files, and that data is used to adds deps on generated code. It’d be 2-3 lines to bring that to dep inference for regular Go code.
given that the JVM package name is defined in a similar manner in protobuf files, a similar strategy to what I used for Go can be used for Java/Scala
(for protobuf, still would need to think about thrift)
w
spending a bit more time on the inference side for JVM next week would be good, yea. cc @ancient-vegetable-10556
a
I can try to answer any questions while I’m out, but I don’t expect to be near a useful keyboard much next week
w
oooh, yea: that’s right. sorry. i can look at it.
e
https://github.com/pantsbuild/pex/pull/1661 is a blocker for 2.{9,10,11}. We know Pex 2.1.70 is broken.
👍 1
h
(I'm 20 minutes into reviewing, almost done.)
c
I have a few PRs, but not time sensitive, so can slip.. :)
h
I'm probably overthinking this...I'd like to try to have Pex lockfile generation be the default for Pants 2.11 because of its several improvements. But seems premature to make the default change without at least one release where people dogfooded. So I'm thinking 1. I do a 2.11 dev release today 2. we ask early adopters like Toolchain and @bitter-ability-32190 to try it in their repo 3. if that goes well, change the default before rc0, then do rc0 in a few days Gives more time for the parametrization stuff @witty-crayon-22786 is doing too
b
I can try the generation and see if consumption "works". But it'd (generation) be a one-time thing since I need to keep my versions more or less.
h
Hm I guess another approach is have us use PANTS_SHA to do the dogfooding. Altho we have gone 12 days since the last release, and there's still parametrization stuff so I think it makes sense to do dev4 today anyways
w
the chances of dev4 being sufficiently polished are slim enough that i don’t think that relying on it for dogfooding makes sense: should use SHAs. the “it’s been a long time since the last dev release” aspect is probably more important
but at the same time: i think that we’re close enough to an rc to just wait.
h
the chances of dev4 being sufficiently polished are slim enough
I actually think it's plausible Pex lockfiles will work pretty well. Note that default tool lockfiles are now working(!) https://github.com/pantsbuild/pants/pull/14811, and they were generated on my M1 w/ Py39 but running on Linux w/ Py37 in CI. John and I already have done ~6 patches to improve things
w
then it sounds like waiting for the rc might be fine too… *shrug
although we have some patches that we know will be bound for 2.11.0 (i have three that i can think of), i think that we could probably cut it at the end of the day, or first thing tomorrow probably.
h
rc0? this is where the over-thinking comes into play: is it fine for me to change the default lockfile generator after rc0? If so, sg Or maybe jump the gun and make the change today
I'm trying to minimize cherry-picking API changes - totally fine to cherry-pick new features that stabilize prior stuff, e.g dep inference for Protobuf
w
yea, and that’s reasonable. but it might suggest eagerly changing the default and then reverting it if the rcs show issues.
👍 1
rather than necessarily holding off the rcs until we have 100% confidence.
👍 1
80% confidence is probably good enough.
h
Stu and I have now landed the API changes we wanted for 2.11.0rc0. I'll plan to do the release tomorrow (Thursday). Any other API changes or new features folks want to land first?
f
did we verify that the various JVM codegen now work?
h
I think so far only verified Protobuf, and we still need dep inference for those. Stu and I are thinking that it's fine to cherry-pick in improvements to those though, like we did with Thrift in 2.10. This will probably be a longer rc period because 
parametrize()
and Pex lockfiles are both big new features
👍 1
w
now that the branch is cut, see https://pantsbuild.slack.com/archives/C0D7TNJHL/p1647888475976099 for tracking blockers for stabilizing
h
Regarding codegen for 2.11, what is our objective again? Fully done, minus docs: • Go Protobuf Needs JVM->protocol inference • Thrift (Java Scrooge, Scala Scrooge, Java Apache) • Avro Java Needs JVM->protocol inference + runtime dep • Protobuf (Java, Scala) Not sure: we dont have in init/ • WSDL
In my opinion, to publicize anything in our release docs blog, we need all of • docs • example-codegen • dep inf for target language -> protocol • runtime deps detection
cc @ancient-vegetable-10556 @fast-nail-55400 @witty-crayon-22786
Is my assessment accurate? If so, how much engineering time if any do we want to put into 2.11 stabilization? We could start with go Protobuf. Do those other languages in 2.12
a
Protobuf definitely needs the Java stuff to be tidied up. I landed the runtime deps detection today though
h
w
given the amount of other stuff that needs stabilizing in 2.11.x, i don’t think that we should spend a whole lot more time on codegen. we can move to further improve these backends based on noise about people needing them.
👍 1
one exception might be WSDL: @witty-family-13337: do you need WSDL support in 2.11.x? do you need any other types of codegen?
👍 1
w
not really as we are still using SBT for that bit. In fact we're evaluating moving to Apache CXF, so not a big deal not having WSDL codegen in 2.11
👍 1
Open in Slack
PreviousNext
Powered by