Some thoughts on the release process: 1. After running Step 2 in the release process to tag the release, the GitHub release is not created in draft mode and then promoted. Instead, it is created as released and announced as soon as the Release GHA workflow completes. Is this intentional? 2. Step 3 seems to not work (and is sort of a moot point if the release is announced to the world). I get this error:

% ./pants run src/python/pants_release/release.py -- test-release
[=== 00:01 Installing and testing the latest released packages ===]
00:36:30.62 [INFO] waiting for pantsd to start...
00:36:32.48 [INFO] pantsd started
00:36:32.61 [INFO] Initializing scheduler...
00:36:36.38 [INFO] Scheduler initialized.
2.25.0a1
Failed to confirm pants version, expected '2.25.0a1', got ''

3. Instructions need to updated to no longer talk about PyPI publishing and also need to add a mention of the PR generated for the documentation site.

Re 2, I think that may be a symptom of https://github.com/pantsbuild/pants/issues/19953. Workaround is to run with

PANTS_NO_NATIVE_CLIENT=1 ./pants ...

Do you have some alternatives in mind?

I'd suggest we create release in draft mode so we can run the smoke tests (and the public repo testing). The announcement workflow should move to a job which triggers when the release is finally taken out of draft mode.

Probably better to publish the sha256 hashes elsewhere (e.g, on the docsite). If someone can compromise the wheels on github releases they can also compromise the sha256 files there, presumably.

Some context will help: I would like to publish the sha256's as part of the wheels.pantsbuild.org since the PyPi Simple API includes the ability to do so. Generating them on the releases means the generator script can download them easily without downloading the actual wheels.

I’m fine with that, I’m just pointing out that it is somewhat fake security, since if I could compromise the wheels, presumably I could also compromise the sha256es residing on the same server I compromised. No?

Well we would record the SHA256's in the wheels.pantsbuild.org repository and not re-download them.

I'd suggest we create release in draft mode so we can run the smoke tests (and the public repo testing). The announcement workflow should move to a job which triggers when the release is finally taken out of draft mode.

Ah, I see. Sounds plausible in theory 👍 although I think that'd be a bit awkward, because a draft release is "private" / requires authing with sufficient privileges to install? (I think?) Our convention of always doing an rc before a stable release (i.e. a stable release has the same code (other than version) as the most recent rc) gives a lot of the benefits. It's seems relatively unproblematic for non-stable releases to be broken, especially as it lets us have a more automated release flow with less required human interaction. 🤔 definitely worth thinking about potential improvements though!

Do you have any insights from Rust, especially about how they schedule pre-release

crater

runs?

For Rust, I was never that familiar with the mechanics of the release process, but I believe... • nightlies (most similar to our dev releases) are fully automated and are broken if they're broken ◦ I found just now there's some process for rolling them back: https://forge.rust-lang.org/infra/policies/broken-nightlies.html ◦ there's https://rust-lang.github.io/rustup-components-history/ that tracks what's available (apparently everything has been available recently, so I dno what the other possible states are 🤷 ) • crater runs are triggered manually on beta releases and then the summarised results are manually triaged https://forge.rust-lang.org/release/crater.html • https://forge.rust-lang.org/release/process.html discusses the release process itself, which seems broadly similar to ours (e.g. PRs with version bumps in some form)

oh cool, and the GH attestations are powered by Sigstore