How does the changelog work with the release branch cutover? Does the author of every PR with a change to

docs/notes/2.23.x.md

need to remember to switch that to 2.24 once that exists?

Yep! It's not the best and yet another thing that'd be better with towncrier-style independent files. https://www.pantsbuild.org/stable/docs/contributions/releases/release-process#identify-pull-requests-needing-changes