https://pantsbuild.org/ logo
#development
Title
# development
w

witty-crayon-22786

05/17/2023, 5:34 PM
we’ve previously discussed moving from an “every SHA is consumable with PANTS_SHA” to “only commits which have been tagged or workflowed would be consumable with PANTS_SHA”. i think that doing that would align well with some automation that i’m looking at in the release process. how would we feel about that? it would mean that a maintainer would need to run a workflow to build wheels for a SHA.
h

hundreds-father-404

05/17/2023, 5:35 PM
Sounds sensible to me. I think we want some way to communicate this with the community, that they know how to ask us for a build
i think that doing that would align well with some automation that i’m looking at in the release process.
Can you please share more?
w

witty-crayon-22786

05/17/2023, 5:38 PM
Can you please share more?
i’d like to move to auto-publishing to pypi and github actions on tag / workflow. while we could currently do that as a conditional step at the end of a full test run, i’m thinking that we could move all publishing (to S3, pypi, github actions, etc) out into a separate job.
❤️ 2
b

bitter-ability-32190

05/17/2023, 5:42 PM
If people want to run from a specific SHA couldn't they clone the repo and use
PANTS_SOURCE=...
?
w

witty-crayon-22786

05/17/2023, 5:51 PM
Yea, but that doesn't allow for testing things in CI, which is useful for validating fixes.
(without a bunch of extra infra)
b

bitter-ability-32190

05/17/2023, 5:57 PM
I suspect you don't need it to be that bleeding edge and can wait for the next published release (which might happen more often with automation) or you do and the added infra cost would then be worth it. My 2c
w

witty-crayon-22786

05/17/2023, 6:12 PM
ok, so: the proposal is to: • move “publish to S3 if on branch” steps out of the
test
workflow, and into a separate
release
workflow which runs when tagged or manually • adjust the docs to indicate that you can ask a maintainer to kick off a SHA build. @happy-kitchen-89482, @curved-television-6568, @average-vr-56795, @wide-energy-11069: fyi. this doesn’t impact the release process yet, but it will.
👍 4
a

average-vr-56795

05/17/2023, 6:24 PM
SGTM, my only fear is we'll find out about breaks to the release process later, but I think those are already pretty rare so I'm not worried :)
👍 1
b

bitter-ability-32190

05/17/2023, 6:55 PM
Does this mean we can stop building wheel on PR?
w

witty-crayon-22786

05/17/2023, 6:55 PM
no, i think that we still want to do that for testing purposes, for precisely the reason DWH mentioned.
👀 1
a

average-vr-56795

05/17/2023, 7:05 PM
Oh we're literally just talking about to copy to S3 step? Very strong yes then
w

witty-crayon-22786

05/17/2023, 7:25 PM
Yea. PR wheel building is already pretty heavily filtered to only run when you're changing something that might impact wheel builds.
b

bitter-ability-32190

05/17/2023, 7:49 PM
That's almost everything though right? Most of the python code gets baked into a wheel?
w

witty-crayon-22786

05/17/2023, 7:50 PM
we don’t run the wheel shards for pure python changes, no
b

bitter-ability-32190

05/17/2023, 7:51 PM
Maybe Im just not paying attention then 😂
w

witty-crayon-22786

05/17/2023, 7:51 PM
the “classify changes” shard is pretty isolated and nice now, thanks to some work Benjy did in the fall
b

bitter-ability-32190

05/17/2023, 7:53 PM
I guess I should say, I want us to use less bespoke stuff and more off-the-shelf. That also means easier automation which I also want. So a lot of my idea/focus of late has started ooching in that direction
w

witty-crayon-22786

05/17/2023, 8:13 PM
what does that mean in this context
b

bitter-ability-32190

05/17/2023, 8:14 PM
NOTHING! I'm a rambler! Jk I'd love to get off S3 and use GitHub's releases for storage. I'd love to support building less things in general (as scie-pants should enable)
Back on topic, my GitHub action addition is building wheels? https://github.com/pantsbuild/pants/pull/19021
w

witty-crayon-22786

05/17/2023, 8:18 PM
yes: changes to github actions definitions could have impacted the definition of the wheel building shard.
i don’t know why the affected line is so large there, but i think that’s a separate issue to look at
b

bitter-ability-32190

05/17/2023, 8:24 PM
Oh wow I just cracked open
test.yaml
oof
w

witty-crayon-22786

05/17/2023, 8:24 PM
it’s generated: no point looking at the actual file
b

bitter-ability-32190

05/17/2023, 8:25 PM
Well I've gotten GitHub action goggles. I don't have Pants-codegen-making-github-action goggles 😂
c

curved-television-6568

05/17/2023, 9:05 PM
having proper GH action BUILD file targets has been a simmering idea in the back of my head for a while…
👀 1
w

witty-crayon-22786

05/17/2023, 9:34 PM
this doesn’t impact the release process yet, but it will.
mmm: i lied about this. it will necessarily affect the release process, since the tag push is what actually triggers the build.
b

broad-processor-92400

05/17/2023, 9:57 PM
(Back off topic, again: https://github.com/pantsbuild/pants/discussions/18235 is a proposal for a moderately generic way to help with 'codegen that needs to be checked in', like GHA config.)
w

witty-crayon-22786

05/17/2023, 10:00 PM
mm. i honestly didn’t grok that one first time round. but yea, that’s interesting.
b

bitter-ability-32190

05/17/2023, 10:13 PM
Yeah we should do more dogfooding in CI
It's yummy