https://pantsbuild.org/ logo
Join Slack
Powered by
The macOS11 arm64 self-hosted runner is failing to...
# development
b
The macOS11 arm64 self-hosted runner is failing to build https://github.com/pantsbuild/pants/pull/21579 with error:
Copy code
/Users/gha/actions-runner/_work/_temp/de80d76e-4003-4167-a229-8268aacd9ef2.sh: line 1: rustup: command not found
I've rebooted the machine but that made no difference. Has there been a change to it recently? (cc @happy-kitchen-89482)
h
No changes that I’m aware of. Yeesh.
That is a mystery to me
b
Are you in a position to look? Or grant me ssh access to look?
h
I have no idea why this happened, but I reinstalled rustup and that job seems to be progressing now
b
sweet; got to green ✅ merged and tagging now
Huh, it's happened again:
h
Huh
That is bizarre
b
Ironically, this is blocking the change that'll allow us to get rid of the self-hosted instance (eventually) 😅
h
It’s strange
Oh
I know what this is…
That patch is intended to work around this issue: https://github.com/Swatinem/rust-cache?tab=readme-ov-file#known-issues
😢 1
Which is badly broken behavior on long-running runners
But maybe it’s not working around it any more for some reaosn
That behavior is exactly what we’re seeing - a cleaned out 
~/.cargo/bin
b
ah, so we get one successful run for each reinstall, that'd correlate with the behaviour I've observed so far: • the releases/jobs on the older branches worked fine (i.e. from before 21511) • not so fine on main nice find 👍
h
I dunno though, here I don’t see the log line 
... Cleaning cargo/bin ...
that I would expect
I guess that happened in a subsequent run
let’s try and find it?
b
ah, hm, but that's using the old version.
h
Yeah, I can’t find the that log line in a recent release PR (it has to be a release because this is all happening during wheel building)
b
from PR https://github.com/pantsbuild/pants/pull/21528. That seems to have the log line that you expected to see to cause this failure
ah, yeah, and then the next (chronologically) build fails: https://github.com/pantsbuild/pants/actions/runs/11582029636/job/32244113512
h
Good find!
b
In https://github.com/pantsbuild/pants/actions, these two "update Pants' own Python to 3.11" builds (recording the context if that matters)
h
Aha, and yet 
cache-bin
was set to 
false
for those
(If you look at the args 
Cache Cargo
was run with
So that is odd
And it did run 
benjyw/rust-cache@61b5b2e17a28350779e9a535e353da2f8b00e832
Which is what I expect
Oh
OH!
Dammit
one sec
I know what this was
I will send a fix, which will have to be merged into all outstanding PR branches
The issue was: when I rebased my patch onto a later version of the upstream action, I screwed up handling the compiled js, so it had basically reverted to the pre-patch state.
b
oh, okay
h
Now I have regenerated it from scratch (which is what I should have done all along)
Basically I was looking at the source .ts, and didn’t realize that the compiled .js didn’t align
So I couldn’t figure out why the action wasn’t doing what it looked like it was doing
And this, children, is why you don’t check in compiled code
b
👍 and ✅ I don't think this particularly needs cherrypicking, unless we need the new version of the action on the old branches (since 21511 wasn't cherrypicked, AFAICT)
h
(although realistically, since github is how actions are deployed, I suppose there is no alternative in this case)
Ah, you are right, it was not cherrypicke
OK, merged, should be good now
👍 1
b
Oops, I think we'll need to reinstall Rust again on the arm64 runner (if you have a moment, @happy-kitchen-89482, thanks). The 2.24.0.dev2 release from a while ago failed its build, so I retried, but it was using the old version of the action that cleared the wrong directory. (https://github.com/pantsbuild/pants/actions/runs/11414440127/job/32407339772#step:22:68)
h
Done
🙏 1
2 Views
Open in Slack
PreviousNext
Powered by