If I wanted to set up a build AMI that had pants p...
# generala
average-breakfast-91545
06/12/2023, 7:33 PMIf I wanted to set up a build AMI that had pants pre-installed, is there a better way than running pants in a dir with the right version in the pants.toml? Is there even any point? I'm unclear on what's cached as part of the Github Actions pants-init action.
average-breakfast-91545
06/12/2023, 7:43 PMContext in case that made no sense: I'm now adding CI to my pants build. Tests are slooow on GH because there's only a couple of vCPUs.
I've stolen your sidebar setup, so on a hot cache, tests take about 2 minutes, of which a minute is restoring pants cache, fetching docker images etc.
I'm planning to self host some ephemeral runners with more CPUs to crunch through cold cache test runs, hosted in AWS so I don't pay data transfer costs for S3.
Anything I can reasonably preinstall is a bonus cos it'll reduce startup time
h
happy-kitchen-89482
06/12/2023, 8:46 PMMakes sense. I'd be interested in hearing back about the experience.
happy-kitchen-89482
06/12/2023, 8:47 PMAnd yes, the best way to bootstrap Pants for embedding into an AMI is to run
pants versiona
average-breakfast-91545
06/12/2023, 8:47 PMRoger, so copy a dummy pants.toml to /tmp/whatever, and pants version in there.
average-breakfast-91545
06/12/2023, 8:48 PMI'll let you know how it goes 🙂
h
happy-kitchen-89482
06/12/2023, 9:00 PMJust make sure it's the right version, obvi
happy-kitchen-89482
06/12/2023, 9:01 PMYeah, you can clean up /tmp/whatever before freezing the AMI. The relevant pants state goes in ~/.cache/pants
e
enough-analyst-54434
06/13/2023, 12:09 AMAnd ~/.cache/nce
👍 1
a
average-breakfast-91545
06/14/2023, 7:35 AMOookay, so shifting the build over to C5.4xlarge machines in my own AWS environment gets us from 17 minute test runs to 5 minutes, which is nice. I've still got slow start up times on some tests, which I want to delve into, and I'm still paying for pantsd daemon startup which I'm going to remove. With cache, the build runs in ~ 30 secs.
It's sort of irritating waiting for the instances to start, but at least it brings my pants build performance inline with the unpants'd build.
Perhaps a stupid question, but if the lockfiles change, does pants use pip cache to reduce the number of libraries it needs to fetch? I'm considering building a nightly image to cut down the time for a cold build when lockfiles change.
2 Views