Is there a way to embed or cache this in our CI Ag...
# general
g
Is there a way to embed or cache this in our CI Agent AMIs? We find that GitHub is rather unreliable. We see issues conservatively once a quarter and it disrupts CI. > Failed to fetch https://github.com/pantsbuild/pants/releases/download/release_2.28.0/pants.2.28.0-cp311-linux_x86_64.pex: [28] Timeout was reached (Failed to connect to github.com port 443 after 129444 ms: Couldn't connect to server) > Wasn't able to fetch the Pants PEX at https://github.com/pantsbuild/pants/releases/download/release_2.28.0/pants.2.28.0-cp311-linux_x86_64.pex. > Check to see if the URL is reachable. > > Exception: > Command '['/var/lib/buildkite-agent/.cache/nce/22652cf60b12e7a187ea0cf21d7bc9d8234bbb630fa94ca6f9c28655bd6a81fb/ptex-linux-x86_64', 'https://github.com/pantsbuild/pants/releases/download/release_2.28.0/pants.2.28.0-cp311-linux_x86_64.pex']' returned non-zero exit status 1. > Error: Failed to establish atomic directory /var/lib/buildkite-agent/.cache/nce/5c54a0c25255df2afb0525da856d49eb81b48cff0e32a9c868f77a296af24319/locks/install-c0ce0c6f977d6f5d0786222a4e158390b5b503257ea667345b3e995f3e2d738f. Population of work directory failed: Boot binding command failed: exit status: 1
f
Yes. In your AMI build script, download and install the Pants launcher binary. Then create a trivial Pants project with just a
pants.toml
set to the version of Pants you want (and the backend packages) to cache in the AMI. Then run
pants --version
and as a side effect that Pants version will be downloaded and its venv built in the AMI image.
👍 1
This is a
Dockerfile
which does something similar for the same purpose of caching Pants in an image.
👀 1
And here is the the
pants.toml
used with the
Dockerfile
. The version is a bit old, so obviously update.
The trade off with this method is that you will have to rebuild the AMI when you want to upgrade Pants.
But you can rebuild the AMI with both old and new versions of Pants cached in the image so your CI jobs can switch Pants version separately from switching AMI versions.
g
We rebuild our CI agent AMI daily anyway because we have a large monorepo that we embed so this works great for us.
👍 1
f
also if I rewrote this script now, I would avoid the
get-pants.sh
script and just download the scie-pants launcher directly from https://github.com/pantsbuild/scie-pants and check its expected sha256 hash.
👀 1
👍 1
g
@fast-nail-55400 this worked great. Thanks!
👍 1