I'm setting up CI/CD for our new pants deployment....
# general
w
I'm setting up CI/CD for our new pants deployment. Initially we have a GH workflow per service target (eg, docker image pushed to ECR), another for a published python package, and one more for lint/test. That's not an ideal shape, but one we evolved into while getting pants set up initially. I've read the docs on running in CI and i've just converted one build to use
--changed-since
, which is a lovely feature, and i'm looking at how to structure the rest. Are there any common patterns with pants, or ones to avoid?
I'm leaning towards a single
pants.yml
workflow, that does install/setup, then runs test/lint and package, against targets that have changes. Then on main runs publish for changed targets. That seems straight forward with pants. When deploying we have post-publish step we need to apply, so would need to know which targets were published (and which were docker), then run those against another GH action to trigger deploys • I was thinking about capturing the output of publish to work out the deploy targets, i can do this with bash but it feels hacky. • I'd also need to limit that to docker targets, to handle publishing packages separately, but using
--changed-since
prevents using other target filtering rules/paths An alternative, would be using an initial job to spin up the env, run tests, and then store that env as an artifact, then a matrix based on the changed targets, which would need to pull env back for publish/deploy. Our pants cache in GH actions is ~4GB, so that feels pretty heft to ship around between jobs...
c
I don't think there is a simple right answer here, for a flavor ours looks (simplified) something like. The general trend has been towards consolidating targets (ex a "build all the docker" job, instead of many tiny jobs)
w
That mirrors my thinking tbh.
How are you distinguishing building docker vs packages?
c
--filter-target-type=python_distribution
&
--filter-target-type=docker_image
w
I thought those filters didn’t work in combination with —changed—since: if you’re using that?
c
pants --filter-target-type=docker_image list --changed-since=origin/main --changed-dependents=transitive
is what I believe it expands to.