Hi All. I'm trying to think of the best way to imp...
# generalk
kind-hydrogen-975
02/26/2022, 2:57 PMHi All. I'm trying to think of the best way to implmenet a workflow that builds and deploys a new image, but only if the test passed, but unmemoized. For me that means that it's new code that passed real tests, and that's the only time I want to deploy it (Failed tests obviously won't deploy, and also memoized successes since those already exist in the deployments). What is the best way to go about doing this using Pants?
I'm assuming there's a cleaner way than parsing pant's and pytest's stdout?
Thanks!
f
fast-nail-55400
02/26/2022, 3:28 PMYou can have Pants write a
junit.xml—test-xml-dirh
happy-kitchen-89482
02/26/2022, 4:47 PMI have some ideas here, but I have some questions first, to be sure I understand the setup:
1) Are you already using Pants, or are you asking more generally?
2) What is your unit of deployment? Sounds like it's a Docker image? Does that contain Python code? How many different ones do you have in your repo?
3) Is there a consistent mapping from tests to units of deployment?
happy-kitchen-89482
02/26/2022, 4:48 PMI ask because typically you might want to separate the two different concerns of A) figuring out what might need to be deployed based on changes and B) testing those things
happy-kitchen-89482
02/26/2022, 4:48 PMYou can achieve A) using Pants's project introspection goals, for example
happy-kitchen-89482
02/26/2022, 4:51 PMSo let's say you want to compare the current git state to some deploy branch state, you can do something like:
happy-kitchen-89482
02/26/2022, 4:51 PM Copy code
./pants --changed-since=deploy_branch list | xargs ./pants filter --target-type=docker_imagehappy-kitchen-89482
02/26/2022, 4:51 PMThat will list all the docker images that are affected by changes between those two git states
happy-kitchen-89482
02/26/2022, 4:56 PMSelectively figuring out which images are good to deploy based on test outcomes is trickier to do in a single Pants run. If any test fails then the Pants run fails, so you'd never get to the packaging goal.
happy-kitchen-89482
02/26/2022, 4:57 PMBut this can probably be composed out of multiple Pants runs
happy-kitchen-89482
02/26/2022, 4:58 PMOr you can write a plugin that consumes test results that are allowed to fail, and analyzes them
happy-kitchen-89482
02/26/2022, 4:58 PMThere are a few ways to go here
k
kind-hydrogen-975
02/27/2022, 6:55 AM@happy-kitchen-89482 thanks for the reply benjy 🙂
so, to answer your questions:
1. We are using Pants, or at least starting to, and trying to figure out the best way to go about it
2. It is indeed a docker image, containing an unpacked zip archive created from the python_awslambda Pants target
3. Well, our monorepo has a bunch of python projects, each containing a Dockerfile. So every directory is a deployable unit.
kind-hydrogen-975
02/27/2022, 6:56 AMOur thinking was, we want to run one big pants run on all the projects, rely on pants caching to not test unchanged projects, and if something both changed and passed the tests - we want to redeploy.
kind-hydrogen-975
02/27/2022, 6:58 AMAlso, we're looking into argo workflows as a CI engine. So I'm trying to think how to fuse these technologies together in a sensible way. That is, if I have an argo step that runs tests and is successful, I want to somehow pass to the next argo step the list of containers that need to be rebuilt and deployed.
e
enough-analyst-54434
02/27/2022, 4:00 PMJust to flesh this out a bit more - I have no clue how to solve - @kind-hydrogen-975 do you have any non-deployable projects that others depend on for shared common code? If so, do you want a green tested change in that style of project to cause a deploy in all projects that depend on its production code?
3 Views