https://pantsbuild.org/ logo
Join Slack
Powered by
hey there! how would I use `pants package` in my g...
# general
h
hey there! how would I use 
pants package
in my github workflows? Ideally I would just need to rebuild only the docker images where changes exist?
b
To package just what has changed is done using 
--changed-since=origin/main --changed-dependents=transitive
. So for example 
pants --changed-since=origin/main --changed-dependents=transitive package
. One detail to sort out is how you plan on versioning your artifacts. Many ways to do this and it depends on what your specific constraints are. You can use 
.pants.bootstrap
to generate versions and pass that in as a build arg and use it for the image tag. Example of what it would look like in 
pants.toml
Copy code
[docker]
build_args = [
    "VERSION_DOCKER",
]
And then in your 
docker_image
Copy code
tags = [
  "{build_args.VERSION_DOCKER}"
]
You could explicitly tag each of your artifacts and maintain that in source control or use git taging with a naming scheme to tag specific artifacts
h
for versioning, I'm using https://github.com/marketplace/actions/git-semantic-version and then I have something like this:
Copy code
pants --docker-build-args="['VERSION=$IMAGE_TAG']" package ::
@brief-branch-21752 what would the publish command look like?
b
It would be the same as the package command. Keep in mind that if you do this then only changed artifacts will be published therefore downstream deployment logic would need to know if a given artifact is supposed to have the version defined. An alternative is to publish everything regardless if it changed and then you know all artifacts share a common version.
👍 1
h
ah good point, thanks for pointing this out.
would the publish just be 
pants --changed-since=origin/main --changed-dependents=transitive publish ::
b
You don't need the 
::
at the end since the changed arguments become your selector.
h
how do I add a tag that has my ecr registry?
b
I haven't used ECR with Pants but you should be able to find some good examples from others using search in slack at the top. Searching "ecr" came up wih a bunch of hits. Example issue that shows someone using the tools to include `docker-credential-ecr-login`https://github.com/pantsbuild/pants/issues/21098 In general searching slack is a great way to find help for specific issues.
Oh and yes you would likely want to add it as a registry
h
when I add it as a build arg it looks like it's taking it as the literal...
Copy code
...
registries=["{build_args.REGISTRY}"],
it results in this:
Copy code
ProcessExecutionFailure: Process 'Building docker image {build_args.REGISTRY}/echo:latest +1 additional tag.' failed with exit code 1.
b
Checkout this syntax: 
{build_args.VERSION_DOCKER}
oh hmm
h
I'm doing this to test: 
pants --docker-build-args="['VERSION=v0.0.1', 'REGISTRY=hello']" publish apps/echo:echo
b
This is an example of defining a registry in your 
pants.toml
file (this is for GCP Artifact Registry):
Copy code
[docker.registries.your-name]
address = "us-central1-docker.pkg.dev"
default = true
extra_image_tags = ["{build_args.VERSION_DOCKER}"]
h
the way I do it is injecting the environment variable during build time through the github workflows, I rely on the previous step to authenticate and output the registry
b
sorry I don't have specific experience with parameterizing the registry like that.
h
no worries then, I'll play around it must be a stupid mistake somewhere
do you know if there are valid docker build args?
b
According to docs you can define any value as a build-arg: https://www.pantsbuild.org/prerelease/docs/docker#build-arguments It might be the case that 
registries
doesn't allow getting values form build_args. An alternative is to get it from the environment which would have a syntax like (not tested) 
env.REGISTRY
.
h
hmm unfortunately env.REGISTRY also takes the literal: ERROR: invalid tag "{env.REGISTRY}/echo:latest": invalid reference format
ah I think I can set them via 
--docker-registries
I'm having some issues here now:
Copy code
Dockerfile.echo:3
--------------------
   1 |     FROM python:3.11-slim
   2 |     COPY --from=echo-srcs /bin/app /bin/app
   3 | >>> COPY --from=echo-deps /bin/app /bin/app
   4 |     EXPOSE 8080
   5 |     ENTRYPOINT ["/bin/app/pex", "--port", "8080", "--host", "0.0.0.0"]
--------------------
ERROR: failed to solve: echo-deps: failed to resolve source metadata for <http://docker.io/library/echo-deps:latest|docker.io/library/echo-deps:latest>: pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
here's the part of my 
BUILD
file that is relevant:
Copy code
docker_image(
    name = "echo-deps",
    skip_push=True,
    image_tags=["latest", "{build_args.VERSION}"],
    instructions = [
        f"FROM {BASE_IMAGE}",
        "COPY apps.echo/_binary-deps.pex /bin/build/_binary-deps.pex",
        "RUN PEX_TOOLS=1 /usr/local/bin/python3.11 /bin/build/_binary-deps.pex venv --scope=deps --compile /bin/app",
    ],
    dependencies=[":_binary-deps"],
)

docker_image(
    name = "echo-srcs",
    skip_push=True,
    image_tags=["latest", "{build_args.VERSION}"],
    instructions = [
        f"FROM {BASE_IMAGE}",
        "COPY apps.echo/_binary-srcs.pex /bin/build/_binary-srcs.pex",
        "RUN PEX_TOOLS=1 /usr/local/bin/python3.11 /bin/build/_binary-srcs.pex venv --scope=srcs --compile /bin/app",
    ],
    dependencies=[":_binary-srcs"],
)

# <https://www.pantsbuild.org/blog/2022/08/02/optimizing-python-docker-deploys-using-pants>
docker_image(
    name = "echo",
    image_tags=["latest", "{build_args.VERSION}"],
    dependencies=[":echo-srcs", ":echo-deps"],
    instructions = [
        f"FROM {BASE_IMAGE}",
        "COPY --from=echo-srcs /bin/app /bin/app",
        "COPY --from=echo-deps /bin/app /bin/app",
        "EXPOSE 8080",
        'ENTRYPOINT ["/bin/app/pex", "--port", "8080", "--host", "0.0.0.0"]',
    ]
)
how do I get it to copy from locally?
b
Do you have a default registry being applied? Maybe try explicitly providing a registry like in the linked docs that are just a placeholder.
h
I guess I can do a localhost registry then right?
I know what the issue is here now, it's that because I specified the default registry, it expects 
--from={default_registry_name}/echo-srcs
but I guess that's not ideal so the previous docker_images need a pattern to not add the default somehow
b
Based on the docs they are saying the registry can be anything since you aren't pushing the image. You are just creating a local reference to the image.
h
right, but the issue here is that I am setting 
--docker-default-repository
so that my final image pushes to the registry properly The rest are also built with that name (which I do not want)
2 Views
Open in Slack
PreviousNext
Powered by