https://pantsbuild.org/ logo
#general
Title
# general
p

plain-truck-80265

08/26/2022, 9:42 AM
Hi all, I am curious if it is possible to interpolete env variables inside
BUILD
files? Seems it is not working for me, so I am just want to be sure 🙂 .
f

flat-zoo-31952

08/26/2022, 11:25 AM
This is not possible, and that is by design.
BUILD
files need to have the same content on any machine, and if you could interpolate environment variables, that would make the content of the
BUILD
file depend on the environment. What are you trying to accomplish where you would need this?
p

plain-truck-80265

08/26/2022, 12:13 PM
Ok it makes sense. The problem I am trying to solve is publishing an image into AWS ECR. Unfortunately the URL of ECR repository looks like this:
Copy code
<account_id>.dkr.ecr.<region>.<http://amazonaws.com|amazonaws.com>
In CI I would like to publish image to different accounts (representing PROD, DEV...). So I was thinking about using something like this in
BUILD
file:
Copy code
docker_image(
    name="docker_image",
    repository="%(env.AWS_ACCOUNT)s.dkr.ecr.%(env.AWS_DEFAULT_REGION)<http://s.amazonaws.com/xxx|s.amazonaws.com/xxx>",
)
I know I can use
[docker.registries]
but I had some difficulties with it. Here are some examples I tried:
Copy code
[docker.registries.ecr]
address = "%(env.AWS_ACCOUNT)s.dkr.ecr.%(env.AWS_DEFAULT_REGION)<http://s.amazonaws.com|s.amazonaws.com>"
repository = "{name}"
default = true
Seems that env variables are not supported inside address option.
Copy code
[docker.registries.ecr]
repository = "%(env.AWS_ACCOUNT)s.dkr.ecr.%(env.AWS_DEFAULT_REGION)<http://s.amazonaws.com/{name}|s.amazonaws.com/{name}>"
default = true
address option is required.
Copy code
[docker.registries.ecr-prod]
address = "<http://1111111111.dkr.ecr.eu-west-1.amazonaws.com|1111111111.dkr.ecr.eu-west-1.amazonaws.com>"
repository = "{name}"

[docker.registries.ecr-dev]
address = "<http://2222222222.dkr.ecr.eu-west-1.amazonaws.com|2222222222.dkr.ecr.eu-west-1.amazonaws.com>"
repository = "{name}"
This works but I do not know how to publish all changed images only into one repository (I mean when CI is running on
test
branch I want to publish all images to
ecr-dev
and when CI is using
main
branch then
ecr-prod
).
f

flat-zoo-31952

08/26/2022, 12:24 PM
I think I had this problem long ago, but I was using a hand-rolled docker-plugin that took the registry as a command-line argument. It looks like the plugin here has another way of dealing with this but I don't know if that method of interpolation will work for repositories. If not, we may need to modify the plugin, and perhaps reach out to @curved-television-6568
hmmm, you might be able to use the `registries` command-line option here
p

plain-truck-80265

08/26/2022, 12:29 PM
Thanks. Think I found the easy solution. I will use default repository for local development:
Copy code
[docker]
default_repository = "<http://111111111.dkr.ecr.west-1.amazonaws.com/{name}|111111111.dkr.ecr.west-1.amazonaws.com/{name}>"
tools = ["docker-credential-ecr-login", "getent"]
env_vars = ["AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY", "AWS_SESSION_TOKEN"]
And in CI I will override it accourding to environment:
Copy code
./pants --docker-default-repository="<http://2222222222.dkr.ecr.eu-west-1.amazonaws.com/{name}|2222222222.dkr.ecr.eu-west-1.amazonaws.com/{name}>" package ::
Hahaha you were faster :D
e

enough-analyst-54434

08/26/2022, 12:36 PM
FWIW, I'm pretty sure the interpolation that failed there for you was dict value interpolation which was very recently fixed on main (2.14.x): https://github.com/pantsbuild/pants/pull/16481
If you want to give that a spin instead of the workaround you can try the 2.14.0a0 release: https://pypi.org/project/pantsbuild.pants/2.14.0a0/
Yeah, on Pants main just now:
Copy code
$ git diff
diff --git a/pants.toml b/pants.toml
index 9fc7a64ed..2fd56afbe 100644
--- a/pants.toml
+++ b/pants.toml
@@ -275,3 +275,7 @@ master = "src/python/pants/notes/master.rst"
 "2.12.x" = "src/python/pants/notes/2.12.x.md"
 "2.13.x" = "src/python/pants/notes/2.13.x.md"
 "2.14.x" = "src/python/pants/notes/2.14.x.md"
+
+[docker.registries.ecr]
+repository = "%(env.AWS_ACCOUNT)s.dkr.ecr.%(env.AWS_DEFAULT_REGION)<http://s.amazonaws.com/{name}|s.amazonaws.com/{name}>"
+default = true
$ AWS_ACCOUNT=foo AWS_DEFAULT_REGION=bar ./pants help docker | grep -A10 docker-registries
  --docker-registries="{'key1': val1, 'key2': val2, ...}"
  PANTS_DOCKER_REGISTRIES
  registries
      default: {}
      current value: {
          "ecr": {
              "default": true,
              "repository": "<http://foo.dkr.ecr.bar.amazonaws.com/{name}|foo.dkr.ecr.bar.amazonaws.com/{name}>"
          }
      } (from pants.toml)
      Configure Docker registries. The schema for a registry entry is as follows:
💯 2
c

curved-television-6568

08/26/2022, 12:53 PM
@plain-truck-80265 glad you found the ability to set the default registry from command line, was going to be my suggestion. Also as John points out, the env interpolation for your configuration you tried with initially will work from Pants 2.14.0 as it was just recently fixed.
p

plain-truck-80265

08/26/2022, 1:33 PM
Thank you @enough-analyst-54434 and @curved-television-6568 I can confirm it works.
👍 1
4 Views