I am trying to pass an envirnmental variable from the host system to the docker build process through pants (on bitbucket pipelines). Specifically the DOCKER_HOST variable, by adding this to pants.toml

[docker]
env_vars=["DOCKER_HOST"]

But when I run

pants -ldebug I get package ::

, I see in the logs an entry which seems to show the environmental variables passed to the sandbox, but DOCKER_HOST is not there (the full log entry is in the thread). Do I need to do anything else to pass it through?

I think you should inspect the sandbox

__run.sh

script, I think that is the definitive way to determine what is passed on cdm-line to the process

Well, this is the __run.sh file; and there are is no mention of `$DOCKER_HOST`_

#!/bin/bash  7 # This command line should execute the same process as pants did internally.
export __UPSTREAM_IMAGE_IDS=
cd /tmp/pants-sandbox-UjpbFi
/usr/bin/docker build $'--pull=False' --tag $'<http://1234567891011.dkr.ecr.eu-west-1.amazonaws.com/my_services:api_lambda-7c3873c1d85e651687f345c875b80a9f951277b27f5ed8e2025e880ff0c0d9bb|1234567891011.dkr.ecr.eu-west-1.amazonaws.com/my_services:api_lambda-7c3873c1d85e651687f345c875b80a9f951277b27f5ed8e2025e880ff0c0d9bb>' --file projects/my_services/api_lambda/Dockerfile.my_services_ 11 api_lambda .

Is this the only way pants configures the build environment; or are there other mechanisms for passing in environmental variables?

how do you set

__UPSTREAM_IMAGE_IDS__

? it may help to look at the output of

pants docker --help-advanced

as it will show the value used and where it came from, along with overridden values if any

I don't; it must be generated somewhere inside pants 😕

right. ok, just to make sure, have you checked that DOCKER_HOST has a defined value..

If I add this to pants.toml, it still does not show up in the __run.sh. Is there any documentation on how the chroot that runs __run.sh is generated?

no that is implementation detail level stuff.

Well, basically the same as https://www.pantsbuild.org/docs/reference-docker

so, is it the same or not? what you look for is this

--docker-env-vars="[<shell_str>, <shell_str>, ...]"
  PANTS_DOCKER_ENV_VARS
  env_vars
      default: []
      current value: [
          "HOME",
          "DEMO=foo-bar"
      ] (from pants.toml)
      Environment variables to set for `docker` invocations.

DOCKER_HOST has a defined value on the build machine; and current value for env_vars is ["DOCKER_HOST", "TEST_VAR=foo"]; as it should.

not sure then what is going on, as I don’t repro this error… running

pants -ldebug package testprojects/src/python/docker/Dockerfile.example

I get:

12:42:26.00 [DEBUG] spawned local process as Some(55270) for Process { argv: ["/opt/homebrew/bin/docker", "build", "--pull=False", "--tag", "test-example:1.2.5", "--file", "testprojects/src/python/docker/Dockerfile.example", "."], env: {"DEMO": "foo-bar", "HOME": "/Users/andreas.stenius", "__UPSTREAM_IMAGE_IDS": ""}, working_directory: None, input_digests: InputDigests { complete: DirectoryDigest { digest: Digest { hash: Fingerprint<b0cdb9bb7020958c58135b9711af41abeb0907f9d62c4bb247ed063e53bc52bc>, size_bytes: 194 }, tree: "Some(..)" }, nailgun: DirectoryDigest { digest: Digest { hash: Fingerprint<e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855>, size_bytes: 0 }, tree: "Some(..)" }, inputs: DirectoryDigest { digest: Digest { hash: Fingerprint<b0cdb9bb7020958c58135b9711af41abeb0907f9d62c4bb247ed063e53bc52bc>, size_bytes: 194 }, tree: "Some(..)" }, immutable_inputs: {}, use_nailgun: {} }, output_files: {}, output_directories: {}, timeout: None, execution_slot_variable: None, concurrency_available: 0, description: "Building docker image test-example:1.2.5", level: Info, append_only_caches: {}, jdk_home: None, platform: Macos_arm64, cache_scope: PerSession, execution_strategy: Local, remote_cache_speculation_delay: 0ns }

Nice. This at least seems to confirm that the env variable should show up in the debug log; which it does not for me. What version of pants are you using? I am on 2.16.0_dev7, so not the stable version.

that was on

main

which is almost the same as

2.16.0a0

Back to basics:

[GLOBAL]
pants_version = "2.16.0a0"
build_file_prelude_globs = ["pants-plugins/macros.py"]

backend_packages = [
  "pants.backend.build_files.fmt.black",
  "pants.backend.python",
  "pants.backend.docker",
  "pants.backend.python.lint.black",
]

[python]
# This will become the default in Pants 2.15.
tailor_pex_binary_targets = false
interpreter_constraints = ["CPython==3.10.*"]

[docker]
env_vars = ["HOME"] 

[docker.registries.ecr_test]
address = "<http://1234567891011.dkr.ecr.eu-west-1.amazonaws.com|1234567891011.dkr.ecr.eu-west-1.amazonaws.com>"
default = true

[environments-preview.names]
linux = "//3rdparty/python:local_linux"
lambda_docker = "//3rdparty/python:lambda_docker_env"

[source]
root_patterns = [
  '3rdparty',
  'lib/lib1',]

yep

I am starting to feel really dumb here. But it seems clear that for some reason pants is not letting the environmental variables through with my setup (neither locally nor on bitbucket pipelines). One experiment I performed was to change the environmental variable DOCKER_HOST locally to "foo"; in which case docker build will fail. But if I put the same setting in pants.toml, pants still happily builds my docker images. I think it is time for my to try to repeat this on a smaller minimal repo.

the environments feature is rather new, and experimental, so may have some rough edges/bugs.

hm, yea. none of the messages that have been pasted in here are for docker runs though: all local. seems like there is a mix of things going on.