https://pantsbuild.org/ logo
Join Slack
Powered by
Cross compiling goLang for Docker... *Is there a w...
# general
a
Cross compiling goLang for Docker... Is there a way to specify the foreign platform when generating the 
go_binary
? I've confirmed that if I build using
Copy code
env GOOS=linux GOARCH=arm go build .
then I create a 
file
target that references the binary result of the above command and copy that file into a 
Dockerfile
image and run using 
./pants run path/to/project:docker_image
The docker image executes the binary as expected. If I instead copy the output of 
go_binary
into the 
Dockerfile
image, I get the failure 
exec /app: exec format error
This indicates the go binary is built for the wrong platform. I've tried
Copy code
env GOOS=linux GOARCH=arm ./pants run path/to/project:docker
without success.
h
Pants isolates processes from the ambient environment by default. You can selectively punch holes through this isolation via this option: https://www.pantsbuild.org/docs/reference-subprocess-environment#env_vars
That said, I think that for Go we tinker with the GOOS and GOARCH ourselves, so I'm not sure even this will work
Hmm, actually we extract the GOOS and GOARCH from running 
go env -json
, so it might well work!
Let us know how it goes with that option
f
You should probably use the 
docker_environment
target type to run the Go compilation inside of a Docker container (and the container image will need a Go SDK installed)
just changing GOOS and GOARCH is likely insufficient unless the Go SDK you were using was built for cross-compilation
(the precompiled package archives for the standard library are usually only built for the host platform)
The Go backend will need support for compiling the standard library from scratch to support cross-compilation (which is also necessary for Go v1.20 support). See https://github.com/pantsbuild/pants/issues/17950
So "native" compilation inside a Docker container (via 
docker_environment
is likely the best method for now)
a
Got it. Thanks for the response. I tried to use the advanced option but it did not succeed.
Copy code
env GOOS=linux GOARCH=arm ./pants --subprocess-environment-env-vars="['GOOS','GOARCH']" --no-local-cache run apps/go/greeter_en:docker
I was hoping not to have to copy the git repo source code into the Dockerfile. Being able to compile a binary in the same way 
Pants
supports 
pex
would be fabulous as would the automatic inference that a 
docker_image
is a dependee of its transitive dependencies. If there is an intent to support this, I'd love to watch it and when I become skilled enough in Go and Pants I'd be happy to contribute. Presently, I'm still just mesmerized by your awesome toolset!
I'm looking into 
docker_enviromment
and it appears to be exactly what I need both for this concern and to support Apple arm64. Is there an example project that demonstrates this new target in action? The docs use terms that aren't familiar to me yet. I'll let you know if I can coax it out on my own!
@fast-nail-55400 It does not appear that 
docker_environment
is supported by 
go_binary
. Am I misunderstanding the docs? Here's what I've tried. • updated my pants version to "2.15.0rc1" • defined a 
docker_environment
in a root level BUILD file
Copy code
docker_environment(
  name="golang_alpine",
  platform="linux_x86_64",
  image="1.17.13-alpine3.16",
)
• referenced said env in 
go_binary
Copy code
go_binary(
    name="greeter_en",
    environment="golang_alpine"
)
• The binary output is referenced by a Dockerfile, left untouched
Copy code
./pants run apps/go/greeter_en:docker
14:55:52.54 [ERROR] 1 Exception encountered:

Engine traceback:
  in `run` goal
  in Resolve transitive targets
  in Resolve direct dependencies of target - apps/go/greeter_en:docker
  in Find all packageable targets in project
  in Find all targets in the project

InvalidFieldException: Unrecognized field `environment=golang_alpine` in target apps/go/greeter_en:greeter_en. Valid fields for the target type `go_binary`: ['_dependencies', 'description', 'main', 'output_path', 'restartable', 'tags'].
f
ah we never added environment support to the Go backend
a
IIUC... • 
docker_environment
is not supported by the Pants Go backend so I cannot tap into it to build go inside a Pants managed Docker container • GOOS and GOARCH are not passed through to the 
go build
sub process using 
--subprocess-environment-env-vars
• 
go_binary
compiles artifacts for the host platform only I'm struggling to configure my multi-module monorepo such that I can • Leverage Pants to test, lint, etc • Produce binary with 1st and 3rd party dependencies • Containerize the same using Docker Specifically the third bullet is presenting challenge. Using the target 
docker_image
, the sandbox does not have the full source tree so I am unclear as to how to include the needed source files, and in what expected directory structure, so I can use then build inside the container. How can I structure my multi-module monorepo, and it's go.mod definitions, so that I can both • Execute goals via Pants • Assemble source so that I can execute Go build commands inside a Dockerfile? Would it be advisable or possible to copy the entire repo as source into the container to then use pants inside the container?
Here is what I figured out. Please let me know if this is a reasonable strategy or if there is a better way to accomplish the above goals. I've attached my example project. The relevant directory structure is:
Copy code
├── apps
│   ├── go
│   │   ├── greeter_en
│   │   │   ├── BUILD
│   │   │   ├── Dockerfile
│   │   │   └── main.go
│   │   └── greeter_es
│   │       ├── BUILD
│   │       └── main.go
├── libraries
│   ├── go
│   │   ├── BUILD
│   │   ├── embed_example
│   │   │   ├── BUILD
│   │   │   ├── hello.txt
│   │   │   ├── lib.go
│   │   │   ├── lib_test.go
│   │   │   └── testdata
│   │   │       └── f.txt
│   │   ├── greeter
│   │   │   ├── BUILD
│   │   │   ├── greet.go
│   │   │   └── greet_test.go
│   │   └── uuid
│   │       ├── BUILD
│   │       ├── uuid_gen.go
│   │       └── uuid_gen_test.go
├── BUILD.pants
├── go.mod
├── pants
└── pants.toml
./libraries/go/BUILD contains:
Copy code
files(
    name = "source-files",
    sources = ["./**/*"]
)
This makes the source files available to the Dockerfile, via 
docker_image
, inside the sandbox. Likewise, ./apps/go/greeter_en/BUILD contains:
Copy code
files(
    name = "source-files",
    sources = ["./**/*.go"]
)

go_package(name="go-package")

go_binary(
    name="greeter_en",
)

docker_image(
    name="docker",
    dependencies=[
        "libraries/go:source-files", 
        "//:go-mod-file",
        ":source-files"
    ]
)
The 
docker_image
> 
dependencies
includes: • all library source files • the go.mod file • the app source files The Dockerfile then copies these source files as:
Copy code
COPY go.mod .
COPY libraries/go ./libraries/go
COPY apps/go/greeter_en .
This makes the source files available inside the container so that 
go build .
etc has what it needs. It works! I can run the local go binary via:
Copy code
./pants run apps/go/greeter_en:greeter_en
and run the docker image via:
Copy code
./pants run apps/go/greeter_en:docker
monorepo_example.zip
@fast-nail-55400 ☝🏽 Does this look convoluted to you or is this a reasonable approach? Is there a better way? Is my message previous to above accurate? Thanks for your help on this!
Open in Slack
PreviousNext
Powered by