Hello everyone, we have been using pantsbuild on o...
# general
n
Hello everyone, we have been using pantsbuild on our mono repo for about a year. Our mono repository consists of Python and several different languages. We have enjoyed using pantsbuild in Python projects so far. Now we are trying to migrate our Golang projects to pantsbuild. We encountered a problem during this migration. Our Golang projects are
cross-platform
projects built with
cgo
. During the build phase, we build for different operating systems and create different executable binaries. Until now, we could run these operations in the
techknowlogick/xgo
docker image on the Linux host and build it according to the target operating systems. Looking at the pants documentation, we thought that the most logical way to do this with pantsbuild was to use
docker_environment
(https://www.pantsbuild.org/docs/reference-docker_environment) and we proceeded accordingly. When we try to build for Windows, we encounter an error like the one below. To fix this error, I changed several parameters of docker_environment, but the problem was not solved. You can find some parameters in the src/BUILD.pants file in the comment line. The attached project contains an example for windows target. To run the project, firstly you should build the docker image with the
pants package src:xgo_windows_build_agent
command. The only difference between this image from the original techknowlogick/xgo image is that some environment variables (CGO_ENABLED, GOOS, GOARCH, etc.) are defined for Windows. This built docker image will later be used together with docker_environment to make the go build. After building the Docker image, we get the below error when we try to build the Go project. We do this build with the
pants package src:bin_windows
command. Error output:
Copy code
❯ pants package src:bin_windows
13:37:24.09 [INFO] Initialization options changed: reinitializing scheduler...
13:37:24.49 [INFO] Scheduler initialized.
13:37:29.12 [ERROR] 1 Exception encountered:

Engine traceback:
  in `package` goal

ValueError: Unable to analyze main package `src:src` for go_binary target src:bin_windows: Failed to analyze Go sources generated from pants-golang/src.

This may be a bug in Pants. Please report this issue at <https://github.com/pantsbuild/pants/issues/new/choose> and include the following data: error:


    ~/Documents/dev/test/pants-golang
If you want to verify whether there is a problem with the docker image, you can verify it with the following commands. • Run the docker image that we previously built with Pants in the project directory with the
docker run -it -v $(pwd):/app --entrypoint=/bin/bash xgo-windows-build-agent
command. • I use the
cd /app
command to switch the current directory as /app in Docker, then when we run the
go build src/main.go
command. After this command, we can see that the
main.exe
file is created. How can we solve this problem, does anyone have an idea? I would appreciate your help. Thanks.
h
Thanks for providing a reproduction (for future reference - a github repo would be preferable to a zip file, but this is OK too). I'll take a look.
n
Next time I'll share it as a github repo. Thanks for your interest. We are waiting.
h
I'm not able to even build the docker image
On macos, at least
ok after futzing with my dockerhub auth setup this seems to work now
I repro this now
OK, this is because building in the
xgo_windows
docker environment creates a cross-compiled binary that cannot run in that environment!
Pants builds a helper binary called
package_analyzer
that it runs to perform static analysis of .go files (to extract their imports).
Pants assumes that the resulting binary will run in the environment on which it was built
But in this case it does not, because it has been x-compiled to target Windows
I don't see a workaround here, that assumption is pretty baked in. Getting this to work would require some code changes.
It's an interesting problem though
next step would be for you to open an issue, in https://github.com/pantsbuild/pants. Thanks!
But I would note that the xgo project is archived so may no longer be under development? https://github.com/karalabe/xgo
And that we have an open ticket for supporting cross-compilation natively using Pants mechanisms: https://github.com/pantsbuild/pants/issues/14197
So perhaps just add your use case to that last ticket, with comments, instead of opening a new one... Thanks!