docker build

on the other hand has no complaints. The single stage build works under Pants as well

I'll repro in a sec. The tl;dr though is to create a git repo with a BUILD file and a go.mod and go.sum for this repo as a git submodule. Then you can try to break the following Dockerfile out into a Go build stage and a Go deploy. It will fail once you multi-stage it and succeed before.

FROM golang:1.16-alpine
WORKDIR /build
COPY submodule-root/cmd/prom-aggregation-gateway/*.go ./
COPY go.* ./
RUN go build -o /prom-aggregation-gateway
WORKDIR /build
# Pants seems to have a bug where it cannot properly deal with multi-stage builds, so do this for now
RUN rm -rf /build
ENTRYPOINT ["/prom-aggregation-gateway"]

Specifically create a Go 1.16 go.mod and go.sum in

submodule-root/cmd/prom-aggregation-gateway

and move them back out into the same directory as the BUILD file

Hmm I can't reproduce it anymore 🤷 . It was a rather strange error so I suspected my IDE edits weren't persisted to disk (even though I also checked on disk and switched from inline Docker to Dockerfile...). Fortunately it's now working in the actual target repo so problem solved for now

I really was able to flip the issue on and off it felt by undoing the two-stage change. Either a Heisenbug or a Heisenusererror