https://pantsbuild.org/ logo
#general
Title
# general
a

aloof-accountant-53042

06/14/2022, 1:07 AM
i seem to be having some trouble using Pants for building go modules that depend on other private modules, and I’m unable to find any docs on the topic. what’s strange is that
go build ./...
seems to work, but
./pants lint ::
does not. I’ve tried clearing out my go mod cache, but no luck. see thread for more info.
Typically, to authenticate with github, we need to run this command in our CI environment:
Copy code
git config --global url."https://${GITHUB_ACTOR}:${GITHUB_TOKEN}@github.com".insteadOf "<https://github.com>"
furthermore, we usually set:
Copy code
export GOPRIVATE=<http://github.com/stedi|github.com/stedi>
In order for go mod to download private repos.
however, given all of the above info, just running
go build ./...
seems to download the modules successfully, but
./pants check ::
does not.
Copy code
› ./pants lint ::
18:03:30.30 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Download Go module <http://github.com/stedi/go-lib-auth@v1.1.3|github.com/stedi/go-lib-auth@v1.1.3>.' failed with exit code 1.
stdout:
{
	"Path": "<http://github.com/stedi/go-lib-auth|github.com/stedi/go-lib-auth>",
	"Version": "v1.1.3",
	"Error": "<http://github.com/stedi/go-lib-auth@v1.1.3|github.com/stedi/go-lib-auth@v1.1.3>: invalid version: git ls-remote -q origin in /private/var/folders/3r/76rkfbyd62xc745g5q302kmw0000gp/T/process-execution968cuF/gopath/pkg/mod/cache/vcs/9937a758f3283202f2e4e0efdaf91a97db92f7904132e022913acebe03b6494a: exit status 128:\n\tfatal: could not read Username for '<https://github.com>': terminal prompts disabled\nConfirm the import path was entered correctly.\nIf this is a private repository, see <https://golang.org/doc/faq#git_https> for additional information."
}
Of course, the docs that point to https://golang.org/doc/faq#git_https just reiterate that one needs to configure git to authenticate using https.
f

fast-nail-55400

06/14/2022, 1:11 AM
Pants runs
go
in the same execution sandbox that it uses for every other process invocation. Environment variables won’t be passed through unless you specifically configure them using the
--golang-subprocess-env-vars
option.
Also private repositories is not something we currently have tests for. So it might or might not work even with the environment configured correctly.
🤞 1
a

aloof-accountant-53042

06/14/2022, 1:14 AM
ah excellent. thanks. would be great if there’s an easier way for me to discover this type of gotcha in the docs. perhaps on a page like https://www.pantsbuild.org/docs/using-pants-in-ci
After a lot of tinkering, I am unable to debug why this isn’t working - go is still giving me the git error above. I’ve tried adding
subprocess_env_vars
to pants.toml, but I’m not entirely sure this is the problem, because the ~/.gitconfig doesn’t contain any environment variables. So I guess my question is, is the git config from
$HOME/.gitconfig
even being used inside the pants sandbox? Another problem I’m running into is that I’m unable to reproduce the failure on my local machine, even after scrubbing my gitconfig, and github environment variables. I’m kind of curious why I can’t get the two to behave the same in the different environments. I’ve tried running it with
--no-pantsd
and
--no-local-cache
, but it still succeeds locally. Any help would be appreciated.
f

fast-nail-55400

06/14/2022, 11:04 PM
So I guess my question is, is the git config from
$HOME/.gitconfig
even being used inside the pants sandbox?
Probably not since the Go backend does not copy that file into the sandbox.
a

aloof-accountant-53042

06/14/2022, 11:05 PM
Thanks - that might explain why it doesn’t work in the CI environment. Do you have any thoughts why this is working locally?
f

fast-nail-55400

06/14/2022, 11:08 PM
Not offhand. A debug log (via Pants option
-ldebug
) would provide more context so I could (hopefully) understand what is going on.
a

aloof-accountant-53042

06/14/2022, 11:16 PM
pants.log
f

fast-nail-55400

06/14/2022, 11:21 PM
nothing stands out to me in that log as to why it works. Are you able to get a debug log from your CI job?
it sounds like Pants will need to add proper support for
GOPRIVATE
in order for private repos to work.
a

aloof-accountant-53042

06/14/2022, 11:28 PM
Ah, that’s unfortunate.
f

fast-nail-55400

06/14/2022, 11:29 PM
Is it just being able to set
GOPRIVATE
and have
git
config visible to the module downloader?
If so, the fix should be relatively confined to just the Pants rules that download module sources.
a

aloof-accountant-53042

06/14/2022, 11:30 PM
ok! let me look into that a bit
and yes - as far as I know those are the only requirements for accessing private modules
f

fast-nail-55400

06/14/2022, 11:33 PM
my naive assessment is the main code change to Pants would be to add a
.gitconfig
file into the sandbox when the backend invokes
go mod download
a

aloof-accountant-53042

06/14/2022, 11:36 PM
that seems reasonable, but I don’t know enough about pants to comment
f

fast-nail-55400

06/14/2022, 11:37 PM
and that could be inferred from seeing
GIT_CONFIG_GLOBAL
or other env variable defined in
--golang-subprocess-env-vars
yeah, I am more just rhetorically trying to think out a potential solution in this thread
m

modern-london-16641

08/14/2023, 2:20 PM
Sorry for bumping a really old thread... Were you able to get this working?
t

thankful-magician-7427

02/12/2024, 4:18 PM
Hello 👋 I've seem to hit the same issue, sorry for bumping the thread (again), is there any workarounds regarding this or a potential solution?