Curious if we can get pants execution to respect

~

? Seeing this error on some go builds:

exit status 128:\n\terror: could not expand include path '~/.gitcinclude'\n\tfatal: bad config line 44 in file /usr/local/git/etc/gitconfig

semi related to this thread: https://pantsbuild.slack.com/archives/C046T6T9U/p1652934048576209

is the error from

git

? or some other tool?

it's similar issue to what we discussed yesterday trying to download go packages

it is easy enough to add an option specific to the go backend to define env vars to pass through

I tried passing through

~

but doesn't seem to have an affect. The following still seems to fail:

[subprocess-environment]
env_vars = [
  "HOME",
  "~",
]

I think

--subprocess-environment-env-vars

may not be used for Go-related subprocesses spawned by Pants because

--golang-subprocess-env-vars

exists.

this would be valid right?

[golang]
expected_version = "1.18"
subprocess_env_vars = [
    "~"
]

almost, you want

"HOME"

— that list is a list of environment variable names

njgrisafi@Nicks-MacBook-Pro ~/workspace/flex-benefits-platform (fix-release)$ pants run cmd/company_management/:
11:49:30.88 [INFO] Initializing scheduler...
11:49:31.14 [INFO] Scheduler initialized.
11:49:35.86 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Download Go module <http://github.com/PRIVATE|github.com/PRIVATE>.' failed with exit code 1.
stdout:
{
	"Path": "<http://github.com/PRIVATE/|github.com/PRIVATE/>",
	"Version": "v0.0.0-20220518171805-7655ed2d8579",
	"Error": "<http://github.com/PRIVATE|github.com/PRIVATE>: git init --bare in /private/var/folders/4y/6r9f4kwj4f9ff6_gzwv3m9br0000gn/T/process-executionx98Dcm/gopath/pkg/mod/cache/vcs/65b7ef3e751f19c66ee80f973172312ed077987c23f151da0cdd7c440085038c: exec: \"git\": executable file not found in $PATH"
}

stderr:

Use `--no-process-cleanup` to preserve process chroots for inspection.

you’ll need to make sure

git

is available to

go

via the

PATH

given to the execution sandbox (via

--golang-subprocess-env-vars

)

ohhh

(Even without using Pants,

go

would need to be able to find

git

but Pants is complicating everything because of how the execution sandbox does not have most environment variables set.)

yeah we're really hoping for an out-of-the box solution but ended up being more work than expected

that’s why it hasn’t graduated from being an alpha yet. It’s through users like you finding the edge cases that we can improve it.

totally understand and we definitely want to help contribute by reporting these issues.

for example, my testing was only for regular Go module downloads. I didn’t try private git repos at all (nor many of the other possible combinations … would be wasted effort if no users actually used those combinations)

so one hack I did was just putting

PATH

in

subprocess_env_vars

it works not but want to know if it's advised or not

that should be fine

actually adding PATH creates a new issue

(open : no such file or directory)

so it somehow messes up the go build process

debug logging should show it (

-ldebug

)

12:20:25.41 [DEBUG] spawned local process as Some(33017) for Process { argv: ["./find_binary.sh", "bash"], env: {"PATH": "/usr/bin:/bin:/usr/local/bin"}, working_directory: None, input_digests: InputDigests { complete: DirectoryDigest { digest: Digest { hash: Fingerprint<fc33a6ab65993fa57d52edbff44d8a8a89faa2ec0669950d238b481da5caf15c>, size_bytes: 91 }, tree: "Some(..)" }, nailgun: DirectoryDigest { digest: Digest { hash: Fingerprint<e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855>, size_bytes: 0 }, tree: "Some(..)" }, input_files: DirectoryDigest { digest: Digest { hash: Fingerprint<fc33a6ab65993fa57d52edbff44d8a8a89faa2ec0669950d238b481da5caf15c>, size_bytes: 91 }, tree: "Some(..)" }, immutable_inputs: {}, use_nailgun: [] }, output_files: {}, output_directories: {}, timeout: None, execution_slot_variable: None, concurrency_available: 0, description: "Searching for `bash` on PATH=/usr/bin:/bin:/usr/local/bin", level: Debug, append_only_caches: {}, jdk_home: None, platform_constraint: None, cache_scope: PerRestartSuccessful }
12:20:25.56 [DEBUG] Completed: Searching for `bash` on PATH=/usr/bin:/bin:/usr/local/bin
12:20:25.56 [DEBUG] Completed: Scheduling: Searching for `bash` on PATH=/usr/bin:/bin:/usr/local/bin
12:20:25.56 [DEBUG] Completed: acquire_command_runner_slot
12:20:25.56 [DEBUG] Running Test binary /bin/bash. under semaphore with concurrency id: 2, and concurrency: 1
12:20:25.56 [DEBUG] Completed: setup_sandbox

huh the path looks correct?

what is

LOCAL_BIN

?

I just set it to

/usr/local/bin

you want:

[golang]
subprocess_env_vars = [
  "PATH=/usr/local/bin",
  "HOME",
]

awesome that is unblocked now! new error is for git auth:

exit status 128:\n\tssh -o ControlMaster=no -o BatchMode=yes: ssh: command not found\n\tfatal: Could not read from remote repository.\n\t\n\tPlease make sure you have the correct access rights\n\tand the repository exists.

with ssh, with

HOME

set it should find the ssh config in $HOME/.ssh — are you using

ssh-agent

at all?

yeah I use ssh by default but there's that suggestion in the go docs to redirect to ssh via gitconfig

if you are not using it, then ssh should have found its config files

looking around not sure where I would set the verbose logging for git

testing now

where is ssh on the system? /usr/bin?

that is correct

probably need /usr/bin on PATH as well then and not just /usr/local/bin

I jsut updated the command to

GIT_SSH_COMMAND=/usr/bin/ssh -o ControlMaster=no -o BatchMode=yes

(prepend

/usr/bin

) and now everything works as expected!

hmm does

PATH=/usr/bin:/usr/local/bin

work?

let me try

interesting but specifying GIT_SSH_COMMAND works

I'll try removing that

ah my point was that enabling batch mode with that option is probably what makes it work

I just reenabled the git config setting and it works

[url "<ssh://git@github.com/>"]
	insteadOf = <https://github.com/>

ack so something particular to your setup then.

yeah but all developers seem to be in the same boat so it has something to do with how machines are configured I guess

s/your/your organization’s/

yeah exactly

yes.

okay cool was testing now a custom plugin and did see that but it outputs the same error above 🙁 the pants run works fine

is there more context that can be shared?

yeah it was just the rule that was failing. was simply doing a go mod download and was seeing the same errors above. Inspecting the process it seems all the PATH vals are correct. I'll look to push a working example so we have something to refer to