https://pantsbuild.org/ logo
r

rapid-bird-79300

05/19/2022, 5:58 PM
Curious if we can get pants execution to respect
~
? Seeing this error on some go builds:
Copy code
exit status 128:\n\terror: could not expand include path '~/.gitcinclude'\n\tfatal: bad config line 44 in file /usr/local/git/etc/gitconfig
semi related to this thread: https://pantsbuild.slack.com/archives/C046T6T9U/p1652934048576209
it seems most users that setup their laptops have
~
by default. One obvious fix is to replace it with
$HOME
but we want to know if there's a cleaner solution than that.
f

fast-nail-55400

05/19/2022, 6:37 PM
is the error from
git
? or some other tool?
r

rapid-bird-79300

05/19/2022, 6:38 PM
it's similar issue to what we discussed yesterday trying to download go packages
f

fast-nail-55400

05/19/2022, 6:42 PM
it is easy enough to add an option specific to the go backend to define env vars to pass through
actually we already define it apparently
--golang-subprocess-env-vars
./pants help-advanced golang
r

rapid-bird-79300

05/19/2022, 6:43 PM
I tried passing through
~
but doesn't seem to have an affect. The following still seems to fail:
Copy code
[subprocess-environment]
env_vars = [
  "HOME",
  "~",
]
oh your saying a different subprocess
f

fast-nail-55400

05/19/2022, 6:44 PM
I think
--subprocess-environment-env-vars
may not be used for Go-related subprocesses spawned by Pants because
--golang-subprocess-env-vars
exists.
r

rapid-bird-79300

05/19/2022, 6:45 PM
this would be valid right?
Copy code
[golang]
expected_version = "1.18"
subprocess_env_vars = [
    "~"
]
f

fast-nail-55400

05/19/2022, 6:46 PM
almost, you want
"HOME"
— that list is a list of environment variable names
if it is just the bare name, then pants will pass through the value to the subprocess
(you can also force a specific value in the list via
KEY=VALUE
syntax)
then when
git
sees the
~
it will consult
HOME
in the environment since Pants will have actually passed it through to the execution sandbox
r

rapid-bird-79300

05/19/2022, 6:51 PM
Copy code
njgrisafi@Nicks-MacBook-Pro ~/workspace/flex-benefits-platform (fix-release)$ pants run cmd/company_management/:
11:49:30.88 [INFO] Initializing scheduler...
11:49:31.14 [INFO] Scheduler initialized.
11:49:35.86 [ERROR] 1 Exception encountered:

  ProcessExecutionFailure: Process 'Download Go module <http://github.com/PRIVATE|github.com/PRIVATE>.' failed with exit code 1.
stdout:
{
	"Path": "<http://github.com/PRIVATE/|github.com/PRIVATE/>",
	"Version": "v0.0.0-20220518171805-7655ed2d8579",
	"Error": "<http://github.com/PRIVATE|github.com/PRIVATE>: git init --bare in /private/var/folders/4y/6r9f4kwj4f9ff6_gzwv3m9br0000gn/T/process-executionx98Dcm/gopath/pkg/mod/cache/vcs/65b7ef3e751f19c66ee80f973172312ed077987c23f151da0cdd7c440085038c: exec: \"git\": executable file not found in $PATH"
}

stderr:
Copy code
Use `--no-process-cleanup` to preserve process chroots for inspection.
f

fast-nail-55400

05/19/2022, 6:52 PM
you’ll need to make sure
git
is available to
go
via the
PATH
given to the execution sandbox (via
--golang-subprocess-env-vars
)
r

rapid-bird-79300

05/19/2022, 6:52 PM
ohhh
f

fast-nail-55400

05/19/2022, 6:53 PM
(Even without using Pants,
go
would need to be able to find
git
but Pants is complicating everything because of how the execution sandbox does not have most environment variables set.)
r

rapid-bird-79300

05/19/2022, 6:53 PM
yeah we're really hoping for an out-of-the box solution but ended up being more work than expected
f

fast-nail-55400

05/19/2022, 6:55 PM
that’s why it hasn’t graduated from being an alpha yet. It’s through users like you finding the edge cases that we can improve it.
💯 1
r

rapid-bird-79300

05/19/2022, 6:56 PM
totally understand and we definitely want to help contribute by reporting these issues.
f

fast-nail-55400

05/19/2022, 6:56 PM
for example, my testing was only for regular Go module downloads. I didn’t try private git repos at all (nor many of the other possible combinations … would be wasted effort if no users actually used those combinations)
we can add a unit test to the backend for this use if you get it working to prevent regressions
r

rapid-bird-79300

05/19/2022, 6:56 PM
so one hack I did was just putting
PATH
in
subprocess_env_vars
it works not but want to know if it's advised or not
f

fast-nail-55400

05/19/2022, 6:57 PM
that should be fine
the reluctance to put PATH in
--subprocess-environment-env-vars
is that it affects all subprocesses spawned by Pants. but putting PATH in the Go-specific
--golang-subprocess-env-vars
is fine since it only affects Go-related subprocesses and is actually needed by them.
so will say it is advised
r

rapid-bird-79300

05/19/2022, 7:06 PM
actually adding PATH creates a new issue
(open : no such file or directory)
so it somehow messes up the go build process
I'm trying to isolate the git binary. I tried using
LOCAL_BIN
(which is
/usr/local/bin
) but still seeing the git command error above
is there any way to debug and see the contents of the PATH when pants is running the goal?
just want to ensure these are getting propagated to the execution sandbox correctly
f

fast-nail-55400

05/19/2022, 7:20 PM
debug logging should show it (
-ldebug
)
you’ll see the
Process
logged
r

rapid-bird-79300

05/19/2022, 7:21 PM
Copy code
12:20:25.41 [DEBUG] spawned local process as Some(33017) for Process { argv: ["./find_binary.sh", "bash"], env: {"PATH": "/usr/bin:/bin:/usr/local/bin"}, working_directory: None, input_digests: InputDigests { complete: DirectoryDigest { digest: Digest { hash: Fingerprint<fc33a6ab65993fa57d52edbff44d8a8a89faa2ec0669950d238b481da5caf15c>, size_bytes: 91 }, tree: "Some(..)" }, nailgun: DirectoryDigest { digest: Digest { hash: Fingerprint<e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855>, size_bytes: 0 }, tree: "Some(..)" }, input_files: DirectoryDigest { digest: Digest { hash: Fingerprint<fc33a6ab65993fa57d52edbff44d8a8a89faa2ec0669950d238b481da5caf15c>, size_bytes: 91 }, tree: "Some(..)" }, immutable_inputs: {}, use_nailgun: [] }, output_files: {}, output_directories: {}, timeout: None, execution_slot_variable: None, concurrency_available: 0, description: "Searching for `bash` on PATH=/usr/bin:/bin:/usr/local/bin", level: Debug, append_only_caches: {}, jdk_home: None, platform_constraint: None, cache_scope: PerRestartSuccessful }
12:20:25.56 [DEBUG] Completed: Searching for `bash` on PATH=/usr/bin:/bin:/usr/local/bin
12:20:25.56 [DEBUG] Completed: Scheduling: Searching for `bash` on PATH=/usr/bin:/bin:/usr/local/bin
12:20:25.56 [DEBUG] Completed: acquire_command_runner_slot
12:20:25.56 [DEBUG] Running Test binary /bin/bash. under semaphore with concurrency id: 2, and concurrency: 1
12:20:25.56 [DEBUG] Completed: setup_sandbox
huh the path looks correct?
I even see this
Copy code
12:20:25.70 [DEBUG] Completed: Determine Go version for /usr/local/bin/go
12:20:25.70 [DEBUG] Completed: Scheduling: Determine Go version for /usr/local/bin/go
then on go download processes:
Copy code
env: {"LOCAL_BIN": "/usr/local/bin/", "__PANTS_CHDIR_TO": "", "__PANTS_GO_SDK_CACHE_KEY": "1.18/darwin/amd64"}
f

fast-nail-55400

05/19/2022, 7:31 PM
what is
LOCAL_BIN
?
don’t you want to set
"PATH=/usr/local/bin"
?
r

rapid-bird-79300

05/19/2022, 7:32 PM
I just set it to
/usr/local/bin
ohhhh
lol dang I thought it just prepended to the path
let me try
f

fast-nail-55400

05/19/2022, 7:32 PM
you want:
Copy code
[golang]
subprocess_env_vars = [
  "PATH=/usr/local/bin",
  "HOME",
]
👍 1
r

rapid-bird-79300

05/19/2022, 7:34 PM
awesome that is unblocked now! new error is for git auth:
Copy code
exit status 128:\n\tssh -o ControlMaster=no -o BatchMode=yes: ssh: command not found\n\tfatal: Could not read from remote repository.\n\t\n\tPlease make sure you have the correct access rights\n\tand the repository exists.
which I assume we can use the
.gitconfig
right?
f

fast-nail-55400

05/19/2022, 7:37 PM
with ssh, with
HOME
set it should find the ssh config in $HOME/.ssh — are you using
ssh-agent
at all?
(if so, you may need to pass through
SSH_AUTH_SOCK
environment variable just like
HOME
is passed through)
r

rapid-bird-79300

05/19/2022, 7:38 PM
yeah I use ssh by default but there's that suggestion in the go docs to redirect to ssh via gitconfig
I'll give mounting the agent a try
f

fast-nail-55400

05/19/2022, 7:38 PM
if you are not using it, then ssh should have found its config files
maybe add
-v
to the ssh invocation (if that can be specfied in the
.gitconfig
)?
👀 1
debugging output from ssh would be useful to see why it could not auth
r

rapid-bird-79300

05/19/2022, 7:48 PM
looking around not sure where I would set the verbose logging for git
GIT_SSH_COMMAND
can be set to override the ssh command used by git
r

rapid-bird-79300

05/19/2022, 7:54 PM
testing now
esstenially appended a
-v
to the existing cmd
👍 1
Copy code
exit status 128:\n\tssh -o ControlMaster=no -o BatchMode=yes -v: ssh: command not found\n\tfatal: Could not read from remote repository.\n\t\n\tPlease make sure you have the correct access rights\n\tand the repository exists."
need ldebug?
f

fast-nail-55400

05/19/2022, 7:56 PM
where is ssh on the system? /usr/bin?
r

rapid-bird-79300

05/19/2022, 7:56 PM
that is correct
f

fast-nail-55400

05/19/2022, 7:57 PM
probably need /usr/bin on PATH as well then and not just /usr/local/bin
r

rapid-bird-79300

05/19/2022, 8:01 PM
I jsut updated the command to
GIT_SSH_COMMAND=/usr/bin/ssh -o ControlMaster=no -o BatchMode=yes
(prepend
/usr/bin
) and now everything works as expected!
wow that was a wild ride but it's definitely working now
f

fast-nail-55400

05/19/2022, 8:01 PM
hmm does
PATH=/usr/bin:/usr/local/bin
work?
r

rapid-bird-79300

05/19/2022, 8:01 PM
let me try
using
"PATH=/usr/bin;/usr/local/bin"
results in
exec: \"git\": executable file not found in $PATH
oh my bad I see the issue
;
->` :` let me try one more time
using
"PATH=/usr/bin:/usr/local/bin"
Copy code
exit status 128:\n\tfatal: could not read Username for '<https://github.com>': terminal prompts disabled\nConfirm the import path was entered correctly.\nIf this is a private repository, see <https://golang.org/doc/faq#git_https> for additional information
f

fast-nail-55400

05/19/2022, 8:08 PM
interesting but specifying GIT_SSH_COMMAND works
probably due to
-o BatchMode=yes
r

rapid-bird-79300

05/19/2022, 8:13 PM
I'll try removing that
hmm same issue
f

fast-nail-55400

05/19/2022, 8:16 PM
ah my point was that enabling batch mode with that option is probably what makes it work
oh
it wants a username
probably need to pass the
USER
env var through as well
r

rapid-bird-79300

05/19/2022, 8:22 PM
I just reenabled the git config setting and it works
Copy code
[url "<ssh://git@github.com/>"]
	insteadOf = <https://github.com/>
but I thought the other approach above didn't require that. Let me see
okay so order matter here.
"PATH=/usr/local/bin:/usr/bin"
will pass
"PATH=/usr/bin:/usr/local/bin"
will fail
I believe that make sense as there are git binaries in both those dirs
f

fast-nail-55400

05/19/2022, 8:33 PM
ack so something particular to your setup then.
r

rapid-bird-79300

05/19/2022, 8:34 PM
yeah but all developers seem to be in the same boat so it has something to do with how machines are configured I guess
f

fast-nail-55400

05/19/2022, 8:34 PM
s/your/your organization’s/
r

rapid-bird-79300

05/19/2022, 8:34 PM
yeah exactly
thank you @fast-nail-55400 for the help this solution works great for now. Probably will make suggestions to the docs to capture this as I'd imagine others may run into the same issue in the future.
last question does
GoSdkProcess
process use the
[golang]
settings?
f

fast-nail-55400

05/19/2022, 10:24 PM
yes.
there’s a conversion rule in the backend to go from
GoSdkProcess
to
Process
and that rule applies options from
--golang-*
(i.e.,
GolangSubsystem
)
r

rapid-bird-79300

05/19/2022, 10:30 PM
okay cool was testing now a custom plugin and did see that but it outputs the same error above 🙁 the pants run works fine
f

fast-nail-55400

05/19/2022, 10:33 PM
is there more context that can be shared?
(e.g., source to the plugin or just the rule that is failing)
r

rapid-bird-79300

05/20/2022, 12:25 AM
yeah it was just the rule that was failing. was simply doing a go mod download and was seeing the same errors above. Inspecting the process it seems all the PATH vals are correct. I'll look to push a working example so we have something to refer to
7 Views