https://pantsbuild.org/ logo
Join Slack
Powered by
Installation of Python dependency from private Git...
# general
m
Installation of Python dependency from private Git repository inside 
docker_environment
Having followed instructions in Pants version control documentation, installation outside a 
docker_environment
is working successfully, however when using a docker environment and the following settings, it is failing. pants.toml
Copy code
...

[subprocess-environment]
env_vars.add = [
  "SSH_AUTH_SOCK"
]

[docker]
env_vars = [
  "SSH_AUTH_SOCK"
]

[environments-preview.names]
pytest_env = "//docker/base_emr_serverless:tests-env-base-emr-7.8.0-python-3.11"
docker/base_emr_serverless/BUILD
Copy code
docker_environment(
  name="tests-env-base-emr-7.8.0-python-3.11",
  image="<http://851725234266.dkr.ecr.us-east-2.amazonaws.com/thirdparty/base-emr-7.8.0-python-3.11:tests-0.1.0-temp-pearl-api|851725234266.dkr.ecr.us-east-2.amazonaws.com/thirdparty/base-emr-7.8.0-python-3.11:tests-0.1.0-temp-pearl-api>"
)
error during 
pants test
Copy code
pip: note: This error originates from a subprocess, and is likely not a problem with pip.
    pip:    Running command git clone --filter=blob:none --quiet 'ssh://****@github.com/PearlHealth/pearl-api.git' /pants-named-caches/pex_root/pip/1/24.2/pip_cache/.tmp/pip-download-uhu5q7ca/pearl-api_74caf017c29e4da7a7141277ba4d88d9
    pip:    Host key verification failed.
    pip:    fatal: Could not read from remote repository.
    pip:    Please make sure you have the correct access rights
    pip:    and the repository exists.
    pip:    ERROR: git clone --filter=blob:none --quiet 'ssh://****@github.com/PearlHealth/pearl-api.git' /pants-named-caches/pex_root/pip/1/24.2/pip_cache/.tmp/pip-download-uhu5q7ca/pearl-api_74caf017c29e4da7a7141277ba4d88d9 exited with 128
Somewhere the passthrough of the SSH agent is failing but I'm not sure where.
Are there any args I can override to make sure the SSH socket gets passed to the right place?
b
You ever solve this issue?
Bump. Has anyone ever gotten installing from private github repo in a docker_environment working?
Otherwise, I'm gonna move to submodules I guess?
b
> pip: Host key verification failed. Have you mounted in 
~/.ssh/known_hosts
? If you're getting the exact error message in the OP, that appears to be the problem on that output; namely host key verification is enabled (is by default on most systems), but the session is not interactive and there is no known hosts file.
I realize you may == Pants, but just want to shed focused light.
b
Not sure how to explicitly mount them in. However, I do have
Copy code
[subprocess-environment]
env_vars.add = [
  "SSH_AUTH_SOCK",
]
and
Copy code
[docker]
env_vars = [
  "SSH_AUTH_SOCK"
]
b
Yeah - I read the 
SSH_AUTH_SOCK
stuff; so you let the docker container see whatever path name is in the 
SSH_AUTH_SOCK
environment variable. You should dig a bit - is that path name mounted in the container? And, I don't know the answer to this question myself, is ssh-agent (the thing pointed at by 
SSH_AUTH_SOCK
) responsible for both authentication of you to the server and host key checking? If so, then having 
SSH_AUTH_SOCK
exposed and the path it exposes being real in the container should be enough, but if not, then you'd also need the known_hosts file available in the container too. Just prodding you to dig a bit.
b
That's fair. Dunno how long to timebox this for.
b
I just figured this out with an experiment. You need all 3 things: 1. 
SSH_AUTH_SOCK
exposed to container 2. The path in 
SSH_AUTH_SOCK
mounted into the container 3. The 
~/.ssh/known_hosts
file mounted into the container Only with all 3 does password-less ssh auto-login work. My rig:
Copy code
cat << EOF | docker build -t example-ssh -
FROM alpine
RUN apk add openssh
EOF
The successful experiment:
Copy code
:; docker run --rm -e SSH_AUTH_SOCK=/ssh-agent -v $SSH_AUTH_SOCK:/ssh-agent -v $HOME/.ssh/known_hosts:/root/.ssh/known_hosts example-ssh ssh jsirois@192.168.0.37 hostname
gill-mini.local
Remove the known_hosts mount and fail:
Copy code
:; docker run --rm -e SSH_AUTH_SOCK=/ssh-agent -v $SSH_AUTH_SOCK:/ssh-agent example-ssh ssh jsirois@192.168.0.37 hostname
Host key verification failed.
Remove the `SSH_AUTH_SOCK`setup and fail differently:
Copy code
:; docker run --rm -v $HOME/.ssh/known_hosts:/root/.ssh/known_hosts example-ssh ssh jsirois@192.168.0.37 hostname
Permission denied, please try again.
Permission denied, please try again.
Received disconnect from 192.168.0.37 port 22:2: Too many authentication failures
Disconnected from 192.168.0.37 port 22
And I think 2 only works on a Linux host. I don't imagine docker on Mac supports mounting in the unix domain socket appropriately, but you may get lucky.
b
We've given up on mac support as a company.
So that's all good
That being said, it's not clear how to mount volumes into a 
docker_environment
b
Ok, well good on you for ditching Macs and at least you now have a focused question for Pants devs. Good luck!
🙏 1
4 Views
Open in Slack
PreviousNext
Powered by