Hi all I m still trying to figure out how to properly connec Pants #general

Hi all, I'm still trying to figure out how to prop...

powerful-eye-58407

06/23/2023, 1:11 PM

Hi all, I'm still trying to figure out how to properly connect with remote bazel cache that uses tls and basic auth. The connection and cache works from bazel builds, but I can't figure out why SSL handshake fails for pants and I could use some help debugging that, it must be something trivial in the setup I think... Details in 🧵

powerful-eye-58407

06/23/2023, 1:14 PM

So, the bazel build command that works (meaning that I can see cache being hit in the service logs):

Copy code

$ bazel build //main:hello-world \
     --remote_cache=<grpcs://username:pass@internalproxy.studio:443> \
     --tls_certificate=ca.crt \
     --tls_client_certificate=client.crt \
     --tls_client_key=key.pem \
     --remote_header='x-bazel-cache=x-bazel-cache'

the proxy needs tls, and uses x-bazel-cache header to direct the traffic to bazel cache instance

powerful-eye-58407

06/23/2023, 1:17 PM

ca.crt and client.crt contains:

Copy code

-----BEGIN CERTIFICATE-----
actual certificate content
-----END CERTIFICATE-----

and key.pem:

Copy code

-----BEGIN PRIVATE KEY-----
key content
-----END PRIVATE KEY-----

I had to modify key.pem to convert it to PKCS8 format (it started with BEGIN RSA PRIVATE KEY originally).

powerful-eye-58407

06/23/2023, 1:17 PM

now, that call hits proxy (that needs tls) and it's forwarded to bazel cache that extracts username & pass from domain

powerful-eye-58407

06/23/2023, 1:18 PM

for pants, my setup doesn't even connect to proxy properly and I get

ssl handshake failed

error in the logs of the service, with

transport error

logs in the pants output

powerful-eye-58407

06/23/2023, 1:19 PM

pants is executed like this on a very basic script:

Copy code

pants test tests/app/test_simple.py --no-local-cache --no-pantsd -ldebug --log-show-rust-3rdparty

powerful-eye-58407

06/23/2023, 1:21 PM

pants.toml

content that I think is relevant:

Copy code

[GLOBAL]
pants_version = "2.16.0"

remote_cache_read = true
remote_cache_write = false
remote_store_address = "<grpcs://username:pass@internalproxy.studio:443>"
remote_instance_name = "main"

remote_ca_certs_path = "certs.pem"
remote_store_headers = "{'X-bazel-cache':'X-bazel-cache'}"

powerful-eye-58407

06/23/2023, 1:22 PM

now, for certs.pem I've tried a bunch of various formats, and I'm not sure if it makes any difference... right now what I have is:

Copy code

-----BEGIN CERTIFICATE-----
ca content
-----END CERTIFICATE-----

-----BEGIN CERTIFICATE-----
cert content
-----END CERTIFICATE-----

-----BEGIN PRIVATE KEY-----
key content
-----END PRIVATE KEY-----

powerful-eye-58407

06/23/2023, 1:23 PM

in the logs I can see that those certificates look to be read properly (?) (twice?)

Copy code

15:20:16.80 [DEBUG] add_pem_file processed 2 valid and 0 invalid certs
15:20:16.80 [DEBUG] add_pem_file processed 2 valid and 0 invalid certs
...
15:20:27.64 [DEBUG] resolving host="internaproxy.studio"
15:20:27.64 [DEBUG] Running Determine Python dependencies for tests/app/test_simple.py:tests under semaphore with concurrency id: 1, and concurrency: 1
...
15:20:28.26 [WARN] Failed to read from remote cache (1 occurrences so far): Unknown: "transport error"

powerful-eye-58407

06/23/2023, 1:31 PM

this looks like certificates are read, but not really used; I've tried removing username:pass from remote store address but it doesn't change anything

powerful-eye-58407

06/23/2023, 1:35 PM

any tips/ideas how to debug this? thanks a lot 🙂

2 Views

Open in Slack

Previous Next