Hi, I am trying to create a wheel from a

python_distribution

target but the

package

goal is failing. Following is the trail of the debug output:

14:26:04.48 [DEBUG] Completed: Resolve transitive targets

14:26:14.43 [DEBUG] Completed: Downloading: <https://github.com/protocolbuffers/protobuf/releases/download/v3.20.1/protoc-3.20.1-osx-aarch_64.zip> (2.58 MB)

14:26:14.43 [DEBUG] Completed: pants.core.util_rules.external_tool.download_external_tool

14:26:14.43 [DEBUG] Completed: Generate Python from Protobuf

14:26:14.43 [DEBUG] Completed: pants.backend.python.util_rules.python_sources.prepare_python_sources

14:26:14.43 [DEBUG] Completed: pants.backend.python.util_rules.python_sources.strip_python_sources

14:26:14.43 [DEBUG] Completed: pants.backend.python.util_rules.python_sources.prepare_python_sources

14:26:14.43 [DEBUG] Canceled: pants.backend.python.util_rules.python_sources.strip_python_sources

`142614.43 [DEBUG] Completed:

package

goal`

14:26:14.43 [DEBUG] computed 1 nodes in 10.502206 seconds. there are 262 total nodes.

14:26:14.43 [ERROR] 1 Exception encountered:

Exception: Error downloading file: error sending request for url (<https://github.com/protocolbuffers/protobuf/releases/download/v3.20.1/protoc-3.20.1-osx-aarch_64.zip>): error trying to connect: proxy authentication required

I have tried adding the subsystem

subprocess-environment

with proxy variables in

pants.toml

but the error remains the same. I am using python 3.9 on an M1 Mac. My question is if the debug output says that package goal completed, why and what is it failing for?

Sounds like you’re running behind a firewall that restricts access to the internet? and you’ve followed the instructions here https://www.pantsbuild.org/docs/restricted-internet-access?

Yes, I have added proxies under

subprocess-environment

in

pants.toml

. But more importantly, what do the debug logs trying to convey?

Completed:

is probably a misleading word, it just means that the attempt completed, not that it was successful…

Ok, any leads on how to get around with it?

https://www.pantsbuild.org/docs/restricted-internet-access is the usual advice, you tried that?

Yes.

And you used the exact same name (including case) in

env_vars.add =

as the names in your env?

curl -O <https://github.com/protocolbuffers/protobuf/releases/download/v3.20.1/protoc-3.20.1-osx-aarch_64.zip>

is downloading the file successfully from the same terminal tab I am running package goal from.

BTW if the env vars you were using were upper case, try lowercase, or vice versa?

I used them in both cases finally which made that error go and now I am facing this certification error.

That’s hard to debug without seeing the contents of that file, which obviously you should not send or post

You mean contents of pants.toml?

No, of the CA certs file

Same error.

Unclear, this is being passed through to the Rust TLS client, which tends to be really strict, see https://github.com/briansmith/webpki/issues/232, but I don’t think that’s the issue here

Same error with or without setting that.

Hmm, I doubt it. Probably that var isn’t being plumbed through for some reason

Yes, its being read. Changed it to:

ca_certs_path = "/path/to/certs/file"

and it threw an error:

File "$HOME/.cache/pants/setup/bootstrap-Darwin-arm64/2.14.0_py39/lib/python3.9/site-packages/pants/engine/internals/scheduler.py", line 215, in __init__

self._py_scheduler = native_engine.scheduler_create(

ValueError: Error reading root CA certs file /path/to/certs/file: No such file or directory (os error 2)

19:25:17.46 [INFO] waiting for pantsd to start...

19:25:22.45 [INFO] waiting for pantsd to start...

19:25:27.43 [INFO] waiting for pantsd to start...

19:25:32.41 [INFO] waiting for pantsd to start...

19:25:37.48 [INFO] waiting for pantsd to start...

^CInterrupted by user:

And you’re pointing it to a .pem file?

No

./pants --ca-certs-path=/etc/ssl/cert.pem

works for me with a valid pem file

Ok, so

package

goal doesn't use curl in the background?

To clarify, this isn’t specific to the package goal, your code depends on some protobufs, so Pants downloads the proto compiler in order to compile them into python code.

This is the stacktrace if this would help:

Engine traceback:

in select

in pants.core.goals.package.package_asset

in pants.backend.python.goals.setup_py.package_python_dist (src/proto:proto-lib)

in pants.backend.python.goals.setup_py.generate_chroot

in pants.backend.python.goals.setup_py.get_sources

in pants.backend.python.util_rules.python_sources.strip_python_sources

in pants.backend.python.util_rules.python_sources.prepare_python_sources

in pants.core.util_rules.source_files.determine_source_files

in pants.engine.internals.graph.hydrate_sources (src/proto/helloworld.proto:protos)

in pants.backend.codegen.protobuf.python.rules.generate_python_from_protobuf

in pants.core.util_rules.external_tool.download_external_tool

in downloaded_file

Traceback (no traceback):

<pants native internals>

Exception: Error downloading file: error sending request for url (<https://github.com/protocolbuffers/protobuf/releases/download/v3.20.1/protoc-3.20.1-osx-aarch_64.zip>): error trying to connect: invalid peer certificate contents: invalid peer certificate: UnknownIssuer

But that downloading code doesn’t use curl, no, it uses Rust’s HTTP client library

Yes, I understand this is not specific to

package

goal.

I’m really not sure it’s related to that issue, TBH

When I point it to a dummy file (with or without .pem extension), it gives me a ValueError saying no file or directory found. This is not the same error when I use the correct cert file.

Interesting

That worked, great. Thanks! This command did not help but then I exported the certs from keychain manually and

package

worked. I exported all root certificates and then only the one which I know curl was using based on issuing authority, bundled them all in one file and it worked. And now in the subsequent runs it doesn't matter if I provide a cert file or not and which file do I provide because it seems it has cached the things required to build that one target. Is there a way to clear that cache because I want to try a couple of things before I proceed from here.

Glad that’s working. To test without the cache you can run with

--no-pantsd --no-local-cache

Cool, let me try that!

Hmm, try nuking ~/.cache/pants/ and .pants.d/ in your repo, and running with those flags

Wouldn’t nuking

~/.cache/pants

mean bootstrapping Pants again?

Yes, but we’re trying to eliminate whatever caches are preventing that download from rerunning