Hey all, does anyone know what I'm doing wrong? `...
# general
s
Hey all, does anyone know what I'm doing wrong?
Copy code
$ pants generate-lockfiles 
16:20:51.53 [INFO] Initializing scheduler...
16:20:53.64 [INFO] Scheduler initialized.
...
16:29:37.97 [ERROR] 1 Exception encountered:

Engine traceback:
  in `generate-lockfiles` goal

IntrinsicError: Error downloading file: error sending request for url (<https://github.com/astral-sh/python-build-standalone/releases/download/20250610/cpython-3.10.18%2B20250610-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz>)
I'm on a network with no direct access to the internet, i have my
http_proxy
and
https_proxy
set correctly, everything else on my system is correctly able to reach the internet through the proxy I have the following in my
pants.toml
as per the documentation:
Copy code
[subprocess-environment]
# pull in proxy settings from environment and pass them through to all subprocesses
env_vars = [
    "http_proxy",
    "https_proxy",
    "no_proxy",
    "HTTP_PROXY",
    "HTTPS_PROXY",
    "NO_PROXY",
]
but still every pants operation requiring the internet fails...
w
Proxies are like half of our problem. If I'm reading that correctly, the problem might be coming from the Pants runner trying to access the network, rather than "pants" itself. https://github.com/pantsbuild/scie-pants?tab=readme-ov-file#firewall-support I don't believe we publish fat scie-pants yet, rather we lazy load.
I think we need to make a good proxy reproduction in our repos, so that we can make sure we've captured this (common) use case adequately. I'm wondering if I can use lxc containers and some config to emulate this
s
Hi @wide-midnight-78598, thanks for this, but i don't believe that's the issue I'm facing. For one, I already have pants installed, the pants runner has been installed for quite some time, and it has already sourced its own interpreter (cpython3.11) For another, i've not had issues with ptex or any scie i've tested / built on this dev machine Additionally, I've tried replicating @breezy-electrician-41537 solution from that linked issue, I've launched a clean bash shell (
env -i bash --noprofile --norc
) and recreated just the
http_proxy
,
https_proxy
,
HTTP_PROXY
,
HTTPS_PROXY
and
PATH
variables, and I still get the same issue
w
Copy code
IntrinsicError: Error downloading file: error sending request for url (<https://github.com/astral-sh/python-build-standalone/releases/download/20250610/cpython-3.10.18%2B20250610-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz>)
What is downloading python 3-10?
s
In fact, I found another GH issue also by @breezy-electrician-41537 that seems to be exactly what I'm facing: https://github.com/pantsbuild/scie-pants/issues/468 except he reports getting it to work by switching to pants 2.27. I tried that, and i'm still getting the same error
w
And I also commented there about proxies being a pain in the ass for us
I'm going to try to make a failing case on my machine, so this is more reliably debuggable
s
Thanks @wide-midnight-78598! Let me know if I can help in any way
šŸ‘ 1
w
If you'd know how to make a failing case using podman/docker/lxc - that would be handy šŸ˜†
b
This has nothing to do with scie-pants as evidenced by the Pants log lines. Once Pants is logging, scie-pants is long done it's work.
Isn't there (for a long time) a Pants Python provider? One for pyenv, one for PBS? Do Pants people even use Pants?
Looking at the IT, the backend is `pants.backend.python.providers.experimental.python_build_standalone`: https://github.com/pantsbuild/pants/blob/1f2bc53974116099bca08e726bdaaaf1d9454f54/src/python/pants/backend/python/providers/python_build_standalone/rules_integration_test.py#L64 @stale-twilight-79248 do you use that backend?
Yeah, so the relevant version of Reqwest seems to enable sniffing proxy env vars by default: https://docs.rs/reqwest/0.12.24/reqwest/#proxies But pantsd doesn't preserve them: https://github.com/pantsbuild/pants/blob/1f2bc53974116099bca08e726bdaaaf1d9454f54/src/python/pants/pantsd/pants_daemon.py#L39C1-L54 @stale-twilight-79248 assuming your answer to "Do you use
pants.backend.python.providers.experimental.python_build_standalone
?" is yes, then you could try running with
--no-pantsd
to see if the proxy problem goes away and report back. That would at least confirm that Pants has a general issue doing downloads via the download intrinsic in proxy environments. If that fixes things, then I think you or a Pants maintainer has more than enough clues to fix this.
s
You're right @brief-scientist-13682, I am using the python-build-standalone backend. I'm trying to convert my existing monorepo to using pants, and my existing tooling is built on PBS
i tested
pants --no-pantsd generate-lockfiles
and it worked!
so i guess that means the issue lies with pantsd itself not capturing / forwarding proxy env vars, even though i've configured it to?
b
You did not configure it to, that doc you found has been misleading since it's inception. Those env vars you poked holes for only apply to Python subprocesses and not to Pants globally. Pants maintainers know about this lie, but it has managed to go unaddressed for years.
s
i see...
b
Yeah, so some Pants interested person - could be you - needs to step up. I just maintain Pex, but thought I'd spend 15 minutes root causing this one.
s
i really appreciate all your help! I'm checking issues on the pants repo / opening a new one if there isn't one. whether or not i can go a step further and make a PR... i don't know. i've only just started using pants as a user, i haven't delved into the codebase yet, but we'll see
oh wow, you didn't just link to the problem file, you pointed me at the exact thing i would need to change in order to fix this. Thank you! I'll try and put together a PR for this
b
Well, you might step back and ask yourself why Pants with no daemon sees all env vars but with the daemon masks some. I don't know the answer myself, but sounds fishy. The daemon is technically optional and just a speed hack.
Presumably, env vars are transmitted by the Pants client to the daemon via the nail gun protocol, but those env vars are not available via standard libc means and then must be manually plumbed to library calls (reqwests case) or subprocesses.
> If you'd know how to make a failing case using podman/docker/lxc - that would be handy šŸ˜† @wide-midnight-78598 FWIW, I've used mitmproxy for proxy and custom cert testing in Pex since 2021 and its worked great. No container needed: + https://github.com/pex-tool/pex/blob/main/testing%2Fmitmproxy.py + https://github.com/pex-tool/pex/blob/0a04879c8561c49ff8fecf558dafb66163fc9a17/tests/integration/scie/test_issue_2810.py#L121-L156
w
Thanks John, I’d already setup some cases inside of containers (for other Pants issues), and then I added proxy functionality into those. Seems to work adequately
s
by the way @wide-midnight-78598 i opened a GH issue and PR for this on the 4th, and haven't had any movement from any maintainers / contributors. Thinking maybe it got lost in the noise and shuffle... would you be able to take a look? https://github.com/pantsbuild/pants/pull/22933
w
Maybe got lost in the end of the year vacations and pre-holiday hectic-ness, I'll try to take a look when I'm available for more than a 15 minute burst 😩