👋 having an issue (i think) passing http_proxy environment to the pip resolver when pants is bootstrapping / resolving plugins in pants 1.26 on a build machine that requires a proxy to access the internet (full logs with -ltrace and PEX_VERBOSE=9 in thread):

10:56:38 15:56:38 [INFO] Resolving new plugins...:
10:56:38   pantsbuild.pants.contrib.avro==1.26.0
10:56:38   pantsbuild.pants.contrib.googlejavaformat==1.26.0
10:56:38   pantsbuild.pants.contrib.python.checks==1.26.0
10:56:38   pantsbuild.pants.contrib.scrooge==1.26.0
10:56:38   pantsbuild.pants.contrib.thrifty==1.26.0
10:56:38 pex: Resolving for:
[SNIP]
10:56:39 Starting new HTTPS connection (1): <http://pypi.org:443|pypi.org:443>
10:57:39 Incremented Retry for (url='/simple/pantsbuild-pants-contrib-avro/'): Retry(total=4, connect=None, read=None, redirect=None, status=None)
10:57:39 WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fc21492b080>, 'Connection to <http://pypi.org|pypi.org> timed out. (connect timeout=15)')': /simple/pantsbuild-pants-contrib-avro/

Hey John! To confirm, are you all using v1 or v2 for 1.26? If v2, you would have

backend_packages2

in your

pants.toml

and the UI would be the new "dynamic UI", where the screen refreshes

this is pants v1 version 1.26

Cool, thanks. With v2, you have to tell Pants to propagate your

http_proxy

value because v2 is more hermetic, but that should be irrelevant here. I think we leak

http_proxy

to the underlying subprocesses Is this your first time running it on this machine, or this is a regression?

correct from pypi.org

Hm okay. I'm rereading our v1 docs on proxies at https://v1.pantsbuild.org/setup_repo.html, and indeed I'd expect only needing to set

http_proxy

and/or

https_proxy

. Does the

REQUESTS_CA_BUNDLE

part look relevant to you?

i don't believe I'd get the ConnectTimeoutError if that were the issue, but I can try

Also, this is unsatisfying, but in the worst case, you could copy the plugin files and treat them as a first-party plugin rather than resolving via PyPI https://github.com/pantsbuild/pants/tree/1.26.x/contrib/avro/src/python/pants/contrib/avro Lmk if you want to take that shortcut and we can help with how to structure things / tweak

pants.toml

thanks! it's weird, i don't see where http_proxy environment variables get dropped either. i attemped to follow the code from pants -> pex -> pip that gets triggered when resolving plugins. i see code for removing PEX-prefixed env variables, etc, but I don't see where

http_proxy

gets dropped

It is weird, I'm trying to retrace https://github.com/pantsbuild/pants/blob/1.26.x/src/python/pants/init/plugin_resolver.py and can't figure it out either. There, we use Pex as a library. But that should not be dropping any env vars outside of some special ones like

PYTHONPATH

thanks, i'll have a look into that we may need to anyway give that the avro isn't developed first-party anymore

i believe the plugin download issues are a regression from pants 1.25 though

It's plausible, 1.26 switched to using Pex 2, which vendors pip to do resolves instead of using a custom resolver. Even though that generally made things much more robust, there were some hiccups

ah interesting. its primarily a scala repo, with a few python binaries kicking around. appreciate all this info!

i believe the plugin download issues are a regression from pants 1.25 though

Also, it may be worth testing if it's fixed in later Pants versions, as they consume newer versions of Pex. I don't think we had any dedicated bug for this that we fixed..but might have been fixed without us knowing. If you want to try, I'd recommend creating a new minimal repository with nothing more than the

./pants

script and a

pants.toml

that loads those plugins. Simply run

./pants --version

to see if resolving the plugins work. This way, you avoid dealing with deprecation warnings

👋 @bored-mechanic-51694 🙂

👋 hey @happy-kitchen-89482!