I have http_proxy and https_proxy set in my .bashrc and I am setting them again in the pants script

Ah, these are env vars, correct? I suspect this would be the issue. Pants runs in a hermetic environment, meaning that it strips most env vars, for better reproducibility and more cache hits. We realized that sometimes you need to be able to set arbitrary env vars, though, like here. We recently added a mechanism to do this when running a test, but that wouldn’t work here as the value needs to be set when building the PEXes. We can add a mechanism to do that for the new alpha release at the end of the week. In general, it sounds like network requests don’t work without both those env vars set, right?

it would be good to confirm that this is the issue by running pants with more verbose debugging

I can run in verbose mode

on second thought, it's likely that pants debugging wouldn't currently reveal anything useful about what's likely going on here

The more helpful thing would be to confirm this:

In general, it sounds like network requests don’t work without both those env vars set, right?

For example, try doing

pip install flake8

without the env vars set

the same timeout happens

Okay, thanks for confirming. It sounds like the issue is indeed us not propagating those two env vars. We will try to get this fixed in time for the next alpha release - we’d love to unblock you from trying out Pants. Thanks for linking to that! We’ve gone back and forth on how much we want some global “set for every single subprocess” option vs. keeping things more scoped. Here, for example, I think we want to automatically look for

http_proxy

and

https_proxy

, with the ability to override if you want. We do that for

CPPFLAGS

,

LDFLAGS

,

LC_ALL

, and

LANG

, as those are very frequently needed. Generally, we’ve been erring on the more conservative side, as it’s harder for us to take away a feature than to add it.

maybe no_proxy as well

I’ve honestly never heard of those 3 env vars, and Google isn’t being super helpful at a quick glance. Are those standard, or specific to your work environment? Nvm, issue was DuckDuckGo instead of Google, hehe.

yes, they are very common

no problem. thanks for giving pants a try and reporting this issue

their syntax for marking sources is horrible so action-env saved me from that pain by just setting PYTHONPATH

Er, this is the type of thing that is hard for us to reason about and what gives us pause with arbitrary env vars. I have a feeling we are going to end up needing this mechanism, but I think for now, stay conservative by only handling those 3 env vars. Opened https://github.com/pantsbuild/pants/issues/10746. Thank you for reporting this! Let us know if we can help with anything else or any feedback 🙂 We’re trying to make 2.0 be really polished and helpful for users.

thank YOU

Sg, thanks!

just remembered something else that used to cause problems in Bazel

We do 🙂 only not documented beyond the reference page (https://www.pantsbuild.org/v2.0/docs/reference-global), and I’m not sure where we would document it. Probably a tool tip on the “BUILD files and targets” page This is the default:

[GLOBAL]
build_patterns = ["BUILD", "BUILD.*"]

This will work with

BUILD.pants

as is. If you want to enforce only

BUILD.pants

, change to

build_patterns = ["BUILD.pants"]

Awesome 👍

mypy stubs and mypy plugins were pretty tricky too

Oof. I’ve literally been working on improving our MyPy implementation all day. It’s so confusing. Pants does not currently support MyPy plugins, but we did with the old v1 engine, so we know it is possible of course.

let me know if i can help, i can share the way it is done in the bazel plugin https://github.com/thundergolfer/bazel-mypy-integration

Thanks for linking to that! Are y’all using

.pyi

files? That’s one of the simple fixes I think we can make. Right now, Python targets like

python_library

validate that the file ends in

.py

. I think we should loosen it to allow

.pyi

, but I wanted to talk through with someone using stubs if they think it makes sense to have in

python_library

rather than a dedicated

python_type_stubs

target. I don’t think we would handle the stubs any differently; it’s more about the BUILD file modeling things better afaict. But that also is clunky - we made a huge effort in 2.0 to cut down on boilerplate by adding dependency inference, for example

dependency inference is very very appealing

It removed the biggest pain point I think a lot of Pants users had. Maintaining BUILD files is horrible, and it’s really easy for them to become stale because no one wants to update them. So, your invalidation gets thrown off. We still need to document it in some more places, but if you want to try it out, it’s enabled by default in 2.0.0a2. You may need to teach Pants about some of your third party dependencies. See https://www.pantsbuild.org/v2.0/docs/python-third-party-dependencies#basic-setup

i will try it out

Thanks for the offer, Christian! Help with MyPy would be the most impactful. https://github.com/pantsbuild/pants/issues/10131 tracks some of the improvements we’re working on as part of 2.0. If you’re curious, this is the first fix I was working on today: https://github.com/pantsbuild/pants/pull/10750 Tomorrow is getting 3rd party requirements to work, (which is embarrassing they weren’t). After that, I want to look into type stubs, including first party. Something that would help a lot is if you could please send a gist with an example source plugin? Ideally with the complexity of the setup you describe above, but minified as much as feasible. Here’s an example first-party plugin that we have for Pylint to audit that our support works: https://github.com/pantsbuild/pants/blob/fdac7c3e26a99fc57707e6193b4f27a1c19cee17/src/python/pants/backend/python/lint/pylint/rules_integration_test.py#L333-L424. It’s helpful for the test to be involved, like living in multiple source files and having dependencies on 3rd party reqs, etc

i love mypy more than python. Yes, I can put together a gist. If the setup is not very complicated I can try forking the repo and do it in a branch.

hey @acceptable-guitar-79854 not sure if this will help you or not but I also need to run everything under proxy (have http_proxy/https_proxy/ca_requests_bundle and another infinite amount of envs that I have to set) and what I had to do to have pants work was comment this line:

# export no_proxy='*'

hey @wooden-thailand-8386 i have that commented out, but the problem persists. Actually I tried with and without it. Also, I set http_proxy, https_proxy and no_proxy in my .bashrc and at the top of the "pants" script

Thanks for the suggestion, Thales! Yeah, this works for Thales because they’re using the v1 engine which is not hermetic, but Cristian is using the v2 engine. @happy-kitchen-89482 is working on fixing this today

thanks for clarifying @hundreds-father-404

@wooden-thailand-8386 what is in your

ca_requests_bundle

? Is it a .pem file with several concatenated certificates?

@happy-kitchen-89482 it is a .crt file with a lot of concatenated certs

cool, so each cert is a Base64-encoded PEM-format I assume

yes, just checked on an online decoder tool haha

Update @acceptable-guitar-79854 that MyPy will now work with 3rdparty libraries and plugins (PEP 561). If you have a moment, it’d be helpful if you could please look over the test cases and think if you know of any other arrangement of 3rd party requirements that Pants should support. https://github.com/pantsbuild/pants/pull/10741

looking now