I would like to prevent Pants from using existing Pip config when building its internal envs, essentially using

--isolated

mode. Is there a way I can do this?

I'm not sure I'm following the question. The Pex default is <https://github.com/pex-tool/pex/blob/b42697bac49c8527523c29924dd65b6ca19c0273/pex/resolve/resolver_options.py#L512> and I don't think Pants exposes a way short of passing arbitrary extra args that would set

--use-pip-config

I would like to set or isolate the Pip options that are used when bootstrapping Pants environments e.g. installed pex and other internal dependencies. I don’t think this is performed via pex given that it’s installing pex. The issue I’m seeing is that users’

PIP_USER_CONF

file may contain invalid config, which can cause this bootstrap of environments to fail, even though Pants itself won’t be using this Pip config.

So with

example-python

something like?:

$ tail -2 ~/.config/pip/pip.conf 

invalid-junk

$ pants --local-store-dir=$(mktemp -d) --named-caches-dir=$(mktemp -d) --no-pantsd package ::
11:45:27.07 [INFO] Completed: Scheduling: Test binary /bin/bash.
11:45:27.07 [INFO] Completed: Scheduling: Test binary /usr/bin/bash.
11:45:31.22 [INFO] Completed: Scheduling: Find interpreter for constraints: CPython==3.12.*
11:45:31.30 [INFO] Canceled: Building build_backend.pex from <resource://pants.backend.python.subsystems/setuptools.lock>
11:45:33.24 [INFO] Completed: Building local_dists.pex
11:45:33.24 [INFO] Completed: Scheduling: Building local_dists.pex
11:45:36.48 [INFO] Completed: Building build_backend.pex from <resource://pants.backend.python.subsystems/setuptools.lock>
11:45:36.48 [INFO] Completed: Scheduling: Building build_backend.pex from <resource://pants.backend.python.subsystems/setuptools.lock> 
11:45:36.49 [ERROR] 1 Exception encountered:

Engine traceback:
  in `package` goal

ProcessExecutionFailure: Process 'Building build_backend.pex from <resource://pants.backend.python.subsystems/setuptools.lock>' failed with exit code 1.
stdout:

stderr:
received exit code 2 during execution of `['/tmp/pants-sandbox-mVrOCa/.tmp/tmp8qur1nbv/pip/bin/python3.12', '-s', '-E', '-m', 'pip', 'install', '-U', 'pip']` while trying to execute `['/tmp/pants-sandbox-mVrOCa/.tmp/tmp8qur1nbv/pip/bin/python3.12', '-s', '-E', '-m', 'pip', 'install', '-U', 'pip']`

Yep, although in our instance it would be specifically a pip.conf including a

index-url=<private-repo>

entry for which pip would receive a 401 for (non-existent link) when querying for these internal dependencies such as pex.

I take it, "don't have this

pip.conf

file present" or "have the

pip.conf

file and auth present" are harder then they sound? The specific issue -- at least in my repo -- is the get-the-right-pip bootstrapping process. There is somewhat related discussion about this problem in the keyring provider case at https://github.com/pex-tool/pex/pull/2592 I was looking at the keyring path for use with an registry that only uses short lived tokens, but ended up going in a different direction.

Pex is "installed" via downloading the Pex PEX. No pip is involved. For example, for Pex 2.74.1, from here: https://github.com/pex-tool/pex/releases/download/v2.74.1/pex This downloading is done via a Rust engine intrinsic action.

I take it, “don’t have this

pip.conf

file present” or “have the

pip.conf

file and auth present” are harder then they sound?

Not necessarily harder, but it’s very useful to rely on the bootstrap process being as hermetic as Pants’ spun-up environments are, so that we can serve all users with less requirements on pre-bootstrap stuff.

There is somewhat related discussion about this problem in the keyring provider case at https://github.com/pex-tool/pex/pull/2592 I was looking at the keyring path for use with an registry that only uses short lived tokens, but ended up going in a different direction.

Having this exposed in Pants would be great. IMO keyring and keyring-provider logic within Pip could be improved quite a lot, we’ve found it to be quite frustrating to discover and validate all potential problem cases in our scenario which requires keyring, the gauth keyring backend and using it in solely subprocess mode.

this looks like a bug here: https://github.com/pex-tool/pex/blob/b42697bac49c8527523c29924dd65b6ca19c0273/pex/pip/installation.py#L258

Pex is bootstrapping the Pip for

pip_version

and failing to apply all the standard options Pex uses for performing

pip download

(like

--isolated

). If you don’t mind filing an issue, I’m AFK and won’t be able to get to a fix for the next several days; possibly not until ~17th December.

Happy to have assisted in uncovering a bug. I’ll take a look and understand the issue fully and write a ticket for you. No pressure on a fix, as we know what the problem is - but of course would be very useful to have in a release at some point.