Is there a good alternative to targeting

pip.conf

for Pants rather than using

.pants.rc

? I understand that

pip.conf

isn’t supported because of PEX but is there a clean workaround?

What do you want from pip.conf?

Looking to use it for specifying extra PyPi indices with relevant creds being stored in a

.netrc

file

Extra indices here: https://www.pantsbuild.org/docs/reference-python-repos#indexes And

~/.netrc

should just work by default.

That is what we already have, what I’m looking for is getting the indices from pip.conf rather than Pants configs (pants.toml, pants.rc etc.)

hi Danny, I think this deserves an issue on GitHub! 🙂 it is a fairly common request from what I heard. The idea is that you have devs with configuration files that have info on custom PyPI indices (internal ones) which they may use when invoking

pip

outside of Pants repo. Then there's Pants configuration that also needs to be kept up-to-date with the PyPI index URLs. What Danny wants is to be able to ask Pants to load the

pip.conf

and the like into the

pants.toml

instead of duplicating the URLs. This is not currently possible, i.e. we cannot load configuration for

python-repos

from a file. However, we do this in a couple of places such as this one

Yeah, so Pex could grow an option to not explicitly ignore Pip conf (which it has explicitly ignored since day 1 of switching to Pip as its internal resolver). That would be tricky, but doable - tricky just because Pex needs to control certain Pip options and would need to yell at you with a warning when it needed to ignore some of your settings; so Pex then needs to learn how to find and parse various generations of pip.conf settings. That aside - pretend Pex has that implemented today - relying on any file not controlled by Pants is a problem. With auth probably the least so, but with extra indexes, imagine this: 1. You use remote caching. 2. You add an extra index to your personal pip.conf locally. Unless Pants tracks your personal pip.conf, it will blithely retrieve old (should be invalid now) cache hits. This could confuse the heck out of you.

Unless I have misinterpreted the needs of Danny, there's no need to involve Pex here at all. We'd just have to extend how the

python-repos

indices are collected using the

fromfile

syntax just as we do in a couple of other subsystems. Pex would still be given what's coming from the

[python-repos].indexes

Unless I have misinterpreted the needs of Danny, there's no need to involve Pex here at all. We'd just have to extend how the

python-repos

indices are collected using the

fromfile

syntax just as we do in a couple of other subsystems. Pex would still be given what's coming from the

[python-repos].indexes

This is correct and what I would be looking to do. Thanks for explaining my case better than I did 😅 I’m happy to create an issue on GitHub for this. @enough-analyst-54434 Would the Pants cache still be an issue for the above? Is so, I’m assuming there isn’t a ignored-by-cache function so that specific files wouldn’t be cached ?

It could be. @fresh-cat-90827 is more current on Pants development than I am at the moment. Last I recalled fromfile just read a file from a known location with no other work done. Your use case would require a variable location (IIRC pip.conf is under ~/Library/Caches/ on Mac and ~/.cache on Linux) and the ability not just to slurp the file but parse it as well to get at the desired index information. To my knowledge we have never done these sorts of gymnastics in the options parsing stage. If so, @fresh-cat-90827 can point these example instances out on the ticket you file (more on that below). If not, this sort of heavy lifting is traditionally done later in @rule code run by the engine after options are parsed. That style code does need to deal with tracking the file contents or at the very least re-reading it every time. Again, to my stale knowledge, we don't do this for files outside the workspace. All this aside, definitely file an issue to track the use case where the details of if and how to implement can be hashed out. It would be good to eliminate this missed expectation and non-DRY it forces on Pants users as Alexey says.