https://pantsbuild.org/ logo
Join Slack
Powered by
```[python-repos] find_links = [ "<https://dow...
# general
b
Copy code
[python-repos]
find_links = [
    "<https://download.pytorch.org/whl/torch_stable.html>",
]
indexes = [
    "<https://pypi.org/simple/>",
    "<https://PRODUCT_KEY@download.prodi.gy>"
]
The above is part of my 
pants.toml
. This is because of a paid python package we rely on. In our 
requirements.txt
, we have a 
prodigy>=1.17.3
dependency. This worked fine until we updated pants to 
2.22
. We moved from pants 
2.20
because of a bug with having python 3.13 installed in our system. However, every time we need to update the lockfiles, we have to 1) downgrade pants, 2) generate lockfiles, 3) update pants again. The error we get with pants 
2.22
is:
Copy code
ERROR: HTTP error 403 while getting <https://s3.eu-west-1.amazonaws.com/xplsn-pypi-eu-west-1.prodi.gy/dist/prodigy-1.17.4-py3-none-any.whl?X-Amz->....
Does anyone know why this is happening and how to fix it?
c
If you have a quick local reproduction case, could you see if you see the same behavior with 
2.23
or 
2.24.0rc3
?
b
We had some other problem with 
2.23
. Something didn't build, I forget what now. I'll try 2.24.0rc3
gratitude thank you 1
I forgot to report here. Version 2.24 errors too:
Copy code
pip:     raise ReadTimeoutError(self._pool, None, "Read timed out.")
pip: pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='<http://s3.eu-west-1.amazonaws.com|s3.eu-west-1.amazonaws.com>', port=443): Read timed out.
Right now we're stuck on 
2.20
, but that version has problems too. Running 
pants fmt ::
fails if python 3.13 is installed on the system (which brew does automatically)
g
Can you confirm that this works if you use pip directly? If yes, does it work with pex directly?
(Reason I'm asking is because Pants delegates almost all dependency management to Pex, which in turn delegates a fair bet of dependency resolution to pip. And both of those can change when you change Pants version.)
h
What is the failure if python 3.13 is installed on the system? I assume it’s related to a resolve failing because of our overly broad default tool interpreter constraints (CPython>=3.7,<4). There are libraries that cannot be resolved for that broad a range any more.
So you may need to build a custom tool lockfile for just the python(s) you actually care about
(that is just addressing the 3.13 issue, not the primary issue above)
Regarding the primary issue: So you get a 
403
on 2.22 and a timeout on 2.24?
b
@happy-kitchen-89482 This is the problem with having python 3.13 on the system:
Copy code
pants fmt ::
05:21:54.72 [INFO] Initializing scheduler...
05:21:54.73 [INFO] Initializing Nailgun pool for 32 processes...
05:21:56.54 [INFO] Scheduler initialized.
05:21:58.11 [ERROR] 1 Exception encountered:

Engine traceback:
  in `fmt` goal

ProcessExecutionFailure: Process 'Find interpreter for constraints: CPython<4,>=3.7' failed with exit code 1.
stdout:

stderr:
No supported version of Pip is compatible with the given targets:
cp310-cp310-macosx_15_0_arm64 interpreter at /Users/arielsalem/.pyenv/versions/3.10.12/bin/python3.10
cp311-cp311-macosx_15_0_arm64 interpreter at /Library/Frameworks/Python.framework/Versions/3.11/bin/python3.11
cp313-cp313-macosx_15_0_arm64 interpreter at /opt/homebrew/Cellar/python@3.13/3.13.1/Frameworks/Python.framework/Versions/3.13/bin/python3.13
cp39-cp39-macosx_15_0_arm64 interpreter at /Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/bin/python3.9
We have 
interpreter_constraints = ["==3.10.12"]
in our 
pants.toml
. 
3.10.12
is installed in the system. Running 
brew uninstall python@3.13
fixes this error
Regarding the primary issue, yes, the errors are those I shared above. Everything works fine on 
2.20
. Answering @gorgeous-winter-99296 this command works
Copy code
pip install -r requirements.txt --extra-index-url <https://PRODUCT_KEY@download.prodi.gy>
My pants.toml has the below:
Copy code
indexes = [
    "<https://pypi.org/simple/>",
    "<https://PRODUCT_KEY@download.prodi.gy>"
]
g
I think with pex installed 
pex3 lock create -r requirements.txt --extra-index-url <https://PRODUCT_KEY@download.prodi.gy>
should be equivalent
h
Yeah, so that error is due to tool ICs, not your code’s ICs (which you have correctly pinned). You may need to override those for the formatters you use.
It was a bad mistake on our part to have default ICs for tools, especially ones so permissive.
b
How do I know what is the pex command that pants generates? That way I can validate if it's as expected of if there's some other flag I can pass @happy-kitchen-89482: how do I override the formatters?
g
If you pass 
-ldebug
you should see the pex command in the logs, probably multiple pex invocations in sequence.
You can also pass 
--keep-sandboxes=on_failure
(or 
=always
) and look in the sandbox script, which you can also edit and rerun manually.
h
You generate “tool lockfiles” for those formatters: https://www.pantsbuild.org/stable/docs/python/overview/lockfiles#lockfiles-for-tools
But this is an unfortunate footgun that we should fix
c
But this is an unfortunate footgun that we should fix
By not having default interpreter constraints? Different ones? Or something else?
h
I think forcing you to generate tool lockfiles for your interpreter(s) of choice…
b
Apologies for the delay in my response. @gorgeous-winter-99296: passing 
-ldebug
shows that the flag passed to pex is 
--index=<https://PRODUCT_KEYC@download.prodi.gy>
. Is there a way to make this become 
--extra-index-url
? @happy-kitchen-89482: I will try tool lockfiles today
g
In terms of pex, --index is extra-index.
Copy code
-i URL, --index URL, --index-url URL
                        Additional cheeseshop indices to use to satisfy
                        requirements. (default: None)
b
🤔 In that case I don't know why generate-lockfiles is failing. I'll try Benjy's solution and downgrade back to pants 2.20.0 if it fixes our problem
Downgrading to 
2.20.0
and creating tool lockfiles solved the problem. Thank you so much!
4 Views
Open in Slack
PreviousNext
Powered by