We have been getting away with one single resolve,...
# general
k
We have been getting away with one single resolve, but after adding
torch
one of my colleagues are getting issues with timeout:
Copy code
pants fix ::
15:54:09.20 [INFO] Canceled: Building black.pex
15:58:53.34 [INFO] Completed: Building autoflake.pex
15:58:53.35 [ERROR] 1 Exception encountered:
Engine traceback:
  in `fix` goal
ProcessExecutionFailure: Process 'Building autoflake.pex' failed with exit code 1.
stdout:
stderr:
There were 3 errors downloading required artifacts:
1. torch 2.7.1 from <https://files.pythonhosted.org/packages/e5/94/34b80bd172d0072c9979708ccd279c2da2f55c3ef318eceec276ab9544a4/torch-2.7.1-cp311-cp311-manylinux_2_28_x86_64.whl>
    pip: pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='<http://files.pythonhosted.org|files.pythonhosted.org>', port=443): Read timed out.
2. nvidia-cublas-cu12 12.6.4.1 from <https://files.pythonhosted.org/packages/af/eb/ff4b8c503fa1f1796679dce648854d58751982426e4e4b37d6fce49d259c/nvidia_cublas_cu12-12.6.4.1-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl>
    pip: pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='<http://files.pythonhosted.org|files.pythonhosted.org>', port=443): Read timed out.
3. nvidia-cudnn-cu12 9.5.1.17 from <https://files.pythonhosted.org/packages/2a/78/4535c9c7f859a64781e43c969a3a7e84c54634e319a996d43ef32ce46f83/nvidia_cudnn_cu12-9.5.1.17-py3-none-manylinux_2_28_x86_64.whl>
    pip: pip._vendor.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='<http://files.pythonhosted.org|files.pythonhosted.org>', port=443): Read timed out.
Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.
15:58:53.40 [INFO] Canceled: Building black.pex
We also got the same issue in pip but running with longer timeout solves the issue. I assume we need to increase timeout on pants such that we have time to download and cache the large
cuda
packages. Does anyone know how to do this? Or is there anyway to download and cache a resolve?
b
Pex has supported all Pip network options, including
--timeout
for quite some time (see: https://github.com/pex-tool/pex/issues/803); so you're left wrestling with Pants to plumb the option. Hopefully Pants folks can help you with that part.
c
You should be able to specify
--timeout
(or other network args) with : https://www.pantsbuild.org/stable/reference/subsystems/pex-cli#global_args Let me know if that helps.
k
Thank for all the help. I realized we could just add an env var:
PIP_DEFAULT_TIMEOUT=120 pants fix ::
After running the first time and caching the wheels, the normal
pants fix ::
command works as expected.