I'm hitting an issue I can't figure out how to deb...
# general
n
I'm hitting an issue I can't figure out how to debug. Specifically, I didn't change my
requirements.txt
at all - I just ran
pants --keep-sandboxes=on_failure generate-lockfiles
to upgrade dependencies but it fails. However, even with
--keep-sandboxes
the
pip
log files get deleted and the errors don't even tell me which library is causing the issue. Details in the thread.
Full log:
Copy code
$ pants  --keep-sandboxes=on_failure generate-lockfiles
18:39:01.74 [INFO] Completed: Generate lockfile for isort
18:44:47.40 [INFO] Preserving local process execution dir /tmp/pants-sandbox-U4DEtO for Generate lockfile for default
18:44:47.40 [INFO] Completed: Generate lockfile for default
18:44:47.40 [ERROR] 1 Exception encountered:

Engine traceback:
  in `generate-lockfiles` goal

ProcessExecutionFailure: Process 'Generate lockfile for default' failed with exit code 1.
stdout:

stderr:
pid 379370 -> /home/oliver/.cache/pants/named_caches/pex_root/venvs/0/d2a0b9aae994f50ed4a65c49c7f3c0993ad42ad6/1057f2c0b7625bfb782881e11db8c74ad22c7fae/bin/python /home/oliver/.cache/pants/named_caches/pex_root/venvs/0/d2a0b9aae994f50ed4a65c49c7f3c0993ad42ad6/1057f2c0b7625bfb782881e11db8c74ad22c7fae/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --isolated --log /tmp/pants-sandbox-U4DEtO/.tmp/pex-pip-log.umymplrn/pip.log -q --cache-dir /home/oliver/.cache/pants/named_caches/pex_root/pip/1/22.2.2/pip_cache download --dest /tmp/pants-sandbox-U4DEtO/.tmp/tmpja1upx9c/home.oliver..pyenv.versions.3.10.6.bin.python3.10 Jinja2 Pillow aiofiles alive_progress av>14.1.0 beautifulsoup4 coral_pytorch einops google-cloud-compute google-cloud-pubsub google-cloud-storage h5py httpx humanize imageio[pyav] jsonschema[format-nongpl] kubernetes kubernetes-stubs-elephant-fork label-studio-sdk>=1.0.16 lakefs<=0.10.0 lxml markdown matplotlib memray nats-py neptune>=1.5.0 numpy<2.0 onnx psycopg[binary]!=3.2.4 pydantic>=2.0 pytest pytest-asyncio==0.23.8 pytest-benchmark pytorchvideo pyyaml qreader>=3.16 requests scipy seaborn>0.12.2 setuptools torch==2.6.0 torchmetrics torchvision==0.21.0 types-Markdown types-Pillow types-PyYAML types-aiofiles types-beautifulsoup4 types-requests wcmatch --index-url <https://pypi.org/simple/> --extra-index-url <https://download.pytorch.org/whl/cu118> --retries 5 --timeout 15 exited with 1 and STDERR:
pip: Ignoring the following environment variables in Pex venv mode:
pip: _PEX_CACHE_ACCESS_LOCK=0|4|/home/oliver/.cache/pants/named_caches/pex_root/access.lck
pip:   error: subprocess-exited-with-error
pip:   
pip:   × pip subprocess to install build dependencies did not run successfully.
pip:   │ exit code: 1
pip:   ╰─> See above for output.
pip:   
pip:   note: This error originates from a subprocess, and is likely not a problem with pip.
pip: error: subprocess-exited-with-error
pip: 
pip: × pip subprocess to install build dependencies did not run successfully.
pip: │ exit code: 1
pip: ╰─> See above for output.
pip: 
pip: note: This error originates from a subprocess, and is likely not a problem with pip.
Note that
/tmp/pants-sandbox-U4DEtO/.tmp
exists but is empty (as you might expect from a directory name "tmp") so I can't see the pip log at
/tmp/pants-sandbox-U4DEtO/.tmp/pex-pip-log.umymplrn/pip.log
and I don't see any other information that could tell me what library is causing the issue. Any ideas?
I was able to find the bad dependency by just doing a
pip install -r requirements.txt
in a different venv but I'd still like to understand how to debug things like this with Pants.
b
You're using Pants older than 2.26.0 right? Because `--preserve-pip-download-log pex-pip-download.log` was added here https://github.com/pantsbuild/pants/pull/21980
You should be able to edit your
__run.sh
and add those same args even if running older Pants.
n
Perfect. Thanks!