plain-sundown-25537
05/06/2021, 3:38 PMtest
(load and list out version) - but when using the same test goal with repl
It said AssertionError: Failed to cache distribution: .deps/idna-2.10-py2.py3-none-any.whl
Any idea how I can go deeper on this?❯ ./pants -linfo repl examples/test/python/greetings/util/test_cbml.py
Traceback (most recent call last):
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmptbz6uhg8/requirements.pex/.bootstrap/pex/pex.py", line 482, in execute
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmptbz6uhg8/requirements.pex/.bootstrap/pex/pex.py", line 138, in activate
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmptbz6uhg8/requirements.pex/.bootstrap/pex/pex.py", line 125, in _activate
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmptbz6uhg8/requirements.pex/.bootstrap/pex/environment.py", line 428, in activate
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmptbz6uhg8/requirements.pex/.bootstrap/pex/environment.py", line 784, in _activate
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmptbz6uhg8/requirements.pex/.bootstrap/pex/environment.py", line 608, in resolve
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmptbz6uhg8/requirements.pex/.bootstrap/pex/environment.py", line 614, in resolve_dists
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmptbz6uhg8/requirements.pex/.bootstrap/pex/environment.py", line 384, in _update_candidate_distributions
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmptbz6uhg8/requirements.pex/.bootstrap/pex/environment.py", line 344, in _load_internal_cache
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmptbz6uhg8/requirements.pex/.bootstrap/pex/environment.py", line 325, in _write_zipped_internal_cache
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmptbz6uhg8/requirements.pex/.bootstrap/pex/util.
❯ find . |grep idna-2.10-py2.py3-none-any.whl
./.cache/pex_root/installed_wheels/0e8ff7cceb2848e5adb0010e6d710edf33748117/idna-2.10-py2.py3-none-any.whl
./.cache/pex_root/installed_wheels/0e8ff7cceb2848e5adb0010e6d710edf33748117/idna-2.10-py2.py3-none-any.whl/idna
./.cache/pex_root/installed_wheels/0e8ff7cceb2848e5adb0010e6d710edf33748117/idna-2.10-py2.py3-none-any.whl/idna/__pycache__
./.cache/pex_root/installed_wheels/0e8ff7cceb2848e5adb0010e6d710edf33748117/idna-2.10-py2.py3-none-any.whl/idna/__pycache__/__init__.cpython-38.pyc
./.cache/pex_root/installed_wheels/0e8ff7cceb2848e5adb0010e6d710edf33748117/idna-2.10-py2.py3-none-any.whl/idna/__pycache__/intranges.cpython-38.pyc
./.cache/pex_root/installed_wheels/0e8ff7cceb2848e5adb0010e6d710edf33748117/idna-2.10-py2.py3-none-any.whl/idna/__pycache__/package_data.cpython-38.pyc
./.cache/pex_root/installed_wheels/0e8ff7cceb2848e5adb0010e6d710edf33748117/idna-2.10-py2.py3-none-any.whl/idna/__pycache__/core.cpython-38.pyc
./.cache/pex_root/installed_wheels/0e8ff7cceb2848e5adb0010e6d710edf33748117/idna-2.10-py2.py3-none-any.whl/idna/__pycache__/idnadata.cpython-38.pyc
hundreds-father-404
05/07/2021, 2:15 AMrepl
in CI?plain-sundown-25537
05/07/2021, 2:25 AMhundreds-father-404
05/07/2021, 2:29 AMmv ~/.cache/pants
to have a new name like ~/.cache/pants2
? (Rename rather than delete so we can grab relevant files)plain-sundown-25537
05/07/2021, 2:33 AMhundreds-father-404
05/07/2021, 2:35 AMplain-sundown-25537
05/08/2021, 6:25 PM.cache/pants//named_caches/pex_root/installed_wheels/0e8ff7cceb2848e5adb0010e6d710edf33748117/idna-2.10-py2.py3-none-any.whl
.cache/pants//named_caches/pex_root/installed_wheels/c63d97c0a0865ea8d73fb42c11069d2326d2545a/pip-21.1.1-py3-none-any.whl/pip/_vendor/idna
.cache/pants//named_caches/pex_root/installed_wheels/c63d97c0a0865ea8d73fb42c11069d2326d2545a/pip-21.1.1-py3-none-any.whl/pip/_vendor/idna/intranges.py
.cache/pants//named_caches/pex_root/installed_wheels/c63d97c0a0865ea8d73fb42c11069d2326d2545a/pip-21.1.1-py3-none-any.whl/pip/_vendor/idna/package_data.py
.cache/pants//named_caches/pex_root/installed_wheels/c63d97c0a0865ea8d73fb42c11069d2326d2545a/pip-21.1.1-py3-none-any.whl/pip/_vendor/idna/compat.py
.cache/pants//named_caches/pex_root/installed_wheels/c63d97c0a0865ea8d73fb42c11069d2326d2545a/pip-21.1.1-py3-none-any.whl/pip/_vendor/idna/idnadata.py
.cache/pants//named_caches/pex_root/installed_wheels/c63d97c0a0865ea8d73fb42c11069d2326d2545a/pip-21.1.1-py3-none-any.whl/pip/_vendor/idna/__init__.py
.cache/pants//named_caches/pex_root/installed_wheels/c63d97c0a0865ea8d73fb42c11069d2326d2545a/pip-21.1.1-py3-none-any.whl/pip/_vendor/idna/core.py
.cache/pants//named_caches/pex_root/installed_wheels/c63d97c0a0865ea8d73fb42c11069d2326d2545a/pip-21.1.1-py3-none-any.whl/pip/_vendor/idna/codec.py
.cache/pants//named_caches/pex_root/installed_wheels/c63d97c0a0865ea8d73fb42c11069d2326d2545a/pip-21.1.1-py3-none-any.whl/pip/_vendor/idna/uts46data.py
happy-kitchen-89482
05/10/2021, 7:02 PMenough-analyst-54434
05/10/2021, 7:23 PMplain-sundown-25537
05/10/2021, 7:24 PMenough-analyst-54434
05/10/2021, 7:28 PMls -lR .cache/pants/named_caches/pex_root/installed_wheels/0e8ff7cceb2848e5adb0010e6d710edf33748117/idna-2.10-py2.py3-none-any.whl/
(I'm using your output but it looks funky - presumably .cache/pants
has a prefix.plain-sundown-25537
05/10/2021, 7:31 PM❯ ls -lR ~/.cache/pants/named_caches/pex_root/installed_wheels/0e8ff7cceb2848e5adb0010e6d710edf33748117/idna-2.10-py2.py3-none-any.whl/
total 0
drwxr-xr-x 10 zhicongliang staff 320 May 8 14:41 idna
drwxr-xr-x 9 zhicongliang staff 288 May 8 14:41 idna-2.10.dist-info
/Users/zhicongliang/.cache/pants/named_caches/pex_root/installed_wheels/0e8ff7cceb2848e5adb0010e6d710edf33748117/idna-2.10-py2.py3-none-any.whl//idna:
total 552
-rw-r--r-- 2 zhicongliang staff 58 May 8 14:41 __init__.py
-rw-r--r-- 2 zhicongliang staff 3299 May 8 14:41 codec.py
-rw-r--r-- 2 zhicongliang staff 232 May 8 14:41 compat.py
-rw-r--r-- 2 zhicongliang staff 11951 May 8 14:41 core.py
-rw-r--r-- 2 zhicongliang staff 42350 May 8 14:41 idnadata.py
-rw-r--r-- 2 zhicongliang staff 1749 May 8 14:41 intranges.py
-rw-r--r-- 2 zhicongliang staff 22 May 8 14:41 package_data.py
-rw-r--r-- 2 zhicongliang staff 202084 May 8 14:41 uts46data.py
/Users/zhicongliang/.cache/pants/named_caches/pex_root/installed_wheels/0e8ff7cceb2848e5adb0010e6d710edf33748117/idna-2.10-py2.py3-none-any.whl//idna-2.10.dist-info:
total 64
-rw-r--r-- 2 zhicongliang staff 4 May 8 14:41 INSTALLER
-rw-r--r-- 2 zhicongliang staff 1565 May 8 14:41 LICENSE.rst
-rw-r--r-- 2 zhicongliang staff 9104 May 8 14:41 METADATA
-rw-r--r-- 2 zhicongliang staff 1116 May 8 14:41 RECORD
-rw-r--r-- 2 zhicongliang staff 0 May 8 14:41 REQUESTED
-rw-r--r-- 2 zhicongliang staff 110 May 8 14:41 WHEEL
-rw-r--r-- 2 zhicongliang staff 5 May 8 14:41 top_level.txt
^^ would this help?enough-analyst-54434
05/10/2021, 7:37 PMPYTHONPATH=~/.cache/pants/named_caches/pex_root/installed_wheels/0e8ff7cceb2848e5adb0010e6d710edf33748117/idna-2.10-py2.py3-none-any.whl python3.8 -c 'import idna; print(idna.__file__)'
Here you can substitute the full path to the python you think or you know Pants is using when attempting to run repl against this target.plain-sundown-25537
05/10/2021, 7:38 PM❯ PYTHONPATH=~/.cache/pants/named_caches/pex_root/installed_wheels/0e8ff7cceb2848e5adb0010e6d710edf33748117/idna-2.10-py2.py3-none-any.whl python3.8 -c 'import idna; print(idna.__file__)'
/Users/zhicongliang/.cache/pants/named_caches/pex_root/installed_wheels/0e8ff7cceb2848e5adb0010e6d710edf33748117/idna-2.10-py2.py3-none-any.whl/idna/__init__.py
enough-analyst-54434
05/10/2021, 7:43 PMplain-sundown-25537
05/10/2021, 8:03 PMenough-analyst-54434
05/10/2021, 8:09 PMplain-sundown-25537
05/10/2021, 8:14 PMenough-analyst-54434
05/10/2021, 8:49 PMplain-sundown-25537
05/11/2021, 5:55 AMidna==2.10
to requirements.txt then run ./pants repl //:idna
on the repo and I saw some errors very similar (see below), then I found out a .cache
folder and I removed .cache
- then the issue went away. Bad news is I didn’t archive the .cache
so the evidence may have gone.
New virtual environment successfully created at /Users/zhicongliang/.cache/pants/setup/bootstrap-Darwin-x86_64/2.5.0.dev3_py37.
22:42:57.54 [INFO] initializing scheduler...
22:42:58.62 [INFO] scheduler initialized.
22:43:20.66 [INFO] Completed: Resolving constraints.txt
22:43:22.29 [INFO] Completed: Building requirements.pex with 1 requirement: idna==2.10.0
Traceback (most recent call last):
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmp6ku0p2rn/requirements.pex/.bootstrap/pex/pex.py", line 482, in execute
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmp6ku0p2rn/requirements.pex/.bootstrap/pex/pex.py", line 138, in activate
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmp6ku0p2rn/requirements.pex/.bootstrap/pex/pex.py", line 125, in _activate
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmp6ku0p2rn/requirements.pex/.bootstrap/pex/environment.py", line 428, in activate
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmp6ku0p2rn/requirements.pex/.bootstrap/pex/environment.py", line 784, in _activate
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmp6ku0p2rn/requirements.pex/.bootstrap/pex/environment.py", line 608, in resolve
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmp6ku0p2rn/requirements.pex/.bootstrap/pex/environment.py", line 614, in resolve_dists
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmp6ku0p2rn/requirements.pex/.bootstrap/pex/environment.py", line 384, in _update_candidate_distributions
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmp6ku0p2rn/requirements.pex/.bootstrap/pex/environment.py", line 344, in _load_internal_cache
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmp6ku0p2rn/requirements.pex/.bootstrap/pex/environment.py", line 325, in _write_zipped_internal_cache
File "/Users/zhicongliang/Documents/GitHub/pynest/.pants.d/tmp6ku0p2rn/requirements.pex/.bootstrap/pex/util.py", line 222, in cache_distribution
AssertionError: Failed to cache distribution: .deps/idna-2.10-py2.py3-none-any.whl
./pants cleanup
and it removes cache/rekick pants.d etc. However in Pants 2.x, we don’t have similar command.
I didn’t know and forgot to delete .cache
in pants folder (I cannot remember we had .cache in Pants 1.x)
Apparently, this issue has been happening on my machine for days and survived through reboots. It seems my cache got polluted (for whatever reasons).
I would suggest we implement some command that allow users to reset “pants” if we haven’t had that yetenough-analyst-54434
05/11/2021, 2:43 PMclean
in v2, but we also want to avoid issues like you hit here.
What's confusing is <buildroot>/.cache
- which sounds like what you deleted to get things working - is a Pants v1 only directory. It's the default location for --cache-read-from
and --cache-read-to
for v1 style caching. Pants v2 does not use this style of caching and so couldn't read contents from that directory even if it wanted to.
Do you have any Pants v2 configuration that specifies <buildroot>/.cache
as the location for anything?plain-sundown-25537
05/11/2021, 3:35 PM./pants help-all |grep cache
and found out
"alias": "always_write_cache",
"description": "Whether PEX should always write the .deps cache of the .pex file to disk or not. This can use less memory in RAM-constrained environments.",
"default": "/Users/zhicongliang/.cache/pants/lmdb_store",
"value": "/Users/zhicongliang/.cache/pants/lmdb_store"
enough-analyst-54434
05/11/2021, 4:04 PM<buildroot>/.cache
comes from just now.
We leak this for the ./pants run|repl ...
cases. Basically, for all non-interactive processes - like those spawned by ./pants test ...
we assemble all needed files in a /tmp/process-executionXYZ
chroot. We also symlink /tmp/process-executionXYZ/.cache/pex_root -> ~/.cache/pants/named_caches/pex_root
. That symlink is not set up for interactive processes and so a directory is created in CWD (the buildroot) for those. That explains your .cache/pex_root
directory. If we had a long listing of that Pex cache directory before it was deleted, I'm pretty sure it would have either had permissions issues or missing files explaining the issue you encountered.
I'll file a dedicated issue to get this leak fixed up here in a bit. Thanks for uncovering this @plain-sundown-25537.plain-sundown-25537
05/11/2021, 8:21 PMhundreds-father-404
05/11/2021, 8:23 PMplain-sundown-25537
05/11/2021, 8:24 PMhundreds-father-404
05/11/2021, 8:29 PMenough-analyst-54434
05/11/2021, 8:30 PM./pants pex -o dist/arbitrary (targets or files or 3rdparty deps) && PEX_TOOLS=1 dist/arbitrary venv dist/create/venv/right/here
Of course 1 step would be nicer.happy-kitchen-89482
05/11/2021, 8:54 PMplain-sundown-25537
05/11/2021, 10:13 PM./pants repl //:xyz
to experiment, and always love to export the venv and start with the editor.enough-analyst-54434
05/12/2021, 12:28 AMrepl
is a top level goal and not Python specific, but in principle, the handwave: ./pants repl --export=create/the/repl/venv/here //:xyz foo:bar
is exactly the right context for any venv we generate. It is (or should be) completely agnostic to the underlying target(s) types beyond they be pythony.rough-minister-58256
05/21/2021, 8:40 PMpex -r requirements.txt -o blah.pex --no-pypi -i http://<magic endpoint>
or just as trivially:
pip install -r requirements.txt --index http://<magic endpoint>
happy-kitchen-89482
05/23/2021, 7:19 PMplain-sundown-25537
05/24/2021, 4:42 AMrough-minister-58256
05/24/2021, 7:01 PMplain-sundown-25537
05/24/2021, 8:23 PMambitious-student-81104
09/08/2021, 1:51 AM~/.cache/pants
is a pants v1 directory, what is the recommended way for deleting cache in pants v2?happy-kitchen-89482
09/08/2021, 1:52 PM