https://pantsbuild.org/ logo
Join Slack
Powered by
I am fighting a bug where generating `requirements...
# general
k
I am fighting a bug where generating 
requirements.pex
always fails the first time. When I say "first time" I mean from a new environment with XDG_CACHE_HOME cleared. Very hard to debug because most of the time it works the second time. Going into the sandbox does not really help because when I run it again it will pass. Any pointers for troubleshooting steps?
This is one of the errors:
Copy code
18:36:01.96 [INFO] Completed: Building pytest.pex from <resource://pants.backend.python.subsystems/pytest.lock>
18:36:01.97 [ERROR] 1 Exception encountered:

Engine traceback:
  in `test` goal

ProcessExecutionFailure: Process 'Building pytest.pex from <resource://pants.backend.python.subsystems/pytest.lock>' failed with exit code 1.
stdout:

stderr:
Failed to spawn a job for /usr/bin/python3.11: [Errno 2] No such file or directory: '/tmp/pyrolite/pants/xdg_cache/pants/named_caches/pex_root/pip/20.3.4-patched/pip.pex/f4e1d8fb1bdc118d736c7140de0d0f99763316c4.lck.work/.bootstrap/pex/venv/README.md'
I notice that the 
no such file or directory
sometimes mention different files. IDK if that helps...
For example, anothre repro got me this error:
Copy code
Failed to spawn a job for /usr/bin/python3.11: encountered NonZeroExit("received exit code 1 during execution of `['/tmp/pyrolite/pants/scie_base/7d19e1ecd6e582423f7c74a0c67491eaa982ce9d5c5f35f0e4289f83127abcb8/cpython-3.9.18+20240107-aarch64-unknown-linux-gnu-install_only.tar.gz/python/bin/python3.9', '-s', '-E', '/tmp/pants-sandbox-ufegZu/.tmp/tmpud1_63yh']` while trying to execute `['/tmp/pyrolite/pants/scie_base/7d19e1ecd6e582423f7c74a0c67491eaa982ce9d5c5f35f0e4289f83127abcb8/cpython-3.9.18+20240107-aarch64-unknown-linux-gnu-install_only.tar.gz/python/bin/python3.9', '-s', '-E', '/tmp/pants-sandbox-ufegZu/.tmp/tmpud1_63yh']`") during bytecode compilation.
stderr was:
Traceback (most recent call last):
  File "/tmp/pants-sandbox-ufegZu/.tmp/tmpud1_63yh", line 46, in <module>
    main(root, relpaths)
  File "/tmp/pants-sandbox-ufegZu/.tmp/tmpud1_63yh", line 31, in main
    compiled, errored = compile(root, relpaths)
  File "/tmp/pants-sandbox-ufegZu/.tmp/tmpud1_63yh", line 23, in compile
    py_compile.compile(abspath, cfile=pyc_abspath, dfile=relpath, doraise=True)
  File "/tmp/pyrolite/pants/scie_base/7d19e1ecd6e582423f7c74a0c67491eaa982ce9d5c5f35f0e4289f83127abcb8/cpython-3.9.18+20240107-aarch64-unknown-linux-gnu-install_only.tar.gz/python/lib/python3.9/py_compile.py", line 142, in compile
    source_bytes = loader.get_data(file)
  File "<frozen importlib._bootstrap_external>", line 1039, in get_data
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pyrolite/pants/xdg_cache/pants/named_caches/pex_root/pip/20.3.4-patched/pip.pex/f4e1d8fb1bdc118d736c7140de0d0f99763316c4.lck.work/.bootstrap/pex/argparse.py'
PANTS_PROCESS_EXECUTION_LOCAL_PARALLELISM=1 seems to fix my issue.
g
Interesting case. What Pants version are you using?
c
we think it might be related to Docker in Mac with virtiofs driver
switching to gFUSE seems to fix it (but make it much slower)
we are looking to try with Docker Volume instead of mount
(We are trying to run pants in Docker)
g
Ah, that makes a lot of sense. Out of curiosity, is /tmp actually a mount or some other fs type? 🙂
k
Yep
13 Views
Open in Slack
PreviousNext
Powered by