https://pantsbuild.org/ logo
Join Slack
Powered by
Hi! I get the following error when running `pants ...
# general
b
Hi! I get the following error when running `pants generate-lockfiles`:
Copy code
creating '/tmp/pants-sandbox-f1coPK/.tmp/tmp3slzt946/build/pyspark-3.5.0.dist-info'
Could not gather lock metadata for 1 project with source artifacts:
1. /tmp/pants-sandbox-f1coPK/.tmp/tmpizqd38ib/usr.bin.python3.10/kenlm-0.2.0.tar.gz: Failed to install extra requirement in venv at /home/telmo/.cache/pants/named_caches/pex_root/venvs/7b553b51d30285d7667c7e705da470155a7acc96/5985ed09b49a653d6596b0e14d134c5456cf1a9f: wheel
STDERR:
Traceback (most recent call last):
  File "/usr/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/telmo/.cache/pants/named_caches/pex_root/venvs/7b553b51d30285d7667c7e705da470155a7acc96/5985ed09b49a653d6596b0e14d134c5456cf1a9f/lib/python3.10/site-packages/pip/__main__.py", line 22, in <module>
    from pip._internal.cli.main import main as _main
ModuleNotFoundError: No module named 'pip._internal'

Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.
Any clue about what this means and/or how I can debug it?
a
Does 
pip install kenlm==0.2.0
work? Removing pants from the equation could make it easier to pinpoint the root cause
b
Yes, that works fine @able-advantage-61346
a
Does pants use a different version of pip compared to your local environment?
b
How do I check pant's pip version?
a
It's probably somewhere in the logs 😅 But you should be able to tell by the Pants version you are using
pants help-advanced python
shows the current value
b
Pants is using 23.1.2, I was on 22.0. I updated the system version and installed kenlm again without any problems
I doubt it's the kenlm package, tbh. That's a long time dependency. I'm rebasing a bunch of commits, got a conflict on the lockfiles, and I'm having this issue while trying to regenerate them
a
Could it be the python version?
b
That link describes a very old pip version
Is there a way to get error logs from pants? What exactly is failing during the build step for that package?
I managed to solve this by downgrading some other (completely unrelated - it does not depend and does not share dependencies with 
kenlm
). This experience is making me scared about pants. How can I get more visibility into errors? It's not the first time I see something like this
h
Hmm, that sounds fragile even if it happened to fix things. So the way to debug this is (as mentioned on the last line of the error) to run Pants with 
--keep-sandboxes=on_failure
. Then Pants will log the location of the sandbox of the failed process.
You can poke around inside that sandbox, and, in particular, see the command Pants is running in 
__run.sh
So now you can run that command in the sandbox without Pants and observe the failure
and that may help diagnose it
In particular it may help demonstrate a pex error outside of Pants, or at least create a standalone reproduction of the problem
Even without all that, a toy repo that reproduces the problem would be golden if you are able to create one
Then I can take a closer look
But the basic problem here is: To generate a lockfile pex needs to know the transitive requirements of your requirements. If some requirement (
kenlm
in this case) is not available as a wheel then pex must try and build the sdist to get its metadata. That is what is failing here.
Specifically, the built step is failing to install 
wheel
, which is a build-time dependency of 
kenlm
So there is a lot of information in that error you pasted
As to why that installation of 
wheel
is failing, that 
No module named 'pip._internal'
seems familiar to me, but I can't find reference to it in issues
Anyway, a repro would be gold!
b
Sorry for the late reply, I didn't see these. I remember passing 
--keep-sandboxes=on_failure
but pants complained about it not knowing this flag - and I decided to ignore it and move forward. Next time I see this I will try again!
🙏 1
19 Views
Open in Slack
PreviousNext
Powered by