https://pantsbuild.org/ logo
Join Slack
Powered by
Hey Pants Team, trying to setup a new project with...
# general
r
Hey Pants Team, trying to setup a new project with 
pdftotext
and followed there guide on adding os dependencies https://github.com/jalan/pdftotext?tab=readme-ov-file#macos Running with poetry I can install dependencies and run tests in IDE. When running with pants I'm getting the following error
Copy code
stderr:
pid 94763 -> /Users/nick/.cache/pants/named_caches/pex_root/venvs/a80988bb3f0e73c2683977f47507e857bda85f08/ac9399dd73ed3d731f384971c893eaef292bacba/bin/python -sE /Users/nick/.cache/pants/named_caches/pex_root/venvs/a80988bb3f0e73c2683977f47507e857bda85f08/ac9399dd73ed3d731f384971c893eaef292bacba/pex --disable-pip-version-check --no-python-version-warning --exists-action a --no-input --use-deprecated legacy-resolver --isolated -q --cache-dir /Users/nick/.cache/pants/named_caches/pex_root/pip_cache wheel --no-deps --wheel-dir /Users/nick/.cache/pants/named_caches/pex_root/built_wheels/sdists/pdftotext-2.2.2.tar.gz/2a9aa89bc62022408781b39d188fabf5a3ad1103b6630f32c4e27e395f7966ee/cp310-cp310-macosx_14_0_arm64.4e5bd424ef6b48eabc1dce8eb560babf.work /Users/nick/.cache/pants/named_caches/pex_root/downloads/2a9aa89bc62022408781b39d188fabf5a3ad1103b6630f32c4e27e395f7966ee/pdftotext-2.2.2.tar.gz --index-url <https://pypi.org/simple/> --retries 5 --timeout 15 exited with 1 and STDERR:
  error: subprocess-exited-with-error
  
  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [13 lines of output]
      Error: $HOME must be set to run brew.
      running bdist_wheel
      running build
      running build_ext
      building 'pdftotext' extension
      creating build
      creating build/temp.macosx-13.5-arm64-cpython-310
      clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -g -fwrapv -O3 -Wall -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -DPOPPLER_CPP_AT_LEAST_0_30_0=1 -DPOPPLER_CPP_AT_LEAST_0_58_0=1 -DPOPPLER_CPP_AT_LEAST_0_88_0=1 -I/usr/local/include -I/Users/nick/.cache/pants/named_caches/pex_root/venvs/a80988bb3f0e73c2683977f47507e857bda85f08/ac9399dd73ed3d731f384971c893eaef292bacba/include -I/Users/nick/.pyenv/versions/3.10.5/include/python3.10 -c pdftotext.cpp -o build/temp.macosx-13.5-arm64-cpython-310/pdftotext.o -Wall -mmacosx-version-min=10.9 -std=c++11
      pdftotext.cpp:3:10: fatal error: 'poppler/cpp/poppler-document.h' file not found
      #include <poppler/cpp/poppler-document.h>
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      1 error generated.
      error: command '/usr/bin/clang' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pdftotext
ERROR: Failed to build one or more wheels
any suggestions on how to resolve this?
c
It looks like you are trying to compile C extensions from source in that example. I'd start with: • Does it work with pip in a virtualenv? • Is there some unusually env requirements:
Copy code
Error: $HOME must be set to run brew.
In a more centralized corporate setting, you could also build the needed wheels with cibuildwheel or whatnot and then upload them to an internal PyPi
r
Thanks you the quick response, yeah to confirm install with pip does work fine which is interesting
c
As a Linux user I'm probably not the best to guess how C header files are found on MacOS in various cases
r
okay even more interesting. I can run the pex directly and it install (peeled away some of the other parts)
same exact process pants ran (remove 
-q
) and it works directly 🤔
but it always fails when executing 
pants test
or 
pants check
which is odd. I'm debugging with 
--keep-sandboxes
taking a look at 
__run.sh
Copy code
$ cat __run.sh
#!/usr/bin/env bash
# This command line should execute the same process as pants did internally.
cd /private/var/folders/9r/wq3dmr1s11vbdxs93rj0017w0000gn/T/pants-sandbox-SNCYX1
env -i CPPFLAGS= LANG=en_US.UTF-8 LC_ALL=en_US.UTF-8 LDFLAGS= PATH=$'/Users/nick/.pyenv/libexec:/Users/nick/.pyenv/plugins/python-build/bin:/Users/nick/.pyenv/plugins/pyenv-virtualenv/bin:/Users/nick/.pyenv/plugins/pyenv-update/bin:/Users/nick/.pyenv/plugins/pyenv-doctor/bin:/Users/nick/.cargo/bin:/Users/nick/.nvm/versions/node/v18.7.0/bin:/Users/nick/.pyenv/plugins/pyenv-virtualenv/shims:/Users/nick/.pyenv/shims:/Users/nick/.pyenv/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/usr/local/munki:/Users/nick/go/bin' PEX_IGNORE_RCFILES=true PEX_PYTHON=/Users/nick/Library/Caches/nce/cc4f254e87ec09cf462fdd6bc2a868254c8e2b5a398cbde07f020e2e19ebba56/bindings/venvs/2.17.1/bin/python PEX_ROOT=.cache/pex_root /Users/nick/Library/Caches/nce/cc4f254e87ec09cf462fdd6bc2a868254c8e2b5a398cbde07f020e2e19ebba56/bindings/venvs/2.17.1/bin/python ./pex --tmpdir .tmp --jobs 9 --pip-version 23.1.2 --python-path $'/Users/nick/.pyenv/versions/3.10.13/bin:/Users/nick/.pyenv/versions/3.10.5/bin:/Users/nick/.pyenv/versions/3.11.4/bin:/Users/nick/.pyenv/versions/3.9.5/bin:/Users/nick/.pyenv/versions/py-3.10.13/bin:/Users/nick/.pyenv/versions/python-3.10.13/bin:/Users/nick/.pyenv/versions/python-3.10.3/bin:/Users/nick/.pyenv/libexec:/Users/nick/.pyenv/plugins/python-build/bin:/Users/nick/.pyenv/plugins/pyenv-virtualenv/bin:/Users/nick/.pyenv/plugins/pyenv-update/bin:/Users/nick/.pyenv/plugins/pyenv-doctor/bin:/Users/nick/.cargo/bin:/Users/nick/.nvm/versions/node/v18.7.0/bin:/Users/nick/.pyenv/plugins/pyenv-virtualenv/shims:/Users/nick/.pyenv/shims:/Users/nick/.pyenv/bin:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/usr/local/munki:/Users/nick/go/bin' --output-file mypy.pex --no-emit-warnings --venv --seed verbose --venv-site-packages-copies --python /Users/nick/.pyenv/versions/3.10.5/bin/python3.10 --console-script mypy $'--sources-directory=source_files' --lock tools/python/ats-file-processor/pants-resolve.lock --no-pypi $'--index=<https://pypi.org/simple/>' --manylinux manylinux2014 --layout packed
will try to debug this
sorry all my PATH stuff is there kinda noisy
c
I suspect it has something to do with some env var being set (or not) in the sandbox You can try running 
__run.sh
and comparing that to running the pex comand at the end without all the preamble
👍 1
r
yeah testing that now will rpeort here
um so this is odd I assume it's caching, I ran the command without preamble and it worked. Then I ran 
pants check
and now it's working 🙃
but that's just my machine collaborating someone will still face this, let me try clearing cache
okay cleared cache and now we're back here ``````
going to do same test
oh man yeah it worked again let me find which env var is causing this
adding 
HOME=/Users/nick
fixed it
is there a way I can make sure this env var is always present?
I would need this for 
test
and 
check
goals (pytest + mypy)
fixed by adding
Copy code
[subprocess-environment]
env_vars.add = [
  "HOME",
]
to my 
pants.toml
🙌 1
c
Glad you got it working! The original ERROR from the package build was so odd, but it seems correct
k
okay I can go to sleep now -- thanks @rapid-bird-79300 😉
🚀 1
12 Views
Open in Slack
PreviousNext
Powered by