I've got an odd failure to build a wheel with nati...
# general
b
I've got an odd failure to build a wheel with native compilation in our CI/CD environment; see thread for more
We've got a dependency on the zstd library, which needs to compile some native extensions. We're getting the following failure:
Copy code
16:11:05.28 [INFO] Completed: Resolving 3rdparty/python/constraints.txt
16:11:05.29 [ERROR] Exception caught: (pants.engine.internals.scheduler.ExecutionError)
  File "/root/.cache/pants/setup/bootstrap-Linux-x86_64/2.7.0rc2_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 236, in _run_inner
    return self._perform_run(goals)
  File "/root/.cache/pants/setup/bootstrap-Linux-x86_64/2.7.0rc2_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 175, in _perform_run
    return self._perform_run_body(goals, poll=False)
  File "/root/.cache/pants/setup/bootstrap-Linux-x86_64/2.7.0rc2_py37/lib/python3.7/site-packages/pants/bin/local_pants_runner.py", line 197, in _perform_run_body
    poll_delay=(0.1 if poll else None),
  File "/root/.cache/pants/setup/bootstrap-Linux-x86_64/2.7.0rc2_py37/lib/python3.7/site-packages/pants/init/engine_initializer.py", line 136, in run_goal_rules
    goal_product, params, poll=poll, poll_delay=poll_delay
  File "/root/.cache/pants/setup/bootstrap-Linux-x86_64/2.7.0rc2_py37/lib/python3.7/site-packages/pants/engine/internals/scheduler.py", line 538, in run_goal_rule
    self._raise_on_error([t for _, t in throws])
  File "/root/.cache/pants/setup/bootstrap-Linux-x86_64/2.7.0rc2_py37/lib/python3.7/site-packages/pants/engine/internals/scheduler.py", line 508, in _raise_on_error
    wrapped_exceptions=tuple(t.exc for t in throws),

Exception message: 1 Exception encountered:

  ProcessExecutionFailure: Process 'Resolving 3rdparty/python/constraints.txt' failed with exit code 1.
stdout:

stderr:
  ERROR: Command errored out with exit status 1:
   command: /var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/venvs/short/112c54e5/bin/python3.7 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/process-executionQxvWPP/.tmp/pip-req-build-nfrv2uy0/setup.py'"'"'; __file__='"'"'/tmp/process-executionQxvWPP/.tmp/pip-req-build-nfrv2uy0/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/process-executionQxvWPP/.tmp/pip-wheel-du7rob9n
       cwd: /tmp/process-executionQxvWPP/.tmp/pip-req-build-nfrv2uy0/
  Complete output (15 lines):
  running bdist_wheel
  running build
  running build_ext
  building 'zstd' extension
  creating build
  creating build/temp.linux-x86_64-3.7
  creating build/temp.linux-x86_64-3.7/zstd
  creating build/temp.linux-x86_64-3.7/zstd/lib
  creating build/temp.linux-x86_64-3.7/zstd/lib/compress
  creating build/temp.linux-x86_64-3.7/zstd/lib/common
  creating build/temp.linux-x86_64-3.7/zstd/lib/decompress
  creating build/temp.linux-x86_64-3.7/src
  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/var/lib/buildkite-agent/.cache/pants/named_caches/pex_root/venvs/short/112c54e5/include -I/usr/include/python3.7m -c zstd/lib/compress/zstd_compress.c -o build/temp.linux-x86_64-3.7/zstd/lib/compress/zstd_compress.o -O2 -DVERSION="1.4.9.1" -DZSTD_MULTITHREAD=1 -Izstd/lib -Izstd/lib/common -Izstd/lib/compress -Izstd/lib/decompress -DZSTD_TRACE=0
  gcc: error trying to exec 'cc1': execvp: No such file or directory
  error: command 'gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for zstd
ERROR: Failed to build one or more wheels
We are running this on Amazon Linux 2; finding
cc1
is apparently not an uncommon problem on that platform, if Stack Overflow is any indication. As an initial test, I've added the explicit location of the
cc1
binary to the
PATH
in our CI/CD environment, but Pants still fails with this error. Is
PATH
not honored when Pants is building native extensions? Is there some other way to affect the environment of how Pants does these things?
h
I think executable_search_paths is the relevant option https://www.pantsbuild.org/docs/reference-pex
b
is
The special string "" will expand to the contents of the PATH env var.
correct? Or is the "special path" supposed to be
"<PATH>"
?
h
Supposed to be <PATH>. I think the markdown escaping is broken. Cc @happy-kitchen-89482. (I'm OOO today)
b
E_TOOMANYESCAPESEQUENCES
😂 2