https://pantsbuild.org/ logo
Join Slack
Powered by
So I'm stumped right now on my docker image. We a...
# general
b
So I'm stumped right now on my docker image. We are trying to create docker images for our devs to use for python projects. I've currently got things building and caching files per the instructions on the site about CI. Unfortunately, once the container is started up and pants is run, it downloads all of the pants configuration, runs installs and eventually we end up with the following issue:
Copy code
[Errno 17] File exists: '__main__.py' -> '/root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/venvs/2.18.0rc3/pex'
Install failed: Command '['/root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/pex_root/venvs/dd209a3851314c876db14642bafdf970fab77250/d95804f90e6d85181506d9552cdd5b3debfc6d3b/bin/python', '/tmp/tmpz92056ym.pex', 'venv', '--prompt', 'Pants 2.18.0rc3', '--compile', '--pip', '--collisions-ok', '--no-emit-warnings', '--disable-cache', '/root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/venvs/2.18.0rc3']' returned non-zero exit status 1.
More information can be found in the log at: /root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/logs/install.log

Error: Failed to establish atomic directory /root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/locks/install-7a42cd626476fbf810095f22d0d42f684e073604d82ffd5b6a93c41956a4e74a. Population of work directory failed: Boot binding command failed: exit status: 1

Isolates your Pants from the elements.

Please select from the following boot commands:

<default> (when SCIE_BOOT is not set in the environment)  Detects the current Pants installation and launches it.
bootstrap-tools                                           Introspection tools for the Pants bootstrap process.
update                                                    Update scie-pants.

You can select a boot command by setting the SCIE_BOOT environment variable.
I've tried to follow the advice of others and simplified the creation of this image, but still have come up empty. The dockerfile in use is
Copy code
ENV XDG_CACHE_HOME=/root/.cache

ARG BUILD_ROOT=/build
ENV BUILD_ROOT=$BUILD_ROOT

ARG SCIE_PANTS_VERSION
ENV SCIE_PANTS_VERSION=$SCIE_PANTS_VERSION
ARG PANTS_VERSION
ENV PANTS_VERSION=$PANTS_VERSION

ENV VENV_DIR=/root/.venv

SHELL ["/bin/bash", "-c"]

RUN --mount=type=bind,source=/,target=$BUILD_ROOT source /opt/rii/rii-python38/enable && \
    source /opt/rh/devtoolset-7/enable && \
    python -m virtualenv "${VENV_DIR}" && \
    source "${VENV_DIR}/bin/activate" && \
    source "${BUILD_ROOT}/bin/activate" && \
    cd /tmp && \
    touch pants.toml && \
    ${BUILD_ROOT}/bin/setup && \
    rm pants.toml
Any help would be greatly appreciated.
Here is the output from one of the log files:
Copy code
2023-10-24 23:50:23,334 INFO] root: Bootstrapping Pants 2.18.0rc3
2023-10-24 23:50:23,334 DEBUG] root: Pants itself is using: cpython 3.9.16
2023-10-24 23:50:23,334 INFO] root: Installing pantsbuild.pants==2.18.0rc3 into a virtual environment at /root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/venvs/2.18.0rc3
2023-10-24 23:50:58,678 INFO] root: New virtual environment successfully created at /root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/venvs/2.18.0rc3.
2023-10-24 23:59:55,953 INFO] root: Bootstrapping Pants 2.18.0rc3
2023-10-24 23:59:55,954 DEBUG] root: Pants itself is using: cpython 3.9.16
2023-10-24 23:59:55,954 INFO] root: Installing pantsbuild.pants==2.18.0rc3 into a virtual environment at /root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/venvs/2.18.0rc3
2023-10-25 00:00:30,085 ERROR] root: Install failed: Command '['/root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/pex_root/venvs/dd209a3851314c876db14642bafdf970fab77250/d95804f90e6d85181506d9552cdd5b3debfc6d3b/bin/python', '/tmp/tmpv3qnt9wx.pex', 'venv', '--prompt', 'Pants 2.18.0rc3', '--compile', '--pip', '--collisions-ok', '--no-emit-warnings', '--disable-cache', '/root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/venvs/2.18.0rc3']' returned non-zero exit status 1.
More information can be found in the log at: /root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/logs/install.log
Traceback (most recent call last):
  File "/root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/pex_root/venvs/dd209a3851314c876db14642bafdf970fab77250/d95804f90e6d85181506d9552cdd5b3debfc6d3b/pex", line 284, in <module>
    sys.exit(func())
  File "/root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/pex_root/venvs/dd209a3851314c876db14642bafdf970fab77250/d95804f90e6d85181506d9552cdd5b3debfc6d3b/lib/python3.9/site-packages/conscript/main.py", line 105, in main
    return ep.load()()
  File "/root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/pex_root/venvs/dd209a3851314c876db14642bafdf970fab77250/d95804f90e6d85181506d9552cdd5b3debfc6d3b/lib/python3.9/site-packages/scie_pants/install_pants.py", line 197, in main
    install_pants_from_pex(
  File "/root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/pex_root/venvs/dd209a3851314c876db14642bafdf970fab77250/d95804f90e6d85181506d9552cdd5b3debfc6d3b/lib/python3.9/site-packages/scie_pants/install_pants.py", line 117, in install_pants_from_pex
    subprocess.run(
  File "/root/.cache/nce/2b6e146234a4ef2a8946081fc3fbfffe0765b80b690425a49ebe40b47c33445b/cpython-3.9.16+20230507-x86_64-unknown-linux-gnu-install_only.tar.gz/python/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/pex_root/venvs/dd209a3851314c876db14642bafdf970fab77250/d95804f90e6d85181506d9552cdd5b3debfc6d3b/bin/python', '/tmp/tmpv3qnt9wx.pex', 'venv', '--prompt', 'Pants 2.18.0rc3', '--compile', '--pip', '--collisions-ok', '--no-emit-warnings', '--disable-cache', '/root/.cache/nce/68f5608a60df9b97aab453d453817a4ded400d1d8ec7ede7ec14bcac83421a7b/bindings/venvs/2.18.0rc3']' returned non-zero exit status 1.
e
@blue-city-97042 this is a pretty wild guess and some reading is required, but does this describe your image distro?: https://github.com/indygreg/python-build-standalone/issues/173 If so, 
science
, the tool used to build scies, was fixed to not rely on libcrypt via https://github.com/a-scie/lift/pull/38, but it looks like scie-pants never has done similar and is still stuck on the May release of PBS: https://github.com/pantsbuild/scie-pants/blob/4287b6789ba55919697a048f4bfc7682800e04df/package/scie-pants.toml#L17-L29
b
so we do have a libcrypt.so installed on the system. I'm not sure I follow the trail to this being a potential issue. Sorry for being obtuse 😞
e
Ok, not it then. If you didn't have libcrypt.so, then the Python Build Standalone interpreter scie pants ships with would fail. Your error shows a failure in the install phase which is the 1st use of the PBS interpreter; so the interpreter failing to runtime link that lib, plus some error masking, might've explained your issue. The fact scie-pants still hasn't upgraded PBS is still an issue waiting to happen, but it appears to not be your issue.
b
hmmm... I'll keep looking, but reproducible builds are import to get the devs productive.
Thanks for all the help up to this point though.
c
g
hi I am facing this issue with pants 2.20 on a linux machine as well. I can confirm that libcrypt.so is on the system and i am using the latest scie-pants. @blue-city-97042 did you figure out the problem or solution?
5 Views
Open in Slack
PreviousNext
Powered by