hi again - one more question (maybe more to do with pex) But i'm following the instructions for optimizing docker images: https://www.pantsbuild.org/blog/2022/08/02/optimizing-python-docker-deploys-using-pants And when I used the

packed

layout on both the pex_binaries - I get t his error:

pex.dist_metadata.MetadataError: Failed to determine project name and version for distribution at /root/.pex/unzipped_pexes/bd0f443c70c12f2ac5b4b95bfa0422e9040f3178/.deps/numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

But using

zipapp

layout it works just fine

Sorry for the trouble. Can you provide a bit more context to the error? What command is this running?

sorry - went off to get dinner prepped

❯ pants package src:remitsrv
11:26:58.71 [INFO] Initializing scheduler...
11:26:58.74 [INFO] Initializing Nailgun pool for 20 processes...
11:27:01.77 [INFO] Scheduler initialized.
11:27:02.54 [WARN] podman found. Podman support is experimental.
11:27:26.76 [INFO] Completed: Building dockerfile_parser.pex from <resource://pants.backend.docker.subsystems/dockerfile.lock>
11:28:02.04 [INFO] Completed: Building local_dists.pex
11:28:02.21 [INFO] Canceled: Building 102 requirements for src/binary-deps.pex from the 3rdparty/python/default.lock resolve: Adyen, Flask-Cors, Flask<2.3.0, MarkupSafe, SQLAlchemy==1.4.29, Werkzeug>=0.15, alembic>=1.13.1, altair... (1626 characters truncated)
11:28:09.94 [INFO] Completed: Building 102 requirements for src/binary-deps.pex from the 3rdparty/python/default.lock resolve: Adyen, Flask-Cors, Flask<2.3.0, MarkupSafe, SQLAlchemy==1.4.29, Werkzeug>=0.15, alembic>=1.13.1, altair... (1626 characters truncated)
11:28:11.22 [INFO] Completed: Building src/binary-srcs.pex
11:28:14.96 [INFO] Completed: Building docker image img-srcs:srcs
11:28:25.89 [INFO] Completed: Building docker image img-deps:deps
11:28:25.89 [ERROR] 1 Exception encountered:

Engine traceback:
  in `package` goal

ProcessExecutionFailure: Process 'Building docker image img-deps:deps' failed with exit code 1.
stdout:
STEP 1/3: FROM python:3.8-slim
STEP 2/3: COPY src/binary-deps.pex /binary-deps.pex
--> Using cache bcd46547a8abaf46f95b97de4029b53f604f6ff756089bff44e1f4aa1265e397
--> bcd46547a8ab
STEP 3/3: RUN PEX_TOOLS=1 /usr/local/bin/python3.8 /binary-deps.pex venv --scope=deps --compile /app
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/root/.pex/unzipped_pexes/bd0f443c70c12f2ac5b4b95bfa0422e9040f3178/__main__.py", line 106, in <module>
    bootstrap_pex(__entry_point__, execute=__execute__, venv_dir=__venv_dir__)
  File "/root/.pex/unzipped_pexes/bd0f443c70c12f2ac5b4b95bfa0422e9040f3178/.bootstrap/pex/pex_bootstrapper.py", line 624, in bootstrap_pex
    maybe_reexec_pex(interpreter_test=interpreter_test)
  File "/root/.pex/unzipped_pexes/bd0f443c70c12f2ac5b4b95bfa0422e9040f3178/.bootstrap/pex/pex_bootstrapper.py", line 380, in maybe_reexec_pex
    target = find_compatible_interpreter(interpreter_test=interpreter_test)
  File "/root/.pex/unzipped_pexes/bd0f443c70c12f2ac5b4b95bfa0422e9040f3178/.bootstrap/pex/pex_bootstrapper.py", line 305, in find_compatible_interpreter
    target = _select_path_interpreter(
  File "/root/.pex/unzipped_pexes/bd0f443c70c12f2ac5b4b95bfa0422e9040f3178/.bootstrap/pex/pex_bootstrapper.py", line 217, in _select_path_interpreter
    for interpreter in candidate_interpreters_iter:
  File "/root/.pex/unzipped_pexes/bd0f443c70c12f2ac5b4b95bfa0422e9040f3178/.bootstrap/pex/pex_bootstrapper.py", line 173, in iter_compatible_interpreters
    valid_or_error = _valid_interpreter(interpreter)
  File "/root/.pex/unzipped_pexes/bd0f443c70c12f2ac5b4b95bfa0422e9040f3178/.bootstrap/pex/pex_bootstrapper.py", line 160, in _valid_interpreter
    return interpreter_test.test_resolve(interp) if interpreter_test else True
  File "/root/.pex/unzipped_pexes/bd0f443c70c12f2ac5b4b95bfa0422e9040f3178/.bootstrap/pex/pex_bootstrapper.py", line 72, in test_resolve
    pex_environment.resolve()
  File "/root/.pex/unzipped_pexes/bd0f443c70c12f2ac5b4b95bfa0422e9040f3178/.bootstrap/pex/environment.py", line 555, in resolve
    for fingerprinted_distribution in self.resolve_dists(
  File "/root/.pex/unzipped_pexes/bd0f443c70c12f2ac5b4b95bfa0422e9040f3178/.bootstrap/pex/environment.py", line 575, in resolve_dists
    self._update_candidate_distributions(
  File "/root/.pex/unzipped_pexes/bd0f443c70c12f2ac5b4b95bfa0422e9040f3178/.bootstrap/pex/environment.py", line 319, in _update_candidate_distributions
    for fingerprinted_dist in distribution_iter:
  File "/root/.pex/unzipped_pexes/bd0f443c70c12f2ac5b4b95bfa0422e9040f3178/.bootstrap/pex/environment.py", line 313, in iter_distributions
    distribution=Distribution.load(dist_path),
  File "/root/.pex/unzipped_pexes/bd0f443c70c12f2ac5b4b95bfa0422e9040f3178/.bootstrap/pex/dist_metadata.py", line 885, in load
    return cls(location=location, metadata=DistMetadata.load(location))
  File "/root/.pex/unzipped_pexes/bd0f443c70c12f2ac5b4b95bfa0422e9040f3178/.bootstrap/pex/dist_metadata.py", line 807, in load
    raise MetadataError(
pex.dist_metadata.MetadataError: Failed to determine project name and version for distribution at /root/.pex/unzipped_pexes/bd0f443c70c12f2ac5b4b95bfa0422e9040f3178/.deps/numpy-1.24.4-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.

stderr:
Error: building at STEP "RUN PEX_TOOLS=1 /usr/local/bin/python3.8 /binary-deps.pex venv --scope=deps --compile /app": while running runtime: exit status 1




Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.

11:28:25.94 [INFO] Wrote log to .pants.d/pants_run_2024_10_02_11_27_01_820_4d375a727e1d4420832b6585da8049a8.json

So this is just extracting the dependencies from the pex into the intermediate docker container.

Huh, weird. I'm not sure what the problem is here. You might be well served by stepping out of the pants world and iterating with the PEX CLI and docker directly, to get a reduced reproducer that may reveal insight