One issue I've come across is being stuck with locally built wheels when I try to build pex packages for a docker container. So I followed the guidance to have pex build for a specific platform, using

extra_build_args = [
    "--platform=linux-x86_64-cp-311-m",
]

and that seems to be doing the right thing, but it can't download all the packages needed to build the correct wheels.

14:34:10.89 [ERROR] 1 Exception encountered:

Engine traceback:
  in `package` goal

ProcessExecutionFailure: Process 'Building 10 requirements for backend.telemetry_adapter/telemetry_adapter_bin.pex from the backend/pants.lock resolve: confluent-kafka<3.0.0,>=2.0.2, pendulum<3.0.0,>=2.1.2, psycopg2-binary<3.0.0,>=2.9.5, pydantic-settings<3.0.0,>=2.2.1, pydantic<3.0.0,>=2.7.4, requests<3.0.0,>=2.28.2, sqlalchemy-json<0.6.0,>=0.5.0, sqlalchemy<3.0.0,>=2.0.1, structlog<25.0.0,>=24.2.0, types-requests<3.0.0,>=2.28.11.15' failed with exit code 1.
stdout:

stderr:
/home/luke/.cache/pants/named_caches/pex_root/installed_wheels/8d43c1ed16877f3d2ed39e5ef3e92aa8885d4427cd7d9ce62683c922a5c0cd07/pex-2.3.1-py2.py3-none-any.whl/pex/atomic_directory.py:291: PEXWarning: [pid:687629, tid:131885720118848, cwd:/tmp/pants-sandbox-Qnl2da]: After obtaining an exclusive lock on /home/luke/.cache/pants/named_caches/pex_root/downloads/.7f01846810177d829c7692f1f5ada8096762d9172af1b1a28d4ab5b77c923c1c.atomic_directory.lck, failed to establish a work directory at /home/luke/.cache/pants/named_caches/pex_root/downloads/7f01846810177d829c7692f1f5ada8096762d9172af1b1a28d4ab5b77c923c1c.lck.work due to: [Errno 17] File exists: '/home/luke/.cache/pants/named_caches/pex_root/downloads/7f01846810177d829c7692f1f5ada8096762d9172af1b1a28d4ab5b77c923c1c.lck.work'
  pex_warnings.warn(
/home/luke/.cache/pants/named_caches/pex_root/installed_wheels/8d43c1ed16877f3d2ed39e5ef3e92aa8885d4427cd7d9ce62683c922a5c0cd07/pex-2.3.1-py2.py3-none-any.whl/pex/atomic_directory.py:302: PEXWarning: [pid:687629, tid:131885720118848, cwd:/tmp/pants-sandbox-Qnl2da]: Continuing to forcibly re-create the work directory at /home/luke/.cache/pants/named_caches/pex_root/downloads/7f01846810177d829c7692f1f5ada8096762d9172af1b1a28d4ab5b77c923c1c.lck.work.
  pex_warnings.warn(
/home/luke/.cache/pants/named_caches/pex_root/installed_wheels/8d43c1ed16877f3d2ed39e5ef3e92aa8885d4427cd7d9ce62683c922a5c0cd07/pex-2.3.1-py2.py3-none-any.whl/pex/atomic_directory.py:291: PEXWarning: [pid:687629, tid:131885728511552, cwd:/tmp/pants-sandbox-Qnl2da]: After obtaining an exclusive lock on /home/luke/.cache/pants/named_caches/pex_root/downloads/.26ca695eeee5f9f1aeeb211ffc12f10bcb6f71e2989988fda61dabd65db878d4.atomic_directory.lck, failed to establish a work directory at /home/luke/.cache/pants/named_caches/pex_root/downloads/26ca695eeee5f9f1aeeb211ffc12f10bcb6f71e2989988fda61dabd65db878d4.lck.work due to: [Errno 17] File exists: '/home/luke/.cache/pants/named_caches/pex_root/downloads/26ca695eeee5f9f1aeeb211ffc12f10bcb6f71e2989988fda61dabd65db878d4.lck.work'
  pex_warnings.warn(
/home/luke/.cache/pants/named_caches/pex_root/installed_wheels/8d43c1ed16877f3d2ed39e5ef3e92aa8885d4427cd7d9ce62683c922a5c0cd07/pex-2.3.1-py2.py3-none-any.whl/pex/atomic_directory.py:302: PEXWarning: [pid:687629, tid:131885728511552, cwd:/tmp/pants-sandbox-Qnl2da]: Continuing to forcibly re-create the work directory at /home/luke/.cache/pants/named_caches/pex_root/downloads/26ca695eeee5f9f1aeeb211ffc12f10bcb6f71e2989988fda61dabd65db878d4.lck.work.
  pex_warnings.warn(
There were 2 errors downloading required artifacts:
1. psycopg2-binary 2.9.9 from <https://files.pythonhosted.org/packages/fc/07/e720e53bfab016ebcc34241695ccc06a9e3d91ba19b40ca81317afbdc440/psycopg2-binary-2.9.9.tar.gz>
    hint: See above for details.
2. pydantic-core 2.20.1 from <https://files.pythonhosted.org/packages/12/e3/0d5ad91211dba310f7ded335f4dad871172b9cc9ce204f5a56d76ccd6247/pydantic_core-2.20.1.tar.gz>
    hint: See above for details.



Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.

I've tried manually deleting those locks, but it doesn't make a difference. Its always these particular packages that have issues, though I'm not sure why. Any thoughts?

try searching the slack history for sandbox, pex verbosity and pip log.. this comes up semi-frequently. One such hit: https://pantsbuild.slack.com/archives/C046T6T9U/p1703013909496829?thread_ts=1703012339.391419&cid=C046T6T9U