#20765 `ImportError: ... incompatible architecture` on arm64 macOS for Pants 2.19.2, 2.20.0rc3, 2.21.0.dev5 Issue created by huonw Describe the bug Running any of the versions published recently on arm64 macOS hits an error attempting to import

psutil

. For instance `PANTS_VERSION=2.19.2 pants`:

ImportError: dlopen(/Users/huon/Library/Caches/nce/f46d2c12132bad9c27e0dd509186c190371abfe1add7e5ca42245466f35bea81/bindings/venvs/2.19.2/lib/python3.9/site-packages/psutil/_psutil_osx.cpython-39-darwin.so, 0x0002): tried: '/Users/huon/Library/Caches/nce/f46d2c12132bad9c27e0dd509186c190371abfe1add7e5ca42245466f35bea81/bindings/venvs/2.19.2/lib/python3.9/site-packages/psutil/_psutil_osx.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/huon/Library/Caches/nce/f46d2c12132bad9c27e0dd509186c190371abfe1add7e5ca42245466f35bea81/bindings/venvs/2.19.2/lib/python3.9/site-packages/psutil/_psutil_osx.cpython-39-darwin.so' (no such file), '/Users/huon/Library/Caches/nce/f46d2c12132bad9c27e0dd509186c190371abfe1add7e5ca42245466f35bea81/bindings/venvs/2.19.2/lib/python3.9/site-packages/psutil/_psutil_osx.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64'))

Workaround: use an earlier release. NB. for 2.19.2, 2.19.2rc0 has essentially the same code (only a documentation difference).

$ PANTS_VERSION=2.19.2 pants                                                                      
Bootstrapping Pants 2.19.2
Installing pantsbuild.pants==2.19.2 into a virtual environment at /Users/huon/Library/Caches/nce/f46d2c12132bad9c27e0dd509186c190371abfe1add7e5ca42245466f35bea81/bindings/venvs/2.19.2
New virtual environment successfully created at /Users/huon/Library/Caches/nce/f46d2c12132bad9c27e0dd509186c190371abfe1add7e5ca42245466f35bea81/bindings/venvs/2.19.2.
Traceback (most recent call last):
  File "/Users/huon/Library/Caches/nce/f46d2c12132bad9c27e0dd509186c190371abfe1add7e5ca42245466f35bea81/bindings/venvs/2.19.2/bin/pants", line 8, in <module>
    entry_point = importlib.import_module(modname)
  File "/Users/huon/Library/Caches/nce/bf0cd90204a2cc6da48cae1e4b32f48c9f7031fbe1238c5972104ccb0155d368/cpython-3.9.18+20240107-aarch64-apple-darwin-install_only.tar.gz/python/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 850, in exec_module
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "/Users/huon/Library/Caches/nce/f46d2c12132bad9c27e0dd509186c190371abfe1add7e5ca42245466f35bea81/bindings/venvs/2.19.2/lib/python3.9/site-packages/pants/bin/pants_loader.py", line 18, in <module>
    from pants.bin.pants_runner import PantsRunner
  File "/Users/huon/Library/Caches/nce/f46d2c12132bad9c27e0dd509186c190371abfe1add7e5ca42245466f35bea81/bindings/venvs/2.19.2/lib/python3.9/site-packages/pants/bin/pants_runner.py", line 14, in <module>
    from pants.base.exception_sink import ExceptionSink
  File "/Users/huon/Library/Caches/nce/f46d2c12132bad9c27e0dd509186c190371abfe1add7e5ca42245466f35bea81/bindings/venvs/2.19.2/lib/python3.9/site-packages/pants/base/exception_sink.py", line 16, in <module>
    import psutil
  File "/Users/huon/Library/Caches/nce/f46d2c12132bad9c27e0dd509186c190371abfe1add7e5ca42245466f35bea81/bindings/venvs/2.19.2/lib/python3.9/site-packages/psutil/__init__.py", line 123, in <module>
    from . import _psosx as _psplatform
  File "/Users/huon/Library/Caches/nce/f46d2c12132bad9c27e0dd509186c190371abfe1add7e5ca42245466f35bea81/bindings/venvs/2.19.2/lib/python3.9/site-packages/psutil/_psosx.py", line 14, in <module>
    from . import _psutil_osx as cext
ImportError: dlopen(/Users/huon/Library/Caches/nce/f46d2c12132bad9c27e0dd509186c190371abfe1add7e5ca42245466f35bea81/bindings/venvs/2.19.2/lib/python3.9/site-packages/psutil/_psutil_osx.cpython-39-darwin.so, 0x0002): tried: '/Users/huon/Library/Caches/nce/f46d2c12132bad9c27e0dd509186c190371abfe1add7e5ca42245466f35bea81/bindings/venvs/2.19.2/lib/python3.9/site-packages/psutil/_psutil_osx.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/huon/Library/Caches/nce/f46d2c12132bad9c27e0dd509186c190371abfe1add7e5ca42245466f35bea81/bindings/venvs/2.19.2/lib/python3.9/site-packages/psutil/_psutil_osx.cpython-39-darwin.so' (no such file), '/Users/huon/Library/Caches/nce/f46d2c12132bad9c27e0dd509186c190371abfe1add7e5ca42245466f35bea81/bindings/venvs/2.19.2/lib/python3.9/site-packages/psutil/_psutil_osx.cpython-39-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64e' or 'arm64'))

Checking the release PEXes confirms that they contain x86-64 objects, for instance, for 2.19.2:

cd $(mktemp -d)

curl -LO <https://github.com/pantsbuild/pants/releases/download/release_2.19.2/pants.2.19.2-cp39-darwin_arm64.pex>

unzip -q pants.2.19.2-cp39-darwin_arm64.pex

file  .deps/psutil-5.9.0-cp39-cp39-macosx_11_0_arm64.whl/psutil/_psutil_{osx,posix}.cpython-39-darwin.so

.deps/psutil-5.9.0-cp39-cp39-macosx_11_0_arm64.whl/psutil/_psutil_osx.cpython-39-darwin.so:   Mach-O 64-bit bundle x86_64
.deps/psutil-5.9.0-cp39-cp39-macosx_11_0_arm64.whl/psutil/_psutil_posix.cpython-39-darwin.so: Mach-O 64-bit bundle x86_64

NB. this is likely related to #20759 and potentially #20756. Pants version • 2.19.2 • 2.20.0rc3 • 2.21.0.dev5 OS macOS Additional info We'll presumably want to do a fast-follow 2.19.3 stable release once we fix this, and this blocks 2.20.0 stable too. pantsbuild/pants