this has gotta be some small platform-specific setting / install that I'm missing but I haven't been able to find anything in the docs or online. when I attempt to build a docker image using the

docker_image

target and a simple Dockerfile on a MAC M1, it looks like the

dockerfile

module used by pants in my setup was compiled for

x86_64

but I need the

arm64

one. this feels like it might have something to do with this blurb on Mac interpreters. but changing the

interpreter_search_paths

property doesn't seem to have any affect, I suspect because that is probably referring to setting the python interpreter for the python backend tools and doesn't affect the docker backend tools. how do I get pants to create and use the

arm64

dockerfile module? here's the actual error:

Traceback (most recent call last):
  File "/private/var/folders/0k/c0crrpz142n_h62r7nnmpnsr0000gn/T/pants-sandbox-IbPeOY/.cache/pex_root/venvs/3d84392e96bb69d7972ace4055cb0144a30b9e8e/a58c33d44cdb8a7bd8160aa976f1a158437bee6a/pex", line 243, in <module>
    runpy.run_module(module_name, run_name="__main__", alter_sys=True)
  File "/Users/zach/.pyenv/versions/3.7.16/lib/python3.7/runpy.py", line 205, in run_module
    return _run_module_code(code, init_globals, run_name, mod_spec)
  File "/Users/zach/.pyenv/versions/3.7.16/lib/python3.7/runpy.py", line 96, in _run_module_code
    mod_name, mod_spec, pkg_name, script_name)
  File "/Users/zach/.pyenv/versions/3.7.16/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/zach/.cache/pants/named_caches/pex_root/venvs/s/b9c7c433/venv/lib/python3.7/site-packages/__pants_df_parser.py", line 202, in <module>
    print(json.dumps([asdict(info) for info in main(*sys.argv[1:])]))
  File "/Users/zach/.cache/pants/named_caches/pex_root/venvs/s/b9c7c433/venv/lib/python3.7/site-packages/__pants_df_parser.py", line 202, in <listcomp>
    print(json.dumps([asdict(info) for info in main(*sys.argv[1:])]))
  File "/Users/zach/.cache/pants/named_caches/pex_root/venvs/s/b9c7c433/venv/lib/python3.7/site-packages/__pants_df_parser.py", line 61, in main
    from dockerfile import Command, parse_file, parse_string  # pants: no-infer-dep
ImportError: dlopen(/Users/zach/.cache/pants/named_caches/pex_root/venvs/s/b9c7c433/venv/lib/python3.7/site-packages/dockerfile.abi3.so, 0x0002): tried: '/Users/zach/.cache/pants/named_caches/pex_root/venvs/s/b9c7c433/venv/lib/python3.7/site-packages/dockerfile.abi3.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/zach/.cache/pants/named_caches/pex_root/venvs/s/b9c7c433/venv/lib/python3.7/site-packages/dockerfile.abi3.so' (no such file), '/Users/zach/.cache/pants/named_caches/pex_root/venvs/s/b9c7c433/venv/lib/python3.7/site-packages/dockerfile.abi3.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/Users/zach/.cache/pants/named_caches/pex_root/venvs/3d84392e96bb69d7972ace4055cb0144a30b9e8e/a58c33d44cdb8a7bd8160aa976f1a158437bee6a/lib/python3.7/site-packages/dockerfile.abi3.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/zach/.cache/pants/named_caches/pex_root/venvs/3d84392e96bb69d7972ace4055cb0144a30b9e8e/a58c33d44cdb8a7bd8160aa976f1a158437bee6a/lib/python3.7/site-packages/dockerfile.abi3.so' (no such file), '/Users/zach/.cache/pants/named_caches/pex_root/venvs/3d84392e96bb69d7972ace4055cb0144a30b9e8e/a58c33d44cdb8a7bd8160aa976f1a158437bee6a/lib/python3.7/site-packages/dockerfile.abi3.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))

One way to build a docker image for a particular platform arch is using

platform

arg inside your dockerfile. https://docs.docker.com/build/building/multi-platform/ You can pass it as a build arg using

extra_build_args

as mentioned here https://www.pantsbuild.org/docs/reference-docker_image#codeextra_build_argscode

so I had tried using a platform-specific build with the platform arg hardcoded in my dockerfile and I got the same error. I can actually build the image just fine without pants just using raw docker and no platform arguments. from the error I'm getting it doesn't look like it's even getting into building the image, it seems to be failing to load the python files that are used to parse the dockerfile. it looks to me like pants is building a virtual environment with which to build the docker image but that the

dockerfile

module getting installed into that environment by pants doesn't work on arm64.

do you use some kind of terminal emulator?

I've never set one up

How are you maintaining multiple versions of python? pyenv?

yeah pyenv

I had this issue when running pants from source but never otherwise. You might have to set-up this

search_path

as mentioned here https://www.pantsbuild.org/docs/python-backend

search_path = ["<PYENV>"]

I've been trying it with

[python-bootstrap]
search_path = [
  "<PYENV>"
]

and also

[python-bootstrap]
search_path = [
  "<PYENV>",
  "/usr/local/bin"
]

and neither seem to help. i'm starting to go down this rabbithole - https://chat.pantsbuild.org/t/9728005/hey-i-m-having-some-python-architecture-issues-when-running-

but did you try to start from scratch? Like delete all the existing cache associated with pants?

I tried deleting everything at

/Users/zach/.cache/pants/...

yesterday (maybe that's not all the caches I need to delete) and it didn't fix anything, so I was going to try deleting those again and also re-bootstrapping pants (I haven't re-installed / re-bootstrapped pants yet)

Wow this is can of worms. I don't remember this being so painful. Try passing them in [docker].env_vars inside pants.toml https://www.pantsbuild.org/docs/docker

yeah remember when computers were going to make our lives easier?