I'm restarting an attempt at adding pants to an existing repo, taking a more incremental approach this time. So I started from scratch by adding my BUILD files etc. but I'm running into a basic platform-specific issue getting the correct pandas wheel to be used in my pex files. I don't really remember running into this last time I started setting up pants and I can't figure out what I'm doing differently. Every time I run a test which has a pandas dependency I get the following error:

ImportError: dlopen(/Users/zach/.cache/pants/named_caches/pex_root/venvs/s/5e00d369/venv/lib/python3.8/site-packages/pandas/_libs/interval.cpython-38-darwin.so, 0x0002): tried: '/Users/zach/.cache/pants/named_caches/pex_root/venvs/s/5e00d369/venv/lib/python3.8/site-packages/pandas/_libs/interval.cpython-38-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/zach/.cache/pants/named_caches/pex_root/venvs/s/5e00d369/venv/lib/python3.8/site-packages/pandas/_libs/interval.cpython-38-darwin.so' (no such file), '/Users/zach/.cache/pants/named_caches/pex_root/venvs/s/5e00d369/venv/lib/python3.8/site-packages/pandas/_libs/interval.cpython-38-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/Users/zach/.cache/pants/named_caches/pex_root/venvs/f0f601382409fe9e35378773ff1ec5df7ceeb1a9/b21965ad1b3cb7aee0410c57d55dd2fd6abe1937/lib/python3.8/site-packages/pandas/_libs/interval.cpython-38-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/zach/.cache/pants/named_caches/pex_root/venvs/f0f601382409fe9e35378773ff1ec5df7ceeb1a9/b21965ad1b3cb7aee0410c57d55dd2fd6abe1937/lib/python3.8/site-packages/pandas/_libs/interval.cpython-38-darwin.so' (no such file), '/Users/zach/.cache/pants/named_caches/pex_root/venvs/f0f601382409fe9e35378773ff1ec5df7ceeb1a9/b21965ad1b3cb7aee0410c57d55dd2fd6abe1937/lib/python3.8/site-packages/pandas/_libs/interval.cpython-38-darwin.so' (mach-o file, but is an incompatible architecture (have 'x86_64', need 'arm64'))

I tried pointing to a local environment to try to force pants to pick up the

arm64

version of the wheel, which the BUILD file I'm testing points to:

local_environment(
    name="local_arm64",
    compatible_platforms=["macos_arm64"]
)

this fails with the same issue. I've also tried pointing to a

x86_64

docker container environment, but the performance is incredibly slow - unacceptably so for local testing (upwards of 10 minutes to build the pex file to run a test). In my previous stab at this I was able to run tests without pointing to any environment and it "just worked". Anyone have any tips for running tests that rely on Pandas from a Mac M1? Pants 2.15.0 if that's important

So virtualenvs in the Python world have a link back to a base interpreter. If you were to update that base interpreter (say re-install what was once x86_64 gets re-installed as arm64), you'd have an issue. So - you know your machine mucking history - did you do anything like that? You could poke around on the command line and actually look at the

/Users/zach/.cache/pants/named_caches/pex_root/venvs/f0f601382409fe9e35378773ff1ec5df7ceeb1a9/b21965ad1b3cb7aee0410c57d55dd2fd6abe1937/bin

venv bin dir and see where its Python is coming from on the system. More vaguely; I thought it was "well known" M1 machines ~only work with Python 3.9+ for arm mode. I am not a Mac user so I'm just parroting there and handwaving.

I've been managing my python interpreter version through pyenv for some time now. I realized that I had been doing the 3.8 builds through a docker environment, so then I blasted my pants cache and adjusted the interpreter constraints to 3.9.5-3.10 and set my pyenv global interpreter to 3.9.5 - re-ran the test and got the same problem. Double checked the cache venv bin like you mentioned (good idea), but it's pointing to my pyenv 3.9 interpreter.

Ok. I'll bow out then with no Mac and not enough details to debug.

I appreciate it though, it gives me some more ideas to stab at

What version of pandas? What's the section of the (pants/pex) lockfile that mentions pandas? Can you reproduce the problem in isolation (e.g. a directory with just

pants.toml

BUILD

and an

import pandas

in a python file)? Reference point: We use pants 2.15 + pandas 1.4.2 natively on macOS

Ah I think that was it! I had

pandas<2.0.0

, and it looks like it was reaching all the way back to

pandas==1.3.2

. I changed it to

pandas<2.0.0,>=1.4.2

and I am now successfully onto a new and more exciting error (exciting being something I know how to fix). Thank you!