Hello I m having a few issues generating a lockfile and gett Pants #general

Hello, I’m having a few issues generating a lockfi...

polite-winter-30234

06/14/2023, 8:40 PM

Hello, I’m having a few issues generating a lockfile and getting a series of unit tests to run. They were running for a point in time then I started getting this issue, any ideas?

Copy code

15:36:41.96 [INFO] Completed: Generate lockfile for pytest
15:36:41.96 [ERROR] 1 Exception encountered:

Engine traceback:
  in `generate-lockfiles` goal
  in Generate Python lockfile

ProcessExecutionFailure: Process 'Generate lockfile for pytest' failed with exit code 1.
stdout:

stderr:
Failed to spawn a job for /Users/david.patrick/.pyenv/versions/3.11.3/bin/python3.11: OSError(8, 'Exec format error') while trying to execute `['/Users/david.patrick/.cache/pants/named_caches/pex_root/venvs/41379f21c5956cff0d35ab1ef014bb72e1fe14dc/ef4cc2dec9377975755d029fef15f12ef315716d.d58dec363b254d6b9c82dd85a86c74d6/bin/python3', '-s', '-c', "import os\nimport sys\n\nfrom pex.common import atomic_directory, safe_open\nfrom pex.interpreter import PythonIdentity\n\n\nencoded_identity = PythonIdentity.get(binary='/Users/david.patrick/.cache/pants/named_caches/pex_root/venvs/41379f21c5956cff0d35ab1ef014bb72e1fe14dc/ef4cc2dec9377975755d029fef15f12ef315716d.d58dec363b254d6b9c82dd85a86c74d6/bin/python3').encode()\nwith atomic_directory('/Users/david.patrick/.cache/pants/named_caches/pex_root/interpreters/46e966945f53ee6afb981b9f833f5cec7ab57c53/96eaad65d98a5d7e3f94b18f9e3dc08deee310c3/6bc5b589e65e042013fc10ed1cc8da335c8ce8de', exclusive=False) as cache_dir:\n    if not cache_dir.is_finalized():\n        with safe_open(\n            os.path.join(cache_dir.work_dir, 'INTERP-INFO'), 'w'\n        ) as fp:\n            fp.write(encoded_identity)\n"]`



Use `--keep-sandboxes=on_failure` to preserve the process chroot for inspection.

careful-address-89803

06/14/2023, 8:54 PM

'Exec format error' is sometimes a platform problem. Can you tell us more about what OS you're working on? If it's WSL, is your python and sources on the Windows side or the Linux side?

polite-winter-30234

06/14/2023, 10:09 PM

I’m currently running this locally on an M1 Mac using the scie binaries.

careful-address-89803

06/14/2023, 11:19 PM

Ah, I'm less familiar with that platform. I'd guess it might be an arm64 vs x86_64 thing, like somehow the python3.11.3 is for the wrong platform. Have you tried blowing away the cache? Some of the maintainers have M1 Macs, they'd probably know more.

polite-winter-30234

06/14/2023, 11:30 PM

Yeah I tried that as well, I feel like I’m out of ideas at this point. Thanks for the help!

broad-processor-92400

06/15/2023, 12:31 AM

(Reference point: I've been running scie-pants fine on an M1 mac, although with python3.9) I think we can narrow down file format errors: What does

arch

say in the terminal you're running pants in? And

file /Users/david.patrick/.cache/pants/named_caches/pex_root/venvs/41379f21c5956cff0d35ab1ef014bb72e1fe14dc/ef4cc2dec9377975755d029fef15f12ef315716d.d58dec363b254d6b9c82dd85a86c74d6/bin/python3

polite-winter-30234

06/15/2023, 1:53 AM

Copy code

david.patrick@name ~ % arch
arm64

broad-processor-92400

06/15/2023, 12:38 PM

What about the

file …

one too? To reveal the format of the Python executable being used

polite-winter-30234

06/15/2023, 12:59 PM

david.patrick@name ~ % file /Users/david.patrick/.cache/pants/named_caches/pex_root/venvs/41379f21c5956cff0d35ab1ef014bb72e1fe14dc/ef4cc2dec9377975755d029fef15f12ef315716d.d58dec363b254d6b9c82dd85a86c74d6/bin/python3

`/Users/david.patrick/.cache/pants/named_caches/pex_root/venvs/41379f21c5956cff0d35ab1ef014bb72e1fe14dc/ef4cc2dec9377975755d029fef15f12ef315716d.d58dec363b254d6b9c82dd85a86c74d6/bin/python3: cannot open

/Users/david.patrick/.cache/pants/named_caches/pex_root/venvs/41379f21c5956cff0d35ab1ef014bb72e1fe14dc/ef4cc2dec9377975755d029fef15f12ef315716d.d58dec363b254d6b9c82dd85a86c74d6/bin/python3' (No such file or directory)

polite-winter-30234

06/15/2023, 1:18 PM

While that file doesn’t exist, I investigated a similar directory and found the following

polite-winter-30234

06/15/2023, 1:19 PM

david.patrick@name ef4cc2dec9377975755d029fef15f12ef315716d.15c7e09731a1404ab76a7e59fefc4a21 % pwd

/Users/david.patrick/.cache/pants/named_caches/pex_root/venvs/41379f21c5956cff0d35ab1ef014bb72e1fe14dc/ef4cc2dec9377975755d029fef15f12ef315716d.15c7e09731a1404ab76a7e59fefc4a21

david.patrick@name ef4cc2dec9377975755d029fef15f12ef315716d.15c7e09731a1404ab76a7e59fefc4a21 % file bin/python3

bin/python3: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit dynamically linked shared library x86_64] [arm64]

bin/python3 (for architecture x86_64):	Mach-O 64-bit dynamically linked shared library x86_64

bin/python3 (for architecture arm64):	Mach-O 64-bit dynamically linked shared library arm64

david.patrick@AMAC0V4KH2272 ef4cc2dec9377975755d029fef15f12ef315716d.15c7e09731a1404ab76a7e59fefc4a21 %

polite-winter-30234

06/15/2023, 1:19 PM

david.patrick@name bin % pwd

/Users/david.patrick/.cache/pants/named_caches/pex_root/venvs/41379f21c5956cff0d35ab1ef014bb72e1fe14dc/ef4cc2dec9377975755d029fef15f12ef315716d.957ee579951e45608da2b716503b9f98/bin

david.patrick@name bin % file python3

python3: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit dynamically linked shared library x86_64] [arm64]

python3 (for architecture x86_64):	Mach-O 64-bit dynamically linked shared library x86_64

python3 (for architecture arm64):	Mach-O 64-bit dynamically linked shared library arm64

polite-winter-30234

06/15/2023, 1:20 PM

Do I need to regenerate that binary it’s attempting to look for? Or potentially change a path?

polite-winter-30234

06/16/2023, 5:10 PM

So with regards to my last question, do I need to generate that pex_root/venv that doesn’t exist or is there some permission issue that’s not allowing it to be created?

broad-processor-92400

06/16/2023, 10:03 PM

I’m not sure, sorry!

careful-address-89803

06/17/2023, 6:14 PM

I think this error is before the binary locator script (the value of the -c arg) is running, it's happening attempting to invoke the python3 from the pex. It looks like that python is a multiarch binary (something that's new to me). I would check up the invocation chain to see if something has the wrong arch. maybe

/Users/david.patrick/.pyenv/versions/3.11.3/bin/python3.11

and the scie itself? You can also use the

--keep-sandboxes=on_failure

to try to step through things manually and inspect the whole chain of executables. Also what version of Pants is your repo using? (I think the pex_root/venv doesn't exist because the pexes are somewhat ephemeral. They should be generated automatically)

polite-winter-30234

06/19/2023, 12:56 PM

Currently I’ve been testing this on 2.15

careful-address-89803

06/19/2023, 6:14 PM

I think the architecture of those binaries would be where I would look next. I would try to confirm that they're all compatible

polite-winter-30234

06/19/2023, 6:15 PM

Actually I have some updates, after playing with the pyenv versions it looks like my version of 3.11.3 was corrupted. After reinstalling it everything works nominally, thanks for the assistance everyone!!

👍 1

Open in Slack

Previous Next