https://pantsbuild.org/ logo
#pex
Title
# pex
h

hundreds-father-404

02/23/2021, 12:41 AM
@happy-kitchen-89482 @witty-crayon-22786 @fast-nail-55400 can one of you please run this:
Copy code
$ wget <https://github.com/pantsbuild/pex/releases/download/v2.1.31/pex>
$ which -a python2.7
# Copy the path to your system Python 2.7

$ <python2.7> ./pex --venv --seed
h

happy-kitchen-89482

02/23/2021, 12:42 AM
Done
that works I think
I get a repl
f

fast-nail-55400

02/23/2021, 12:43 AM
Copy code
tdyas@paradox:~$ /usr/bin/python2.7 ./pex --venv --seed
Python 2.7.16 (default, Dec 21 2020, 23:00:36)
[GCC Apple LLVM 12.0.0 (clang-1200.0.30.4) [+internal-os, ptrauth-isa=sign+stri on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>>
h

hundreds-father-404

02/23/2021, 12:44 AM
Hm, interesting. I get a 100 return code, which I have no idea what that means
Copy code
pex.executor.NonZeroExit: received exit code 100 during execution of `['/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python', '-s', '-E', '/private/var/folders/hm/qjjq4w3n0fsb07kp5bxbn8rw0000gn/T/tmpy3fBiI', '--no-pip', '--no-setuptools', '--no-wheel', '/Users/eric/.pex/venvs/d747c0f95e20fa8c832124357460aedcfa97f5dc/800ae8ef047bd783543e8282c22bfdbee7b7fca8.54c57459721a4b908085a315d20233b8']` while trying to execute `['/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python', '-s', '-E', '/private/var/folders/hm/qjjq4w3n0fsb07kp5bxbn8rw0000gn/T/tmpy3fBiI', '--no-pip', '--no-setuptools', '--no-wheel', '/Users/eric/.pex/venvs/d747c0f95e20fa8c832124357460aedcfa97f5dc/800ae8ef047bd783543e8282c22bfdbee7b7fca8.54c57459721a4b908085a315d20233b8']`
Thank you both! I wonder if this is a weird mac Silicon thing..
e

enough-analyst-54434

02/23/2021, 12:44 AM
@hundreds-father-404 you were getting 100s from Python 2.7 last week too right?
h

hundreds-father-404

02/23/2021, 12:45 AM
Precisely, this is the minimal repro I've come up with so far. I can't reproduce using Pyenv 3.9
e

enough-analyst-54434

02/23/2021, 12:45 AM
So only for --venv with --seed?
h

hundreds-father-404

02/23/2021, 12:45 AM
Yes,
--venv
is fine and
--unzip --seed
is fine
e

enough-analyst-54434

02/23/2021, 12:46 AM
But if you run a --venv PEX file presumably it then fails.
Which would mean its venv creation that fails.
So, if you remember back from November / December (I sure wouldn't) - venv for Python 2.7 uses virtualenv: https://github.com/pantsbuild/pex/blob/master/pex/tools/commands/README.venv.md
👀 1
h

hundreds-father-404

02/23/2021, 12:47 AM
Ah, yep, my bad.
/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python ./pex --venv
fails for me, whereas
/System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python ./pex --venv -o test.pex
does succeed until running
./test.pex
e

enough-analyst-54434

02/23/2021, 12:47 AM
I wonder if you could eliminate the Pex variable and see if your system python 2.7 works with virtualenv of any version.
👍 1
h

hundreds-father-404

02/23/2021, 12:48 AM
Great, will try! I just tried to pyenv install 2.7 and it failed due to arch, so I wouldn't be surprised virtualenv doesn't work on mac silicon either for 2.7
lol system python 2.7 doesn't have pip in order to install virtualenv I can keep investigating @enough-analyst-54434 if you'd like, but I'm comfortable enough classifying this as likely a mac Silicon edge case and ignoring. This works on non-Silicon macOS, and it works on Py3
e

enough-analyst-54434

02/23/2021, 12:52 AM
No no no.
Yeah - you don't want to install virtualenv that way anyhow.
You just do what the README over in Pex does
Curl it down and run it.
👍 1
h

hundreds-father-404

02/23/2021, 12:58 AM
Copy code
❯ /System/Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python ./virtualenv_16.7.10_py --no-pip --no-setuptools --no-wheel dest_venv
New python executable in /Users/eric/code/pants/dest_venv/bin/python
ERROR: The executable /Users/eric/code/pants/dest_venv/bin/python is not functioning
ERROR: It thinks sys.prefix is '/Users/eric/code/pants' (should be u'/Users/eric/code/pants/dest_venv')
ERROR: virtualenv is not compatible with this system or executable

❯ ./dest_venv/bin/python
[1]    15655 killed     ./dest_venv/bin/python
e

enough-analyst-54434

02/23/2021, 1:00 AM
Ok - presumable one of those was exit 100?
h

hundreds-father-404

02/23/2021, 1:26 AM
Ah ha. https://github.com/pantsbuild/pex/blob/1714ef4d1ffa331c9880a80e4da0490eac710b66/pex/tools/commands/virtualenv_16.7.10_py#L1681-L1711
proc.returncode
is
-9
at the end of the try block, and
proc_stdout
is
""
.
Copy code
❯ python2
>>> import os.path
>>> os.path.normcase(os.path.realpath(""))
'/Users/eric/code/pants'
norm_home_dir
is correctly being set to
/Users/eric/code/pants/dest_venv/bin/python
, but that doesn't ==
proc_stdout
, i.e. the cwd
/Users/eric/code/pants/dest_venv
, so the script is erroring with an exit code of 100 and that log. The fundamental issue seems to be that
/Users/eric/code/pants/dest_venv/bin/python
fails with an exit code 137 when run directly,
-9
when run in the virtualenv script. But it's obfuscated by a confusing error message because the script is not eagerly validating the process had a zero exit code
e

enough-analyst-54434

02/23/2021, 1:45 AM
Ok - great. Thanks for tracking that down. Its good to know what's going on. So, -9 is a sigkill coming from ... somewhere. So that's the last breadcrumb to follow if you dare.
h

hundreds-father-404

02/23/2021, 1:47 AM
I know the rc for the below is 137, which corresponds to
sigkill -9
iiuc. Is something like this best investigated via core dump?
Copy code
❯ ./dest_venv/bin/python
[1]    15655 killed     ./dest_venv/bin/python
e

enough-analyst-54434

02/23/2021, 1:50 AM
No. A core dump is for a runtime issue in code like an illegal memory access.
It seems like some system service must identify something bas about the process and kill it. On linux, for example, if the system runs low on memory a daemon - the oomkiller - randomly picks processes to start killing. This is different since its targetd at system python 2.7 and there is no memory pressure issue. But its imilar in that it must be a daemon or the kernel doing it.
So ... I have no clue on macOs, but its probably in some sort of system (kernel) log file.
👍 1