cool-easter-32542
09/19/2023, 2:16 PM~/code/pantsbuild/pex (main) $ python -m pex -o testpex.pex
~/code/pantsbuild/pex (main) $ ./testpex.pex
Python 3.9.17 (main, Aug 8 2023, 12:13:14)
[Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> print('hello')
hello
>>> quit()
~/code/pantsbuild/pex (main) $
If we pass a python command line option like -q, which normally quiets the repl preamble, the pex command hangs indefinitely:
~/code/pantsbuild/pex (main) $ python -q
>>> quit()
~/code/pantsbuild/pex (main) $ ./testpex.pex -q
# hangs ... and requires Ctrl-C
Motivation
This came up while testing a fix to address
pantsbuild/pants#3423 so we can pass through args to pants repl for the python shell. For a python shell, pants runs a requirements.pex as an interactive process to give a repl to the user.
Usually if you want to pass args to a python repl it is going to be either interpreter options like these or it will be -i to run a script/module in interactive mode.
Diagnosis
Under the hood it hangs because pex gets into an infinite loop of re-executing itself with the same argument. We deal with python options inside `Pex.execute_interpreter`:
args = sys.argv[1:]
python_options = []
for index, arg in enumerate(args):
# Check if the arg is an expected startup arg.
if arg.startswith("-") and arg not in ("-", "-c", "-m"):
python_options.append(arg)
else:
args = args[index:]
break
# The pex was called with Python interpreter options
if python_options:
return self.execute_with_options(python_options, args)
The first block of code is attempting to partition the input args into python options like -q or -B and a remainder of command line args that should be passed through when we execute_with_options. But when the args supplied are only python options, we do not set args = [] because the args = args[index:] does not get triggered. Therefore we end up execing something like this:
# cmdline
['<path_to_python>/python3.9', '-q', '<path_to_unzipped_pex>/__main__.py', '-q']
later in os.execv(python, cmdline) within execute_with_options which puts us in an infinite loop because the pex will encounter the -q again as a python option.
This is simple to fix, a PR will be forthcoming.
pantsbuild/pexcool-easter-32542
09/23/2023, 9:36 PM