https://pantsbuild.org/ logo
Join Slack
Powered by
Is there any known issue with PEX interfering with...
# general
n
Is there any known issue with PEX interfering with 
PYTHONBREAKPOINT=0
?
Copy code
import os


if __name__ == "__main__":
    os.environ['PYTHONBREAKPOINT'] = '0'
    print(os.environ['PYTHONBREAKPOINT'])
    breakpoint()
    print('DONE!')
If I run ths in my IDE, it exits immediately. If I run this with 
pantsd run proj/app/pybreakpoint_test
I get the debugger. This is true if I set the environment variable before invoking the run as well.
After reading the PEP a little closer ((https://peps.python.org/pep-0553/#environment-variable) I realized PEX invokes Python with 
ignore_environment=1
and 
no_user_site=1
which disables any 
PYTHON*
settings. Does Pants support disabling the 
-E
flag PEX passes to the interpreter?
h
If I build this directly in Pex, the right thing happens, so this is to do with some specific way Pants is using Pex here:
Copy code
$ cat python_env/breakpoint.py 

import os


if __name__ == "__main__":
    os.environ['PYTHONBREAKPOINT'] = '0'
    print(os.environ['PYTHONBREAKPOINT'])
    breakpoint()
    print('DONE!')

$ python -m pex -D python_env/ -m breakpoint -o python_env.pex
$ ./python_env.pex 
0
DONE!
$
So I recommend running the 
pants run --keep-sandboxes=always ...
and looking at the command in 
__run.sh
in the relevant sandbox to see how Pants is invoking Pex here, and use that to figure out which Pex flags are causing this behavior. Then we can figure out if/how to modify.
n
How do you know which one is the relevant sandbox? The logs indicated at least a dozen saved caches in a single 
run
goal. Picking the last one from the log messages doesn't seem promising (but certainly looks like the one that tried in invoke the 
pex_binary
target): Paraphrasing since it's at work: 
./python ./pex_v2.1.34 -tmpdur .tmp --jobs 20 --pip-version xx --cert xx --python-path xx --output-file proj.app.iralpha/iralpha.pex --no-emit-warnings --python-shebang=xx --venv prepend --requirements-pex xx --interpreter-constraint xx --entry-point iralpha.iralpha --sources-directory=source_files --lock xx --no-pypi --index=xx --find-links=xx --manylinux manylinux2014 --layout zipapp
Doesn't look obvious from this command that 
-E
is being explicitly set. I do see some references to this option in the PEX codebase though; you're positive you get the same behavior by just ordinarily pex'ing the file w/o Pants?
Also just FYI - reason for reporting this is sometimes the occasional 
breakpoint()s
make it into some of our less rigorous code and it crashes in production, even with 
PYTHONBREAKPOINT=0
h
It should say something like "preserving sandbox for interactive run"
or words to that effect
If I run pex without Pants I get the behavior you'd expect (immediate exit, no breakpoint), that is, the "raw python" behavior
n
Ok gotcha. I'll take a closer look tomorrow then.
h
So once you have the pex cmd line that fails you can play with it until you get it to work and then we can see what exactly is causing the issue
2 Views
Open in Slack
PreviousNext
Powered by