From within a

PEX_INTERPRETER=1

how can I invoke the PEX's entry_point/script as if I had just invoked the pex (but still using this running Python environment)

Forget Pex and answer the question like you would re invoking any entry point in a plain old Python interpreter. Do you just need to know how to find out what the entry point value is?

That's an option, yes.

PEX offers no code APIs so that's the only option.

Fix debugpy to not try and lookup the entry point by name

I have 0 clues about debugpy and how it works. Are there docs I can read?

There's a wiki. The TL;DR is that: • it needs to be executed in the same environment as the code you intend to debug. Pants does this with an additional

PEX_PATH

entry on the test runner PEX • It must be invoked as a module and then acts like a Python interpreter after it's args. E.g.

python -m debugpy <debugpy_args> <normal_py_interpreter_args>

Does

python -m debugpy <debugpy> PEX

not work?

Thats what I'm trying right now, and getting strange results (on the debugpy side). I wanted to see if the entry-point way would also work so I could decide.

Ok, well that's like I said. You'd need to get the ep from PEX-INFO json.

I'm going to drill harder on why I can't just invoke the pex, then if I get too annoyed try that. Thanks

Yeah, battle annoyance - that's certainly the most straightforward and robust path if you can figure it out.

OK last bit before I pause for sleep.

PEX_MODULE=debugpy python pytest_runner.pex --listen 127.0.0.1:5678 --log-to-stderr  pytest_runner.pex

exhibits the odd behavior

PEX_MODULE=debugpy python pytest_runner.pex --listen 127.0.0.1:5678 --log-to-stderr -c 'import pytest;pytest.console_main()

(the entry point) doesn't 🤔

I'm pretty sure the issue is PEX re-exec. If you avoid that, no issues. Materials:

$ pex debugpy -m debugpy -o debugpy.pex
$ pex pytest -c pytest -o pytest.loose.pex --layout loose
$ pex pytest -c pytest -o pytest.pex

Troubles due to the PEX doing a re-

exec

into

~/.pex/unzipped_pexes/...

:

$ ./debugpy.pex --listen 127.0.0.1:5678 pytest.pex --version
0.06s - Error importing debugpy._vendored.force_pydevd (with sys.path entry: '/home/jsirois/.pex/installed_wheels/2a39e7da178e1f22f4bc04b57f085e785ed1bcf424aaf318835a1a7129eefe35/debugpy-1.6.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl')
Traceback (most recent call last):
  File "/home/jsirois/.pex/installed_wheels/2a39e7da178e1f22f4bc04b57f085e785ed1bcf424aaf318835a1a7129eefe35/debugpy-1.6.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_defaults.py", line 60, in on_pydb_init
    __import__(module_name)
ModuleNotFoundError: No module named 'debugpy'
0.00s - Could not connect to 127.0.0.1: 36263
Traceback (most recent call last):
  File "/home/jsirois/.pex/installed_wheels/2a39e7da178e1f22f4bc04b57f085e785ed1bcf424aaf318835a1a7129eefe35/debugpy-1.6.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py", line 493, in start_client
    s.connect((host, port))
ConnectionRefusedError: [Errno 111] Connection refused
Could not connect to 127.0.0.1: 36263
Traceback (most recent call last):
  File "/home/jsirois/.pex/installed_wheels/2a39e7da178e1f22f4bc04b57f085e785ed1bcf424aaf318835a1a7129eefe35/debugpy-1.6.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl/debugpy/_vendored/pydevd/pydevd.py", line 3473, in main
    debugger.connect(host, port)
  File "/home/jsirois/.pex/installed_wheels/2a39e7da178e1f22f4bc04b57f085e785ed1bcf424aaf318835a1a7129eefe35/debugpy-1.6.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl/debugpy/_vendored/pydevd/pydevd.py", line 1418, in connect
    s = start_client(host, port)
  File "/home/jsirois/.pex/installed_wheels/2a39e7da178e1f22f4bc04b57f085e785ed1bcf424aaf318835a1a7129eefe35/debugpy-1.6.4-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_comm.py", line 493, in start_client
    s.connect((host, port))
ConnectionRefusedError: [Errno 111] Connection refused

No troubles - no re-exec for loose:

$ ./debugpy.pex --listen 127.0.0.1:5678 pytest.loose.pex/ --version
pytest 7.2.0

So you'd need to know the actual location of the PEX. That is done in one of 3 ways: 1. Make a loose PEX. 2. Use the PEX_TOOLS=1 to install the PEX in a venv at a directory you know. 3. When building the PEX use

--seed verbose

Pants does 3 as part of VenvPex creation to get a ~50ms boost over calling into the --venv PEX and suffering a re-

exec

.

$ pex pytest -c pytest -o pytest.pex --seed verbose
{"pex_root": "/home/jsirois/.pex", "python": "/home/jsirois/bin/pex.venv/bin/python", "pex": "/home/jsirois/.pex/unzipped_pexes/3ee2eaac643589ce50a865add8ae6e2ea2f52640/__main__.py"}
$ ./debugpy.pex --listen 127.0.0.1:5678 /home/jsirois/.pex/unzipped_pexes/3ee2eaac643589ce50a865add8ae6e2ea2f52640/__main__.py --version
pytest 7.2.0

Ahh 3 looks the most promising I think.

$PEX_MODULE=debugpy python pytest_runner.pex --listen 127.0.0.1:5678 .cache/pex_root/venvs/e04f6a4578bcc4f46dc29bec06904079cd804d92/a5d126d40b834824d76a778912f43e6ad41c876b/pex --version
pytest 7.0.1

☝️ thats from within the Pants sandbox for the call (albeit with the entry_point for the runner set back to whatever it was pre-debugpy). I also think it's safe to assume that path will exist (which the shim script can't assume) because as it stands we're executing from that very venv. I'll have to try it out and tweak it and comment as such