Hi all, I can see that

[pytest]
extra_requirements.add = ["ipdb"]

Is scheduled to be dropped in

2.18

What’s the recommended way for using

ipdb

in tests moving forward? I’ve got the following in my

pants.toml

[test]
extra_env_vars = ["PYTHONBREAKPOINT=ipdb.set_trace"]

And this in my

BUILD

python_tests(
    name="tests0",
    dependencies=["//:reqs0#ipdb"]
)

In my dummy example I can see

ipdb

is a transitive dependency, so it’s included in the pex build

➜  pants-test pants dependencies --transitive test_app.py
21:28:55.35 [INFO] Initializing scheduler...
21:28:58.44 [INFO] Scheduler initialized.
//:reqs0#Flask
//:reqs0#ipdb
//:reqs0#pytest
//main.py:root
//requirements.txt:reqs0

But when I set a breakpoint, it just gives me plain old

pdb

even though the

PYTHONBREAKPOINT

environment is present and

ipdb

is available

pants test --debug :: -- -s
<OUTPUT REDACTED>
(Pdb) os.environ.get("PYTHONBREAKPOINT")
'ipdb.set_trace'

What am I missing

Well, did you read the doc / warning?: https://www.pantsbuild.org/docs/reference-pytest#extra_requirements It looks like the link does not render right, but that

Custom tool versions are now installed from named resolves, as described at [Lockfiles](doc:python-lockfiles).

should point to https://www.pantsbuild.org/docs/python-lockfiles#lockfiles-for-tools

I read the warning and don’t really understand what it wants me to do. From the above everything to me looks like

ipdb

should be working. The absence of explicitly doing something like:

[python.resolves]
pytest = "3rdparty/python/pytest.lock"

[pytest]
install_from_resolve = "pytest"  # Use this resolve's lockfiles.
requirements =["//3rdparty/python:pytest"]  # Use these requirements from the lockfile.

Means it should use my default lockfile right? (where ipdb is present)

Definitely check out https://www.pantsbuild.org/docs/python-lockfiles#lockfiles-for-tools - have you created a custom tool lockfile like it directs?

Apologies that was just an example stolen from the docs. I’ve tried adding the following to my

pants.toml

[pytest]
install_from_resolve = "python-default"
requirements = [
  "//:reqs0#pytest",
  "//:reqs0#ipdb",
]

Which is present in my

BUILD

python_requirements(
    name="reqs0",
)

Setting a breakpoint in my test still gives me the same pdb debugger, but ipdb is importable:

(Pdb) import ipdb
(Pdb) import os
(Pdb) os.environ.get("PYTHONBREAKPOINT")
'ipdb.set_trace'
(Pdb)

This is after running

pants generate-lockfiles ::

So PYTHONBREAKPOINT is only respected for Python >= 3.7 (https://peps.python.org/pep-0553/), what does

sys.version_info

say?

[python]
interpreter_constraints = ['==3.11.*']

(Pdb) import sys
(Pdb) sys.version_info
sys.version_info(major=3, minor=11, micro=4, releaselevel='final', serial=0)

Ok, well, no clue. Would you agree at this point it does not look like a Pants issue? You have ipdb on sys.path, you have the right interpreter version, and you have the env var leaked.

I’ve tried doing

pants export ::

and running pytest manually inside the virtualenv

PYTHONBREAKPOINT=ipdb.set_trace pytest -s test_app.py

And this doesn’t work either, so I’m stumped. I don’t think this is a pants issue.

Look at the venv! Does it use

-E

? I bet the

pytest

script uses

-E

- check the shebang.

I think:

[pytest]
extra_requirements.add = ["ipdb"]

worked in 2.16 but I’d have to double check. I’m currently on 2.17

Ok. And if you can

head -1 $(which pytest)

in the exported + activated venv, it would be nice to confirm

-E

in the shebang.

(3.11.4) ➜  pants-test git:(main) ✗ head -1 $(which pytest)
#!/Users/jack/code/pants-test/dist/export/python/virtualenvs/python-default/3.11.4/bin/python3.11 -sE

Yeaaah … You nailed it

Ok, and based on that I'll be very surprised if this ever worked.

I must have misremembered this ever working 🤔 But the docs do have a lot of references to using ipdb with pytest, so makes me think this has worked at some point after 2.4.0, I’m unsure. I’m unsure what I’d be setting the

entry_point

of pytest to from your suggestion https://www.pantsbuild.org/docs/reference-pytest#entry_point iPart of me thinks I’ve misremembered this working, but then the docs have lots of references to using

Can you cat the

pytest

console script here?

#!/Users/jack/code/pants-test/dist/export/python/virtualenvs/python-default/3.11.4/bin/python3.11 -sE
# -*- coding: utf-8 -*-
import re
import sys
from pytest import console_main
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(console_main())

But altering

[pytest]
install_from_resolve = "python-default"
entry_point="pytest:console_main"
requirements = [
  "//:reqs0#pytest",
  "//:reqs0#ipdb",
]

doesn’t appear to change the behaviour

Ok, that was my one wild lazy guess at a workaround. I think this will require an issue filed + work.

To clarify one point - if you don't specify otherwise, Pants will install pytest from a default, built-in lockfile. If you want to use your default user-code lockfile you have to point to it:

[pytest]
install_from_resolve = "python-default"

But also note that if you do this naively it means that your tests will depend on the entire lockfile, and be invalidated more often than you'd like. So you may also want to set

requirements =[...]

to just the actual needed requirement targets (pytest, ipdb and anything pytest plugins, for example)

Thank you both for your help: I have filed an issue here: https://github.com/pantsbuild/pants/issues/19977