Hello :wave: I’ve hit a curious issue with unit t...
# general
Hello 👋 I’ve hit a curious issue with unit tests, which I’ve replicated with a toy example. If you clone the repo and run
./pants test ::
will succeed, but
will fail with the error
Copy code
test/foo/dpfs/module_a/test_module_a.py:1: in <module>
    from module_a.dpfs import get_domain, plus_one
E   ModuleNotFoundError: No module named 'module_a.dpfs'
This is strange because
also imports
from module_a.dpfs import plus_one
, but the test for that module passes. This also works:
Copy code
$ ./pants repl test/foo/modules/module_a:module_a
Python 3.10.6 (main, Aug 11 2022, 13:49:25) [Clang 13.1.6 (clang-1316.] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from module_a import get_domain, plus_one
./pants check ::
passes Next, if you remove the file
rm test/foo/dpfs/module_a/__init__.py
, then
./pants test ::
will pass for all files, but
./pants check ::
will fail with the error:
Copy code
test/test_module.py: error: Duplicate module named "test_module" (also at "test/foo/modules/module_a/test_module.py")
In summary, it looks like when running the unit-test
, the file
will load before
because the path
is added to the front of
. This can be fixed by deleting
, but then type-checking fails because of a collision on
. What is the right way out of this mess? Am I starting off on the wrong foot by putting all tests under a parent test directory?
The issue is a general Python one (not Pants specific). You cannot split a package across multiple paths unless you pick one of: + Use no
: See https://peps.python.org/pep-0420/ + Use single line
: See https://packaging.python.org/en/latest/guides/packaging-namespace-packages/#creating-a-namespace-package
In other words, Python doesn't merge duplicate init.py packages unless they all have the same single line magic inside, it picks one winner and disregards all others.
Another factor that plays in are source roots. Make sure
pants roots
prints what you expect.
Merging packages is not the desired result here. Let me phrase this another way. Why is
treated as package
instead of
, thereby causing it to collide with
, the very module it is trying to test, when the source root is
Can you provide the output of
pants roots
Ah sorry - missed the repo link. Checking that out...
git rm -f test/foo/modules/module_a/__init__.py
Copy code
$ cat pyproject.toml
namespace_packages = true
explicit_package_bases = true
Copy code
$ pants lint check test ::
21:40:44.92 [INFO] Completed: Format with Black - black made no changes.
21:40:44.92 [INFO] Completed: Format with isort - isort made no changes.
21:40:44.92 [INFO] Completed: Lint using Pylint - pylint succeeded.
Partition: ['CPython>=3.10.*']
************* Module .pylintrc
.pylintrc:1:0: R0022: Useless option value for '--disable', 'bad-continuation' was removed from pylint, see <https://github.com/PyCQA/pylint/pull/3571>. (useless-option-value)
.pylintrc:1:0: W0012: Unknown option value for '--disable', expected a valid pylint message and got 'wrong-input-order' (unknown-option-value)

Your code has been rated at 10.00/10

PYLINTHOME is now '/home/jsirois/.cache/pylint' but obsolescent '/home/jsirois/.pylint.d' is found; you can safely remove the latter

✓ black succeeded.
✓ isort succeeded.
✓ pylint succeeded.
21:40:44.92 [INFO] Completed: Typecheck using MyPy - mypy - mypy succeeded.
Success: no issues found in 4 source files

✓ mypy succeeded.
21:40:44.93 [INFO] Completed: Run Pytest - test/foo/modules/module_a/test_module.py - succeeded.
============================= test session starts ==============================
platform linux -- Python 3.10.7, pytest-7.0.1, pluggy-1.0.0 -- /home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/0c926e0b/venv/bin/python3.10
cachedir: .pytest_cache
rootdir: /tmp/pants-sandbox-MBWvs2
plugins: forked-1.4.0, cov-3.0.0, xdist-2.5.0
collecting ... collected 2 items

test/foo/modules/module_a/test_module.py::test_get_domain PASSED         [ 50%]
test/foo/modules/module_a/test_module.py::test_plus_one PASSED           [100%]

- generated xml file: /tmp/pants-sandbox-MBWvs2/test.foo.modules.module_a.test_module.py.xml -

============================== 2 passed in 0.61s ===============================

21:40:44.93 [INFO] Completed: Run Pytest - test/test_module.py:module_b - succeeded.
============================= test session starts ==============================
platform linux -- Python 3.10.7, pytest-7.0.1, pluggy-1.0.0 -- /home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/0c926e0b/venv/bin/python3.10
cachedir: .pytest_cache
rootdir: /tmp/pants-sandbox-8iwZM5
plugins: forked-1.4.0, cov-3.0.0, xdist-2.5.0
collecting ... collected 1 item

test/test_module.py::test_plus_y PASSED                                  [100%]

- generated xml file: /tmp/pants-sandbox-8iwZM5/test.test_module.py.module_b.xml -

============================== 1 passed in 0.19s ===============================

✓ test/foo/modules/module_a/test_module.py succeeded in 0.89s (memoized).
✓ test/test_module.py:module_b succeeded in 0.61s (memoized).

Name                                   Stmts   Miss  Cover
src/bar/modules/module_b/__init__.py       5      0   100%
src/foo/modules/module_a/__init__.py       7      0   100%
TOTAL                                     12      0   100%

Wrote xml coverage report to `dist/coverage/python`
I'll let you read up on those mypy options, but the Pants repo itself has to use them as well.
That worked, thank you!