https://pantsbuild.org/ logo
Join Slack
Powered by
Hello. I'm back with a namespace problem^^ I've sw...
# general
r
Hello. I'm back with a namespace problem^^ I've switched to explicit namespaces and this gives me another set of problems. 🙂 I send the link to my fictional repo. On this one, there are two different projects (cards and dice) under the same namespace (turbocorp). If I try to run mypy via 
./pants check ::
mypy yells at me saying there is a duplicate module named 
turbocorp
which is true but also the purpose of the namespace 🙂 Do you know how I can fix that? I try to use 
--exclude
but I can't find a way to please mypy 😕 https://github.com/Gaasmann/monorepants
The mypy error:
Copy code
(monorepants39) nicolas@bunker:~/learning/monorepants%./pants check ::
12:03:16.63 [ERROR] Completed: Typecheck using MyPy - mypy failed (exit code 2).
py/projects/dice/src/turbocorp/__init__.py: error: Duplicate module named "turbocorp" (also at "py/projects/cards/src/turbocorp/__init__.py")
py/projects/dice/src/turbocorp/__init__.py: note: Are you missing an __init__.py? Alternatively, consider using --exclude to avoid checking one of them.
Found 1 error in 1 file (errors prevented further checking)
h
Your source root config looks good to me, can you confirm that you get what you expect when you run 
./pants roots
?
And your explicit namespace invocations look right to me as well
BTW thanks for the github repro, that makes life so much easier!
Although there is no 
pants
script in there
Adding 
--namespace-packages
to the mypy command line does not help
r
Ah yes, I have the pex file for pants, not sure why I didn't use the script here
--namespace-packages are for implicit/PEP420 namespaces, it doesn't help 😕
the roots are correct
h
Yeah, digging into this is, it looks like MyPy is correctly understanding the namespace packages in terms of the naming hierarchy, but is then stumped by the non-empty 
__init__.py
So it gets the split nature of 
turbocorp
as a package right, but then fails on its nature as a module
But per https://github.com/python/mypy/pull/9742/ explicit namespace packages should be supported
And they would always have this problem if so
So this appears to be a mypy thing, not a Pants thing
Guido is saying, in effect, “run mypy separately on each source root”
Namespace packages are properly supported as upstream deps, but not as files to be checked
@hundreds-father-404 we already partition mypy runs by interpreter etc, looks like we should also partition by source root?
e
@ripe-cpu-85141 thanks for the example repo. Does this repo also exhibit the pylint issue you were encountering when you were trying implicit namespace packages instead?
I don't get the same style errors with it. I delete all 
__init__.py
(setup mypy PEP-420 namespace package support so that works) and enable pylint:
Copy code
$ git st
On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
	new file:   mypy.ini

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	deleted:    pants-plugins/tutu/__init__.py
	modified:   pants.toml
	deleted:    py/projects/cards/src/turbocorp/__init__.py
	deleted:    py/projects/cards/src/turbocorp/cards/__init__.py
	deleted:    py/projects/cards/src/turbocorp/cards/core/__init__.py
	deleted:    py/projects/cards/tests/card/__init__.py
	deleted:    py/projects/cards/tests/deck/__init__.py
	deleted:    py/projects/dice/src/turbocorp/__init__.py
	deleted:    py/projects/dice/src/turbocorp/dice/__init__.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	pants
$ git diff origin/master pants.toml mypy.ini
diff --git a/mypy.ini b/mypy.ini
new file mode 100644
index 0000000..47946c8
--- /dev/null
+++ b/mypy.ini
@@ -0,0 +1,3 @@
+[mypy]
+namespace_packages = true
+explicit_package_bases = true
diff --git a/pants.toml b/pants.toml
index f56f189..4bf35d3 100644
--- a/pants.toml
+++ b/pants.toml
@@ -1,9 +1,10 @@
 [GLOBAL]
-pants_version = "2.12.0+git5d31cdb5"
+pants_version = "2.12.0"
 backend_packages = [
     "pants.backend.python",
     "pants.backend.python.lint.black",
     "pants.backend.python.lint.flake8",
+    "pants.backend.python.lint.pylint",
     "pants.backend.python.typecheck.mypy",
     "pants.backend.awslambda.python",
     "pants.backend.codegen.protobuf.python",
I then find:
Copy code
$ ./pants lint ::
11:05:23.32 [WARN] Completed: Format with Black - black made changes.
  py/projects/cards/src/turbocorp/cards/core/deck.py
11:05:23.45 [ERROR] Completed: Lint with Flake8 - flake8 failed (exit code 1).
pants-plugins/tutu/register.py:4:80: E501 line too long (87 > 79 characters)
pants-plugins/tutu/register.py:39:80: E501 line too long (81 > 79 characters)
pants-plugins/tutu/register.py:72:80: E501 line too long (87 > 79 characters)
pants-plugins/tutu/register.py:74:80: E501 line too long (87 > 79 characters)
py/projects/cards/src/turbocorp/cards/core/deck.py:7:80: E501 line too long (91 > 79 characters)
py/projects/cards/src/turbocorp/cards/core/deck.py:25:80: E501 line too long (120 > 79 characters)


11:05:25.52 [ERROR] Completed: Lint using Pylint - pylint failed (exit code 30).
************* Module entrypoint
py/projects/dice/src/turbocorp/dice/entrypoint.py:1:0: C0114: Missing module docstring (missing-module-docstring)
py/projects/dice/src/turbocorp/dice/entrypoint.py:1:0: C0116: Missing function or method docstring (missing-function-docstring)
************* Module test_dummy
py/projects/cards/tests/test_dummy.py:1:0: C0114: Missing module docstring (missing-module-docstring)
py/projects/cards/tests/test_dummy.py:1:0: C0116: Missing function or method docstring (missing-function-docstring)
py/projects/cards/src/turbocorp/cards/entrypoint.py:1:0: C0114: Missing module docstring (missing-module-docstring)
py/projects/cards/src/turbocorp/cards/entrypoint.py:4:0: C0116: Missing function or method docstring (missing-function-docstring)
py/projects/cards/src/turbocorp/cards/entrypoint.py:8:0: C0116: Missing function or method docstring (missing-function-docstring)
py/projects/cards/src/turbocorp/cards/entrypoint.py:8:12: W0613: Unused argument 'nope' (unused-argument)
py/projects/cards/src/turbocorp/cards/entrypoint.py:8:18: W0613: Unused argument 'blah' (unused-argument)
************* Module test_deck
py/projects/cards/tests/deck/test_deck.py:1:0: C0114: Missing module docstring (missing-module-docstring)
py/projects/cards/tests/deck/test_deck.py:4:0: C0116: Missing function or method docstring (missing-function-docstring)
************* Module test_card
py/projects/cards/tests/card/test_card.py:1:0: C0114: Missing module docstring (missing-module-docstring)
py/projects/cards/tests/card/test_card.py:1:0: E0401: Unable to import 'pytest' (import-error)
py/projects/cards/tests/card/test_card.py:14:0: C0116: Missing function or method docstring (missing-function-docstring)
************* Module conftest
py/projects/cards/tests/conftest.py:1:0: C0114: Missing module docstring (missing-module-docstring)
py/projects/cards/tests/conftest.py:1:0: E0401: Unable to import 'pytest' (import-error)
py/projects/cards/tests/conftest.py:7:0: C0116: Missing function or method docstring (missing-function-docstring)
************* Module game
py/projects/cards/src/turbocorp/cards/game.py:1:0: C0114: Missing module docstring (missing-module-docstring)
py/projects/cards/src/turbocorp/cards/game.py:4:0: C0116: Missing function or method docstring (missing-function-docstring)
************* Module deck
py/projects/cards/src/turbocorp/cards/core/deck.py:25:0: C0301: Line too long (120/100) (line-too-long)
py/projects/cards/src/turbocorp/cards/core/deck.py:1:0: C0114: Missing module docstring (missing-module-docstring)
py/projects/cards/src/turbocorp/cards/core/deck.py:12:0: C0115: Missing class docstring (missing-class-docstring)
py/projects/cards/src/turbocorp/cards/core/deck.py:17:4: C0116: Missing function or method docstring (missing-function-docstring)
py/projects/cards/src/turbocorp/cards/core/deck.py:27:4: C0116: Missing function or method docstring (missing-function-docstring)
py/projects/cards/src/turbocorp/cards/core/deck.py:31:4: C0116: Missing function or method docstring (missing-function-docstring)
py/projects/cards/src/turbocorp/cards/core/deck.py:35:4: C0116: Missing function or method docstring (missing-function-docstring)
************* Module register
pants-plugins/tutu/register.py:1:0: C0114: Missing module docstring (missing-module-docstring)
pants-plugins/tutu/register.py:24:0: C0115: Missing class docstring (missing-class-docstring)
pants-plugins/tutu/register.py:24:0: R0901: Too many ancestors (8/7) (too-many-ancestors)
pants-plugins/tutu/register.py:28:0: C0115: Missing class docstring (missing-class-docstring)
pants-plugins/tutu/register.py:32:0: C0115: Missing class docstring (missing-class-docstring)
pants-plugins/tutu/register.py:32:0: R0903: Too few public methods (0/2) (too-few-public-methods)
pants-plugins/tutu/register.py:37:0: C0115: Missing class docstring (missing-class-docstring)
pants-plugins/tutu/register.py:42:0: C0115: Missing class docstring (missing-class-docstring)
pants-plugins/tutu/register.py:47:0: C0115: Missing class docstring (missing-class-docstring)
pants-plugins/tutu/register.py:51:0: C0115: Missing class docstring (missing-class-docstring)
pants-plugins/tutu/register.py:64:0: C0116: Missing function or method docstring (missing-function-docstring)
pants-plugins/tutu/register.py:81:0: C0116: Missing function or method docstring (missing-function-docstring)
pants-plugins/tutu/register.py:88:0: C0116: Missing function or method docstring (missing-function-docstring)
pants-plugins/tutu/register.py:19:0: C0411: standard import "import logging" should be placed before "from pants.core.target_types import ResourceSourceField" (wrong-import-order)

------------------------------------------------------------------
Your code has been rated at 5.38/10 (previous run: 5.38/10, +0.00)



✕ black failed.
✕ flake8 failed.
✕ pylint failed.

(One or more formatters failed. Run `./pants fmt` to fix.)
The modules 2 namespace packages are checked and nowhere is there a "E1101: Module 'audit' has no 'lib' member (no-member)".
r
@enough-analyst-54434 I don't use pylint so I'm not sure of the issue you mention
e
🤦I'm sorry about that! That was George - different thread.
Do you have a link to a thread that explains why you had to move away from PEP-420 namespace packages?
r
e
Ah, then you were just missing:
Copy code
$ git diff origin/master mypy.ini
diff --git a/mypy.ini b/mypy.ini
new file mode 100644
index 0000000..47946c8
--- /dev/null
+++ b/mypy.ini
@@ -0,0 +1,3 @@
+[mypy]
+namespace_packages = true
+explicit_package_bases = true
That works in your example repo.
You need that second line too.
r
The explicit_package_bases one?
e
Yup
I updated that other thread with this answer.
r
ah yes, it looks good, I need to test on my work project
thanks!
e
r
Ah yes. I should study the repo as you're using pants 🙂
5 Views
Open in Slack
PreviousNext
Powered by