polite-angle-19513
08/25/2021, 3:49 PM__path__ = __import__("pkgutil").extend_path(__path__, __name__) # type: ignore
However, in a simplified pants-test setup I cannot pants to do what I think it ought too.
It's been a very frustrating and quite time consuming attempt to adopt Pants. I feel like I'm so close to a working setup and then no matter what I do it seems to work as expected in many ways and then falls over in others. (And vice versa when I setup with PEP420; mypy works in all and pylint fails - opposite to the pkgutil namespace setup).
The Pants docs, while referencing namespace packages, wasn't enough for me to glean the best way.
Maybe another monorepo user might have some wisdom on the matter? (TY)
.
├── mypy.ini
├── pants
├── pants.toml
├── pylintrc
├── pyproject.toml
└── src
└── python
├── hankai-dep (depends hankai-util)
│ ├── BUILD
│ ├── hankai
│ │ ├── init.py
│ │ └── dep
│ │ ├── init.py
│ │ └── dep.py
│ ├── poetry.lock
│ ├── poetry.toml
│ ├── pyproject.toml
│ └── requirements.txt
└── hankai-util (independent)
├── BUILD
├── hankai
│ ├── init.py
│ └── util
│ ├── init.py
│ └── util.py
├── poetry.lock
├── poetry.toml
├── pyproject.toml
└── requirements.txt
hankai-util is independent, hankak-dep is dependent on hankai-util
src/python/hankai-util/BUILD
python_requirements()
python_library(
name="lib",
dependencies=[
":loguru",
":pydantic",
],
sources=[
"hankai/**/*.py",
]
)
src/python/hankai-dep/BUILD
python_requirements()
python_library(
name="lib",
dependencies=[
":loguru",
":pydantic",
"src/python/hankai-util:lib",
],
sources=[
"hankai/**/*.py",
]
)
---------------------- LIST
./pants list ::
src/python/hankai-dep:colorama
src/python/hankai-dep:dist
src/python/hankai-dep:lib
src/python/hankai-dep:loguru
src/python/hankai-dep:pydantic
src/python/hankai-dep:requirements.txt
src/python/hankai-dep:typing-extensions
src/python/hankai-dep:win32-setctime
src/python/hankai-util:colorama
src/python/hankai-util:dist
src/python/hankai-util:lib
src/python/hankai-util:loguru
src/python/hankai-util:pydantic
src/python/hankai-util:requirements.txt
src/python/hankai-util:tqdm
src/python/hankai-util:typing-extensions
src/python/hankai-util:win32-setctime
---------------------- DEPENDENCIES/DEPENDEES
./pants dependencies ::
src/python/hankai-dep/hankai/__init__.py:../lib
src/python/hankai-dep/hankai/dep/__init__.py:../../lib
src/python/hankai-dep/hankai/dep/dep.py:../../lib
src/python/hankai-dep:loguru
src/python/hankai-dep:pydantic
src/python/hankai-dep:requirements.txt
src/python/hankai-util/hankai/__init__.py:../lib
src/python/hankai-util/hankai/util/__init__.py:../../lib
src/python/hankai-util/hankai/util/util.py:../../lib
src/python/hankai-util:dist
src/python/hankai-util:loguru
src/python/hankai-util:pydantic
src/python/hankai-util:requirements.txt
./pants dependencies src/python/hankai-_dep_:lib
src/python/hankai-dep/hankai/__init__.py:../lib
src/python/hankai-dep/hankai/dep/__init__.py:../../lib
src/python/hankai-dep/hankai/dep/dep.py:../../lib
src/python/hankai-dep:loguru
src/python/hankai-dep:pydantic
src/python/hankai-util/hankai/__init__.py:../lib
src/python/hankai-util/hankai/util/__init__.py:../../lib
src/python/hankai-util/hankai/util/util.py:../../lib
./pants dependencies src/python/hankai-util:lib
src/python/hankai-util/hankai/__init__.py:../lib
src/python/hankai-util/hankai/util/__init__.py:../../lib
src/python/hankai-util/hankai/util/util.py:../../lib
src/python/hankai-util:loguru
src/python/hankai-util:pydantic
./pants dependees src/python/hankai-util:lib
src/python/hankai-dep:lib
src/python/hankai-dep/hankai/__init__.py:../lib
src/python/hankai-dep/hankai/dep/__init__.py:../../lib
src/python/hankai-dep/hankai/dep/dep.py:../../lib
src/python/hankai-util:dist
---------------------- LINT
*./pants lint :: (*Successful)
./pants lint src/python/hankai-util:lib (Successful)
./pants lint src/python/hankai-dep:lib (FAILS) - Expected that the hankai-util deps would be referenced since ./pants knows about them as shown above.
src/python/hankai-dep/hankai/dep/dep.py20: E0611: No name 'util' in module 'hankai' (no-name-in-module)
src/python/hankai-dep/hankai/dep/dep.py20: E0401: Unable to import 'hankai.util' (import-error)
src/python/hankai-dep/hankai/dep/dep.py158: E1101: Module 'hankai' has no 'util' member (no-member)
./pants lint src/python/hankai-dep:lib src/python/hankai-util:lib (Successful) - But I expected the explicit dependency to be used and to not have to directly reference it.
---------------------- TYPECHECK
./pants typecheck :: (FAILS) - I'm not doing PEP420. Using pkgutil namespace packaging. How am I supposed to use pants with any form of namespace package support??
src/python/hankai-util/hankai/__init__.py: error: Duplicate module named
"hankai" (also at "src/python/hankai-dep/hankai/__init__.py")
src/python/hankai-util/hankai/__init__.py: note: Are you missing an init.py? Alternatively, consider using --exclude to avoid checking one of them.
Found 1 error in 1 file (errors prevented further checking)
./pants typecheck src/python/hankai-util:lib (Successful)
./pants typecheck src/python/hankai-dep:lib (FAILS) - Expected the hankai-util deps would be referenced since ./pants knows about them as shown above.
src/python/hankai-dep/hankai/dep/dep.py21: error: Cannot find implementation
or library stub for module named "hankai.util" [import]
import hankai.util
^
src/python/hankai-dep/hankai/dep/dep.py21: note: See https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-imports
Found 1 error in 1 file (checked 3 source files)
./pants typecheck src/python/hankai-dep:lib src/python/hankai-util:lib (FAILS) as before with ::
src/python/hankai-util/hankai/__init__.py: error: Duplicate module named
"hankai" (also at "src/python/hankai-dep/hankai/__init__.py")
src/python/hankai-util/hankai/__init__.py: note: Are you missing an init.py? Alternatively, consider using --exclude to avoid checking one of them.
Found 1 error in 1 file (errors prevented further checking)
============================================
I feel like I'm so close to a working but am just not getting there.
Maybe another monorepo user might have some wisdom on the matter?
Thank you all for your consideration and help.witty-crayon-22786
08/25/2021, 3:54 PM./pants roots
?polite-angle-19513
08/25/2021, 4:07 PMwitty-crayon-22786
08/25/2021, 4:08 PMsrc/python
src/python
off the front of those directories, you’ll get import
statements like import hankai-util.hankai.util
etcmarker_filenames
strategy, because you need multiple source roots.polite-angle-19513
08/25/2021, 4:13 PMwitty-crayon-22786
08/25/2021, 4:14 PM