https://pantsbuild.org/ logo
Join Slack
Powered by
some question about `namespace package` . I have f...
# general
h
some question about 
namespace package
. I have following project structure. I make 
src/python/some_package
as namespace package. Then I do 
pants package src/python/some_package/package1:dist
, it generate the whl and tar files. When I check the 
package1-0.1.0.dist-info/namespace_packages.txt
inside the 
dist/package1-0.1.0-py3-none-any.whl
, it is empty. I am wondering if pants support namespace package correctly?
Copy code
.
├── 3rdparty
│   └── python
│       └── python-default.lock
├── dist
│   ├── package1-0.1.0-py3-none-any.whl
│   └── package1-0.1.0.tar.gz
├── pants.toml
├── src
│   └── python
│       └── some_package
│           ├── lib1
│           │   ├── __init__.py
│           │   ├── BUILD
│           │   ├── lib1.py
│           │   └── pyproject.toml
│           ├── package1
│           │   ├── __init__.py
│           │   ├── BUILD
│           │   ├── package1.py
│           │   └── pyproject.toml
│           └── package2
│               ├── __init__.py
│               ├── BUILD
│               ├── package2.py
│               └── pyproject.toml
└── test
    └── python
        └── tests
            └── some_package
                └── package1
                    ├── __init__.py
                    ├── BUILD
                    ├── pyproject.toml
                    └── test_package1.py
And this is the content of 
src/python/some_package/package1/BUILD
Copy code
python_sources(
    name="package1",
    dependencies=[
        "//src/python/some_package/lib1:lib1",
    ]
)

python_requirements(
    name="reqs",
    source="pyproject.toml",
)

python_distribution(
    name="dist",
    provides=python_artifact(
        name="package1",
        version="0.1.0",
    ),
    dependencies=[
        ":package1",  # Reference your code
    ],
    wheel_config_settings={"--python-tag": ["py39"]},
)
And this is the command to check the generated whl file content:
Copy code
% unzip -l dist/package1-0.1.0-py3-none-any.whl
Archive:  dist/package1-0.1.0-py3-none-any.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
        0  06-26-2025 18:59   some_package/package1/__init__.py
       89  06-26-2025 18:59   some_package/package1/package1.py
      138  06-26-2025 18:59   package1-0.1.0.dist-info/METADATA
       91  06-26-2025 18:59   package1-0.1.0.dist-info/WHEEL
        1  06-26-2025 18:59   package1-0.1.0.dist-info/namespace_packages.txt
       13  06-26-2025 18:59   package1-0.1.0.dist-info/top_level.txt
      577  06-26-2025 18:59   package1-0.1.0.dist-info/RECORD
---------                     -------
      909                     7 files
Copy code
% unzip -p dist/package1-0.1.0-py3-none-any.whl package1-0.1.0.dist-info/namespace_packages.txt
It is empty
And this is my 
source
setting in pants.toml
Copy code
[source]
root_patterns = [
  "/src/python",
  "/test/python"
]
h
I think you are misunderstanding what “namespace packages” are in Python.
It doesn’t quite mean what it sounds like
It’s a mechanism for allowing the contents of subpackages of some package to be split across multiple artifacts
In Python2 that required some explicit machinery in the 
__init__.py
. But Python3 has “implicit namespace packages” - any package with no 
__init__.py
is implicitly a namespace package.
But those implicit namespace packages are not supposed to be listed in namespace_packages.txt
so Pants is doing the right thing here
Basically your setup seems fine to me at first glance
h
In my above repo, 
some_package
is the namespace package. Underneth are lib1, package1 and package2 need to be published/released separately. I think my understanding is the same as yours, isn’t it?
h
Ah yes, so that is indeed serving as a namespace package whose subpackages can be split across multiple distributions, and your understanding is correct, my apologies.
h
So if 
some_package
is not listed in namespace_package.txt, when user include both pakcage1 and packge2, how python know these two packages should be merged into 
some_package
instead of override each other?
h
The import mechanism figures it out
namespace_package.txt
is for pkg_resources-style namespace packages, where otherwise the import mechanism can’t figure it out…
it’s a hack
but this should “just work”
h
I see. I think I probably misunderstand 
namespace_package.txt
. Need to take a look. So basically you are saying if user include both package1 and package2, there is no collision, right?
h
Python packaging is annoyingly unstandardized, but yes, in python 3 if you have an empty parent package with no 
__init__.py
then things should work as you expect
h
Just read the doc here: https://packaging.python.org/en/latest/guides/packaging-namespace-packages/#pkg-resources-style-namespace-packages. Seems namespace_package.txt is a legacy way of doing it. Thanks a lot @happy-kitchen-89482!
10 Views
Open in Slack
PreviousNext
Powered by