https://pantsbuild.org/ logo
Join Slack
Powered by
I’m having troubles with packaging my python modul...
# general
l
I’m having troubles with packaging my python modules with Poetry backend. My directory structure is like this
Copy code
projectname
|-- src
|   |-- python
|       |-- data-models
|           |-- src
|               |-- data_models
|                   |-- __init__.py
|                   |-- module1.py
|           |-- tests
|           |-- BUILD
|           |-- pyproject.toml
|           |-- README.md
The 
BUILD
is like this
Copy code
python_sources(
    name="data_models",
    sources=[
        "src/data_models/**/*.py",
    ],
    dependencies=["3rdparty/python#pydantic"],
)

resource(name="pyproject", source="pyproject.toml")

python_distribution(
    name="dist",
    dependencies=[
        ":pyproject",
    ],
    provides=python_artifact(
        name="data_models",
        version="0.1.0",
    ),
    wheel_config_settings={"--build-option": ["--python-tag", "py39"]},
    generate_setup=False,
)
And the 
pyproject.toml
is like this
Copy code
[tool.poetry]
name = "data-models"
version = "0.1.0"
readme = "README.md"
packages = [{include = "data_models", from = "src"}]
classifiers = [
    "Intended Audience :: Developers",
    "Programming Language :: Python :: 3.9",
]

[tool.poetry.dependencies]
python = "^3.9"
pydantic = "^1.10.0"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"
When I tried to build the package with 
pants package
, it complained that 
src/python/data-models/src/data_models does not contain any element
(the full errors are in the thread) However, when I ran 
poetry build
inside 
data-models
, everything works well. I might be missing something when it comes to configure pants to use poetry as its build backend. Any help is greatly appreciated!
✅ 1
Copy code
17:54:19.29 [INFO] Preserving local process execution dir /private/var/folders/c2/q42zyfjj71zdhqqqvphcgg3c0000gn/T/pants-sandbox-R9B8tV for Run poetry.core.masonry.api for projectname/src/python/data-models:dist
17:54:19.29 [ERROR] 1 Exception encountered:

Engine traceback:
  in `package` goal

ProcessExecutionFailure: Process 'Run poetry.core.masonry.api for projectname/src/python/data-models:dist' failed with exit code 1.
stdout:

stderr:
Traceback (most recent call last):
  File "/private/var/folders/c2/q42zyfjj71zdhqqqvphcgg3c0000gn/T/pants-sandbox-R9B8tV/chroot/projectname/src/python/data-models/../../../../../.cache/pex_root/venvs/0bf5c3e53f6fcad347171b4bb21ffa0ecc0d04a1/6dbd1ad02be6f05bac60e2625c04c1725b542ebe/pex", line 232, in <module>
    exec(ast, globals_map, locals_map)
  File "backend_shim.py", line 28, in <module>
    wheel_path = backend.build_wheel(dist_dir, wheel_config_settings) if build_wheel else None
  File "/Users/username/.cache/pants/named_caches/pex_root/venvs/s/867013a5/venv/lib/python3.9/site-packages/poetry/core/masonry/api.py", line 56, in build_wheel
    return WheelBuilder.make_in(
  File "/Users/username/.cache/pants/named_caches/pex_root/venvs/s/867013a5/venv/lib/python3.9/site-packages/poetry/core/masonry/builders/wheel.py", line 78, in make_in
    wb = WheelBuilder(
  File "/Users/username/.cache/pants/named_caches/pex_root/venvs/s/867013a5/venv/lib/python3.9/site-packages/poetry/core/masonry/builders/wheel.py", line 59, in __init__
    super().__init__(poetry, executable=executable)
  File "/Users/username/.cache/pants/named_caches/pex_root/venvs/s/867013a5/venv/lib/python3.9/site-packages/poetry/core/masonry/builders/builder.py", line 83, in __init__
    self._module = Module(
  File "/Users/username/.cache/pants/named_caches/pex_root/venvs/s/867013a5/venv/lib/python3.9/site-packages/poetry/core/masonry/utils/module.py", line 79, in __init__
    PackageInclude(
  File "/Users/username/.cache/pants/named_caches/pex_root/venvs/s/867013a5/venv/lib/python3.9/site-packages/poetry/core/masonry/utils/package_include.py", line 29, in __init__
    self.check_elements()
  File "/Users/username/.cache/pants/named_caches/pex_root/venvs/s/867013a5/venv/lib/python3.9/site-packages/poetry/core/masonry/utils/package_include.py", line 66, in check_elements
    raise ValueError(
ValueError: /private/var/folders/c2/q42zyfjj71zdhqqqvphcgg3c0000gn/T/pants-sandbox-R9B8tV/chroot/projectname/src/python/data-models/src/data_models does not contain any element
h
Hmm, debugging this would require enough knowledge of poetry's PEP517 backend to know what "check_elements()" does
What does 
pants roots
show?
It looks like 
src/python/data-models/src
should be a source root, and if it is, then I suspect 
packages = [{include = "data_models", from = "src"}]
is tripping you up, since that is not relative to the source root (or to the repo root, I forget which of those two is relevant in this case)
You can find out more by running with 
--keep-sandboxes=on_failure
and then looking at the filesystem structure inside the logged sandbox path
You ran poetry build inside data-models, whereas Pants is running things relative to the repo root (or the source root, again, I forget if it strips source roots in this case, but you can see for yourself in the sandbox)
l
@happy-kitchen-89482 Thank you very much for your comments. So I’ve tried different things since I made this post. And think that you’re right. I removed the inner src-layout and add more 
source_roots
. It seems to work now.
h
Great!
2 Views
Open in Slack
PreviousNext
Powered by