Hi all. I have the following root_patterns defined in my project:

[source]
root_patterns = [
  "/shared/delt",
  "/dags/dag_capstone",
  "/dags/dag_basic",
  "/dags/dag_simple",
  "src",
  "tests"
]

Executing

pants roots

yields:

dags/dag_basic
dags/dag_basic/src
dags/dag_basic/tests
dags/dag_capstone
dags/dag_capstone/src
dags/dag_capstone/tests
dags/dag_simple
dags/dag_simple/src
dags/dag_simple/tests
shared/delt
shared/delt/src
shared/delt/tests

In 'shared/delt', I have the following BUILD:

poetry_requirements(
    name="poetry",
)

resources(name="pyproject", sources=["pyproject.toml", "README.md"])

python_sources(name="lib", sources=["src/delt/**/*.py"])

python_distribution(
    name="dist",
    dependencies=[":pyproject", ":lib", ":poetry"],
    generate_setup=False,
    wheel=True,
    sdist=True,
    provides=python_artifact(name="delt"),
)

I have a test file ('shared/delt/tests/test_api.py') with the following imports:

import datetime as dt

import pandas as pd
from delt.api import _join_endpoint_to_base_url, get_results_luchtmeetnet_endpoint

The 'delt' library is a poetry project that is stored in 'shared/delt/src/delt/...' When I run

pants test shared/delt/tests

, I get the following error:

==================================== ERRORS ====================================
________________ ERROR collecting shared/delt/tests/test_api.py ________________
ImportError while importing test module '/private/var/folders/bw/mm734qgd40g3tb30ncw5gwvc0000gp/T/pants-sandbox-SikTI8/shared/delt/tests/test_api.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/Users/user/.pyenv/versions/3.9.5/lib/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
shared/delt/tests/test_api.py:4: in <module>
    from delt.api import _join_endpoint_to_base_url, get_results_luchtmeetnet_endpoint
E   ModuleNotFoundError: No module named 'delt'
- generated xml file: /private/var/folders/bw/mm734qgd40g3tb30ncw5gwvc0000gp/T/pants-sandbox-SikTI8/shared.delt.tests.test_api.py.xml -
=========================== short test summary info ============================
ERROR shared/delt/tests/test_api.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 1.38s ===============================



✕ shared/delt/tests/test_api.py failed in 2.21s.

However,

pants package shared/delt:dist

runs fine with this configuration. When I change the BUILD file to:

poetry_requirements(
    name="poetry",
)

pants test shared/delt/tests

runs without errors. However, this breaks

pants package shared/delt:dist

because that's no longer defined.

pants test ...

seems to break because of the line

python_sources(name="lib", sources=["src/delt/**/*.py"])

Can someone explain this behavior? Clearly something is going wrong with the roots, but I'm a little stuck. I don't see why adding

python_sources(name="lib", sources=["src/delt/**/*.py"])

to the BUILD file would break tests. Thanks in advance!

Sorry for the trouble! Do you have other BUILD files in subdirectories of delt, if so, what’s in them?

Hi Huon. That's OK. I'm really happy with pants 😄. in 'shared/delt/src/delt' there's a BUILD file (placed there by

pants tailor

) that contains

python_sources()

. There's also a BUILD file in 'shared/delt/src/delt/IO' (also containing

python_sources()

) Interestingly enough, I just got everything to work by adding

python_sources(sources="**/*.py")

to 'shared/delt/src/delt/BUILD' and changing 'shared/delt/BUILD' to:

poetry_requirements(
    name="poetry",
)

resources(name="pyproject", sources=["pyproject.toml", "README.md"])

python_distribution(
    name="dist",
    dependencies=[":pyproject", ":poetry", "shared/delt/src/delt:delt"],
    generate_setup=False,
    wheel=True,
    sdist=True,
    provides=python_artifact(name="delt"),
)

Now both

pants test

and

pants package

succeed. I don't quite understand why this works.

Ah okay. I think previously the Python files were owned by multiple targets, and Pants was getting confused (potentially with poor diagnostics 😞 was there any warnings about imports / inference when it wasn’t working?). In particular, code like

import delt

ends up being a bit ambiguous: which

delt

files is it trying to import? By ensuring there’s only one

python_sources

referring to each file, there’s no such ambiguity.

Actually, what I said earlier is still not correct. I now use the following distribution in 'shared/delt/BUILD'

python_distribution(
    name="dist",
    dependencies=[":pyproject", ":poetry", "shared/delt/src/delt:delt"],
    generate_setup=False,
    wheel=True,
    sdist=True,
    provides=python_artifact(name="delt"),
)

And I changed 'shared/delt/src/delt/BUILD' back to:

python_sources()
# Instead of python_sources(sources=["**/*"])

The previous solution was creating new issues, I suspect precisely because of what you mention. Anyway, using the above notation works 🙂.