I’ve got an issue with an undeclared dependency inside Python tests. I’ve got a test with a dependency that uses an underlying C++ Package: GDCM. Before pants we would perform a

conda install gdcm

inside of tox to run our tests. Without having access to Conda an alternative I’ve found is to use

python-gdcm

instead which is a Python wrapper for GDCM. I’ve tried to attach this dependency to the tests using a `python_requirement`:

python_requirement(
    name="gdcm",
    requirements=["python-gdcm"],
)

python_tests(
    name="tests",
    sources=[
        "tests/**/test_*.py",
        "tests/test_*.py",
    ],
    dependencies=[
        ":conftest",
        ":test-resources",
        ":gdcm",
    ],
)

However this doesn’t work and

python-gdcm

doesn’t get installed into the underlying testing PEXs unless I add this snippet to one of my test files:

try:
    import gdcm # noqa
except ImportError:
    pass

How can I have the tests depend on

python-gdcm

without the attempted import? Or alternatively, how can I give Pants access to the underlying gdcm binary (for example, one that was installed by brew or conda on the host system)?

think what u r probably missing is a

module_mapping

that will let pants know that

python-gdcm

exposes the

gdcm

module. See: https://www.pantsbuild.org/docs/python-third-party-dependencies#use-modules-and-module_mapping-when-the-module-name-is-not-standard

Oh, I already have the module mapping. I just didn’t include it in the example above.

You will have to add explicit dependency if you aren’t importing it explicitly anywhere inside your code

I understand the concept of explicitly adding undeclared dependencies - but on my system this doesn’t work without adding the Try/Except ImportError block: Here is what’s returned when the Try/Except block is included:

❯ ./pants dependencies packages/example/tests/test_example.py | grep requirements
requirements:reqs#numpy
requirements:reqs#pytest
requirements:reqs#python-gdcm

And now here’s what happens when I comment it out:

❯ ./pants dependencies packages/example/tests/test_example.py | grep requirements
requirements:reqs#numpy
requirements:reqs#pytest

Everything else is the same on the BUILD file for those two examples:

python_tests(
    name="tests",
    sources=[
        "tests/**/test_*.py",
        "tests/test_*.py",
    ],
    dependencies=[
        ":conftest",
        ":test-resources",
        "requirements:reqs#python-gdcm",
    ],
)

So I think what’s happening is dependencies are attached to the “:tests” target. If you run your test like, it should work.

./pants test directory-to-tests:tests

The file’s dependencies are only being shown for the imports you have.

Unfortunately not 😞, the test still fails when calling the file directly, or by calling the

tests

target directly. But they succeed when I uncomment the try/except block.

By the way you should use

dependencies --transitive

Gotcha, I end up getting my additional dependencies from the lib with

--transitive

but the behavior is still the same: even though the

requirements:reqs#python-gdcm

target is explicitly declared and attached to the

tests

target via dependencies - it doesn’t actually get attached as a dependency unless the try/except ImportError block exists.

where do you put your module mapping?

We’ve got a bit of a different directory structure than most pants repos, but my build file @

packages/example/BUILD

looks like this:

python_sources(
    name="lib",
    sources=[
        "example/**/*.py",
        "example/*.py",
    ],
)

python_tests(
    name="tests",
    sources=[
        "tests/**/test_*.py",
        "tests/test_*.py",
    ],
    dependencies=[
        "requirements:reqs#python-gdcm",
    ],
)


python_distribution(
    name="dist",
    dependencies=[
        ":lib",
    ],
    provides=setup_py(
        name="example",
        version="0.0.0",
    ),
    generate_setup=True,
)

And my BUILD file @

requirements/BUILD

looks like this:

python_requirements(
    name="reqs",
    source="requirements.txt",
    module_mapping={
        "python-gdcm": ["gdcm"],
    },
)

I am not able to reproduce. Although I use lockfiles. https://github.com/ShantanuKumar/pants-multi-poetry/commit/4492b44330c5723dcee5e2fa31605de375e18e1a

💀 I’m having trouble reproducing on a new, minimal repo myself. Let me keep plugging on this and see if I can get it to reproduce