Hey everyone. I'm relatively new to pants. Recently joined a team that has implemented pants on our monorepo. One of our project is a django app and I'm trying to add psycopg-pool to the depedencies. In our pyproject.toml I have done that with:

psycopg = {extras = ["binary", "pool"], version = "^3.2.3"}

. This works great when I run the app, both when using

pants run ...

and using the venv provided by

pants export

. However, most of our tests now fail because it doesn't seem to have the psycopg-pool package. I have absolutely no clue how this can happen. This is the BUILD.pants in the test folder:

python_tests(
    name="tests",
    resolve="cell-manager",
    dependencies=[
        ":data",
        "src/applications/cell_manager:deps",
        "src/applications/cell_manager/service:django_settings"
    ]
)

resources(
    name="data",
    sources=["data/*"],
)

and this is the BUILD.pants file that creates ``src/applications/cell_manager:dep`

# Copyright 2022 Airborne Composites Automation.

poetry_requirements(
    name="cell_manager_requirements",
    resolve="cell-manager",
    module_mapping={
        "django-storages": ["storages"],
        "django-cors-headers": ["corsheaders"],
        "opencv-python-headless": ["cv2"],
        "pytest-lazy-fixture": ["pytest_lazyfixture"],
        "django-json-widget": ["django_json_widget"],
        "django-reverse-admin" : ["django_reverse_admin"],
        "django-test-migrations" : ["django_test_migrations"],
        "django-unicorn" : ["django_unicorn"]
    },
)

python_sources(
    name="deps",
    resolve="cell-manager",
    dependencies=[
        ":cell_manager_requirements",
        "./static",
        "./logs",
        "./templates",
        "./locale",
        "./config:config",
        "./service",
        "./service:django_settings",
    ],
)

pex_binary(
    name="manage",
    resolve="cell-manager",
    entry_point="manage.py",
    dependencies=[
        ":deps",
    ],
    restartable=True,
)

pex_binary(
    name="uvicorn",
    resolve="cell-manager",
    entry_point="uvicorn",
    dependencies=[
        ":deps",
    ],
)

pex_binary(
    name="celery-worker",
    resolve="cell-manager",
    entry_point="celery",
    dependencies=["./service:django_settings",]
)

I've tried a few things with the module mapping and and overrides, but can't seem to get it to work. But I guess that might not be needed since it does work for non-test scenario.

Have you tried setting

psycopg-pool

as an override dependency of

psycopg

inside your

poetry_requirements

? Here’s the docs, along with an example of what that looks like from my own BUILD file:

overrides={
        "instructor": {
            "dependencies": [":poetry#openai", ":poetry#anthropic", ":poetry#cohere"]
        },
        "logfire": {
            "dependencies": [
                ":poetry#opentelemetry-instrumentation-aiohttp-client",
                ":poetry#opentelemetry-instrumentation-fastapi",
                ":poetry#opentelemetry-instrumentation-httpx",
                ":poetry#opentelemetry-instrumentation-psycopg",
                ":poetry#opentelemetry-instrumentation-psycopg2",
                ":poetry#opentelemetry-instrumentation-redis",
                ":poetry#opentelemetry-instrumentation-sqlalchemy",
            ]
        },
        "lancedb": {"dependencies": [":poetry#tantivy"]},
        "sqlalchemy": {"dependencies": [":poetry#asyncpg", ":poetry#greenlet"]},
        "slack-sdk": {"dependencies": [":poetry#aiohttp"]},
    },

This way, you can force the inclusion of

psycopg-pool

wherever

psycopg

is depended on.

Thanks Krishnan, I have changed the poetry requirements to the following:

poetry_requirements(
    name="cell_manager_requirements",
    resolve="cell-manager",
    module_mapping={
        "django-storages": ["storages"],
        "django-cors-headers": ["corsheaders"],
        "opencv-python-headless": ["cv2"],
        "pytest-lazy-fixture": ["pytest_lazyfixture"],
        "django-json-widget": ["django_json_widget"],
        "django-reverse-admin" : ["django_reverse_admin"],
        "django-test-migrations" : ["django_test_migrations"],
        "django-unicorn" : ["django_unicorn"]
    },
    overrides={
        "psycopg": {"dependencies": [":cell_manager_requirements#psycopg-pool", ":cell_manager_requirements#psycopg-pool"]}
    }
)

I then get the following error:

ValueError: The explicit dependency `src/applications/cell_manager:cell_manager_requirements#psycopg-pool` of the target at `src/applications/cell_manager:cell_manager_requirements#psycopg` does not provide enough address parameters to identify which parametrization of the dependency target should be used.
Target `src/applications/cell_manager:cell_manager_requirements` can be addressed as:

which then lists all my poetry requirements, without the psycopg-pool package

"psycopg": {"dependencies": [":cell_manager_requirements#psycopg-pool", ":cell_manager_requirements#psycopg-pool"]}

Why is the

psycopg-pool

package included twice here in the dependencies? Additionally, are you using parametrization over multiple resolves / multiple other values?

Ah sorry, one of them should have been psycopg-binary`

Are you still getting the same failure? If so, do you see a target in the error message that might correspond to

psycopg-binary

or

psycopg-pool

?

Yep same error, neither of them show up in the list

I think this issue is related: https://github.com/pantsbuild/pants/issues/20706 To get around it, you might have to add

psycopg-binary

and

psycopg-pool

as explicit dependencies in your

pyproject.toml

file

Yeah I think that has worked