I'm having a couple Python dependency issues when ...
# general
r
I'm having a couple Python dependency issues when running
pants test ::
that I'm struggling to resolve. Most of my tests pass fine but a handful are failing due to missing dependencies. I'm also getting complaints that pants cannot infer an owner for
pytest
- for this repo (and all of our existing repos) we maintain a regular
requirements.txt
and then a separate list that also includes dependencies necessary for local testing. Is it possible to set up something similar in pants? I have
extra_requirements
and a
lockfile
specified under my
[pytest]
header in my
pants.toml
.
b
I think if you have explicit imports of
pytest
, you need it be loaded into the resolve that runs tests (i.e. included in a
python_requirements()
call). That is, e.g.
python_requirements(name="dev", source="dev-requirements.txt")
. Pants fine-grained dependency tracking will generally that these "dev" deps won't be included in packaged code (unless there's an
import pytest
in that 'real' code). This does annoyingly mean you end up with a duplicated pytest (and potentially plugin) definitions. I believe this is being resolved in an upcoming release. https://github.com/pantsbuild/pants/issues/16044 and https://github.com/pantsbuild/pants/issues/12449 are related
For missing dependencies, more specifics will be required for deeper insight. IME, it usually comes up due to dynamic dependencies, e.g. using
sqlalchemy
to access a Postgres database requires a postgres driver like
asyncpg
, but
sqlalchemy
doesn't literally depend on that package, so we declare:
Copy code
python_requirements(
    source="exported_requirements.txt",
    overrides={
        "sqlalchemy": {"dependencies": ["backend#asyncpg", "backend#greenlet"]},
    }
)
which means any
import sqlalchemy
will have pants include the
asyncpg
package too. Alternatively, this can be set on a more fine-grained basis (e.g. if one particular file has an extra dynamic dependency, add the override to the
python_sources(...)
target itself). Does that sound relevant?