Hi, i have a package that uses a pyproject.toml file, and i’d like to run tests on it. So first i define the requirements, the sources, and i can build the distribution. But when i run the tests, the interpreter doesn’t use the requirements i gave in the pyproject.toml file. I suppose it’s because Pants infer the dependencies, but i think i overwrite them in python_tests though.

$ pants test ::
08:44:52.65 [INFO] Initialization options changed: reinitializing scheduler...
08:44:56.46 [INFO] Scheduler initialized.
08:44:58.39 [INFO] Completed: List contents of artifacts produced by src/ml:ml_distribution
08:44:59.56 [INFO] Completed: Building local_dists.pex with 1 requirement: mydist-0.0.1-py3-none-any.whl
08:45:00.45 [INFO] Completed: Building pytest_runner.pex
08:45:00.45 [ERROR] 1 Exception encountered:

Engine traceback:
  in `test` goal

ProcessExecutionFailure: Process 'Building pytest_runner.pex' failed with exit code 1.
stdout:

stderr:
Failed to resolve requirements from PEX environment @ /tmp/pants-sandbox-0jSube/local_dists.pex.
Needed cp312-cp312-manylinux_2_39_x86_64 compatible dependencies for:
 1: setuptools
    Required by:
      mydist 0.0.1
    But this pex had no ProjectName(raw='setuptools', validated=False, normalized='setuptools') distributions.
 2: numpy
    Required by:
      mydist 0.0.1
    But this pex had no ProjectName(raw='numpy', validated=False, normalized='numpy') distributions.
 3: torch
    Required by:
      mydist 0.0.1
    But this pex had no ProjectName(raw='torch', validated=False, normalized='torch') distributions.

My BUILD file at the package root looks like this :

python_requirements(name="ml_requirements", source="pyproject.toml", resolve="python-ml")
python_sources(name="lib", sources=["src/**/*.py"], resolve="python-ml")

resource(name="pyproject", source="pyproject.toml")
python_distribution(
    name="ml_distribution",
    provides=python_artifact(name="mydist"),
    dependencies=[":pyproject", ":lib"],
    generate_setup = False,
)

python_tests(name="tests", sources=["tests/**/test_*.py"], dependencies=[":ml_distribution", ":ml_requirements"], resolve="python-ml")

What is your output for

pants dependencies --transitive :ml_distribution

?

For now my package is just a placeholder so that’s why there are not a lot of files :

3rdparty/python/ml.lock:_python-ml_lockfile
src/ml/pyproject.toml:ml_requirements
src/ml/src/ml_datasets/__init__.py:../../lib
src/ml/src/ml_datasets/placeholder.py:../../lib
src/ml/src/ml_production/__init__.py:../../lib
src/ml/src/ml_production/train.py:../../lib
src/ml/src/ml_training/__init__.py:../../lib
src/ml:ml_requirements#numpy
src/ml:ml_requirements#torch
src/ml:pyproject

I'm having a similar issue. Seems like there is a bug related to

python_distribution

and running tests: https://github.com/pantsbuild/pants/issues/15801

For some reason, if i put my dependancies inside the main dependancy area, it failed as above : pyproject.toml

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "mydist"
version = "0.0.1"
description = ""
readme = "README.md"
requires-python = ">=3.12"

dependencies = [
    "setuptools",
    "numpy",
    "torch",
    "pytest",
]

But if i put all my dependancies inside the optional category :

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "mydist"
version = "0.0.1"
description = ""
readme = "README.md"
requires-python = ">=3.12"

dependencies = [

]

[project.optional-dependencies]
test = [
    "setuptools",
    "numpy",
    "torch",
    "pytest"
]

Then the tests run fine 🤔 I saw here that the optional dependancies are used as a tag : https://www.pantsbuild.org/2.20/reference/targets/python_requirements Is there a better way to fix this issue than putting dependancies as optional ? Is this related to the

--intransitive

argument and is there a way to overwrite it somehow ?