Recently I posted issue #15543 on GitHub (the slack link was not functioning at that time) where I indicated difficulties we have with running

pants test

on a monorepo with several independent Python subprojects. In a nutshell, some of the tests for a subproject will only succeed if the subproject is pip installed (or something equivalent) -- we use e.g.

importlib.metadata.version()

to verify the installed version. The reply I got on the issue helped me somewhat further, but I am still stuck. The layout of the subprojects is as follows

src/
  python/
    my-project/
      my_project/
        BUILD
        __init__.py
    	main.py
    	other.py
      tests/
    	BUILD
        __init__.py
    	test_my_project.py
      pyproject.toml
      BUILD

We use poetry-style dependencies, documented in

pyproject.toml

. When running tests manually, we simply activate the subproject's virtual envirnment, poetry install the subproject, and run pytest. I understand that pants uses a different approach, and does not by default install the subproject in a virtual environment, only the dependencies that that subproject has. I was told that to install the subproject, I would need to create a

python_distribution

target, and have the tests depend on that. This is where I get stuck. As far as I can see from experimenting, I need to define the distribution target in the top-level BUILD file (the one under

my-project

. But I seem to be unable to declare a dependency on that target from within the BUILD file under

tests

. When I specify

runtime_package_dependencies

with that target name I specified in the top-level BUILD file, I get a ResolveError that the target was not found in the my-project/test namespace. And when I prefix the target with the full

src/python/my-project

prefix, I get a ResolveError like

file 'src/python/my-project/src/python/my-project'

does not exist. Any help in setting up the dependency between the tests and the distribution target would be greatly appreciated.

you can run

./pants list $dir:

to see the names of targets in a directory

Thanks for the hint. This weekend is a holiday weekend here in the Netherlands, and next week I'm away. But my teammates are following this thread, so hopefully there'll be some progress when I return.

My wild guess is you need to provide the path relative to pants in

runtime_package_dependencies

, if the test target is below the

python_distribution

So

runtime_package_dependencies = ["src/python/my-project:my-project-dist]

This is bit of gotcha when accessing targets which aren't in sub directories.

Thanks for the help and suggestions! I managed to make the test depend on a python distribution. However the test fails to pick up some dependencies of the python distribution. The dependencies of the test file are correctly listed by

./pants dependencies --transitive

, however

./pants test ::

fails with

ProcessExecutionFailure: Process 'Building pytest_runner.pex' failed with exit code 1.

saying it Failed to resolve some of the requirements that were visible in

./pants dependencies

see https://github.com/pantsbuild/pants/issues/15688 for more info and https://github.com/koffie/pants_example/ for a minimal github repo reproducing the bug.

I don't think you should be using

resources

https://github.com/koffie/pants_example/blob/master/src/python/my-subproject/BUILD#L4= It should be

python_sources

https://www.pantsbuild.org/docs/reference-python_sources I would avoid it completely and use the target generated under https://github.com/koffie/pants_example/blob/master/src/python/my-subproject/my_subproject/BUILD which would be called

my_subproject

and you can add it as dependency directly in dist dependencies as

./my_subproject:my_subproject

Also you don't need to add explicit dependency on

pyproject.toml

.

hi @refined-addition-53644, Thanks for the feedback. The suggestion to use

./my_subproject:my_subproject

as dependency indeed makes things a lot cleaner. The explicit dependency on

pyproject.toml

however is something we do need, it is something we directly got from the pants documentation (see pep 517 on https://www.pantsbuild.org/docs/python-distributions ), and if I remove it the distribution doesn't get build correctly since the pyproject.toml is then not included in the chroot where the distribution is build, and hence all the essential configuration for building the project is missing. You can see the result of this in the following:

pants_example maartenderickx$ ./pants package src/python/my-subproject:my_subproject_distribution
13:13:38.28 [INFO] Completed: Building build_backend.pex from setuptools_default_lockfile.txt
13:13:38.78 [INFO] Wrote dist/UNKNOWN-0.0.0-py3-none-any.whl