hallowed-church-3949905/25/2022, 12:34 PM
on a monorepo with several independent Python subprojects. In a nutshell, some of the tests for a subproject will only succeed if the subproject is pip installed (or something equivalent) -- we use e.g.
to verify the installed version. The reply I got on the issue helped me somewhat further, but I am still stuck.
The layout of the subprojects is as follows
We use poetry-style dependencies, documented in
. When running tests manually, we simply activate the subproject's virtual envirnment, poetry install the subproject, and run pytest.
I understand that pants uses a different approach, and does not by default install the subproject in a virtual environment, only the dependencies that that subproject has. I was told that to install the subproject, I would need to create a
target, and have the tests depend on that.
This is where I get stuck. As far as I can see from experimenting, I need to define the distribution target in the top-level BUILD file (the one under
. But I seem to be unable to declare a dependency on that target from within the BUILD file under
. When I specify
with that target name I specified in the top-level BUILD file, I get a ResolveError that the target was not found in the my-project/test namespace. And when I prefix the target with the full
prefix, I get a ResolveError like
does not exist.
Any help in setting up the dependency between the tests and the distribution target would be greatly appreciated.
witty-crayon-2278605/25/2022, 5:49 PM
to see the names of targets in a directory
./pants list $dir:
./pants list :
hallowed-church-3949905/25/2022, 8:15 PM
refined-addition-5364405/25/2022, 8:25 PM
, if the test target is below the
This is bit of gotcha when accessing targets which aren't in sub directories.
runtime_package_dependencies = ["src/python/my-project:my-project-dist]
worried-boots-8292605/27/2022, 5:06 PM
./pants dependencies --transitive
./pants test ::
saying it Failed to resolve some of the requirements that were visible in
ProcessExecutionFailure: Process 'Building pytest_runner.pex' failed with exit code 1.
see https://github.com/pantsbuild/pants/issues/15688 for more info and https://github.com/koffie/pants_example/ for a minimal github repo reproducing the bug.
refined-addition-5364405/27/2022, 5:27 PM
It should be
I would avoid it completely and use the target generated under
which would be called
and you can add it as dependency directly in dist dependencies as
Also you don't need to add explicit dependency on
worried-boots-8292605/30/2022, 11:22 AM
as dependency indeed makes things a lot cleaner. The explicit dependency on
however is something we do need, it is something we directly got from the pants documentation (see pep 517 on https://www.pantsbuild.org/docs/python-distributions ), and if I remove it the distribution doesn't get build correctly since the pyproject.toml is then not included in the chroot where the distribution is build, and hence all the essential configuration for building the project is missing. You can see the result of this in the following:
pants_example maartenderickx$ ./pants package src/python/my-subproject:my_subproject_distribution
13:13:38.28 [INFO] Completed: Building build_backend.pex from setuptools_default_lockfile.txt
13:13:38.78 [INFO] Wrote dist/UNKNOWN-0.0.0-py3-none-any.whl
Of course our end goal is not necessarily to have our submodules installed during testing, however we do have some tests that at least at the moment require some of our submodules to be installed in order for them to pass. We are also looking into ways to make our tests pass without the package being installed, but in our migration process to pants we were trying to first just making pants work with our code as is, and later trying optimising so that tests don't depend on an installed package.
def test_installed() -> None:
spec = importlib.util.find_spec("my_submodule")
assert spec is not None