Hello

pants

developers 👋, I would like to get rid of the following WARN messsages when I run tests:

The target libs/image_processing/kaiko/image_processing/mask/statistics.py:../../../kaiko-image-processing imports `numpy`, but Pants cannot safely infer a dependency because more than one target owns this module, so it is ambiguous which to use: ['libs/geometry:kaiko-geometry-poetry-reqs#numpy', 'libs/geometry:kaiko-geometry-reqs#numpy', 'libs/image_processing:kaiko-image-processing-reqs#numpy'].

My project structure is as follows:

├── BUILD
├── libs
│   ├── geometry
│   │   ├── BUILD
│   │   ├── pyproject.toml
│   │   └── requirements.txt
│   └── image_processing
│       ├── BUILD
│       ├── pyproject.toml
│       └── requirements.txt
└── pants.toml

And my

root

pattern is:

[source] # <https://www.pantsbuild.org/docs/source-roots>
marker_filenames = ["pyproject.toml"]

My

libs/image_processing/BUILD

file is as follows:

python_requirements(name="kaiko-image-processing-reqs")

python_sources(
    name="kaiko-image-processing",
    sources=["kaiko/image_processing/**/*.py"],
    dependencies=[
        ":kaiko-image-processing-reqs#opencv-python",
        "libs/geometry:kaiko-geometry",
    ],
)

I could list in

dependencies

all external dependencies, like I did for

opencv-python

but my developers push back quite hard on boilerplate, and this will make my life to integrate

pants

harder. Is there a workaround so that

pants

doesn't go looking for dependencies higher up than the enclosing root? I would like it picks up

'libs/image_processing:kaiko-image-processing-reqs#numpy'

without further ado here. (edited)

I ran into this problem yesterday actually. It's because you have a

pyproject.toml

AND a

requirements.txt

. I removed the

requirements.txt

file to fix it. Would that work for you?

Damn, no that wouldn't because we use

requirements.txt

for provisioning local development sandboxes, but

pyproject.toml

for deployment 😕

No worries - You could remove the

reqs

lines in the BUILD file, which would fix it at least until you

tailor

again. There is probably a better solution - as my experience with pants (the build system) is limited to the last few days

When you have multiple requirements like this, you need to separate them into their own dedicated resolves. Then for your source targets, you specify which resolve they should use (may be multiple using

parametrize

). See: https://www.pantsbuild.org/docs/python-third-party-dependencies#warning-multiple-versions-of-the-same-dependency https://www.pantsbuild.org/docs/targets#parametrizing-targets https://www.pantsbuild.org/docs/targets#field-default-values <- is useful if you need to set the resolve for a whole subtree in your project. https://www.pantsbuild.org/docs/reference-python#enable_resolves

Thanks @curved-television-6568, I'll try your suggestions 👍 Regarding not declaring the dependencies, I don't see how

pants

could decide which dependencies to pick up (between

requirements.txt

or

pyproject.toml

. We want to use the

requirements.txt

when running tests for example, but not when packaging (

requirements.txt

is for sandboxes, but not for distribution).

Cool. So to answer the last question there, you’d provide one resolve to your

python_tests

targets, matching the resolve for your

python_requirements

target you want to use, then for your

python_distribution

you include your

python_sources

with the parametrized version that uses the resolve you want to use for packaging, which will need to be parameterized on resolve like:

python_sources(…, resolve=parametrize("a", "b"), …)

where “a” and “b” are your resolve names. https://www.pantsbuild.org/docs/reference-python_tests#coderesolvecode

I see, awesome, will try it out 👍

You definitely can use multiple resolves, as Andreas mentioned. But one thing Pants enables, and is often simpler, is to have a single universe of requirements for your entire repo (often expressed in a single global requirements.txt or similar). Pants then automatically uses the relevant subset of those requirements in each situation (e.g., when running a test or packaging a pex). The advantage of this is that you get compatibility across the codebase, and there is a lot less manual management of requirements.

Hello Benjy, funnilly this "global requirements.txt" was the initial state of our monorepo, but all developers except myself expressed that they'd rather have smaller sandboxes and hence multiple requirements.txt files.

Ah, Pants creates those smaller sandboxes for you!

But how would different projects have different (sometimes conflicting) requirements then?

It’s definitely a different way of thinking about Python requirements than the standard way