Hi. Sorry for another question. Hopefully this one should be easy. Pants appears to be ignoring the

install_from_resolve

setting for pytest. I have the following in project.toml:

[python]
interpreter_constraints = [ ">=3.10" ]
enable_resolves = true
default_resolve = "default"

[python.resolves]
default = "3rdparty/python/default.lock"
torch-cpu-test = "3rdparty/python/torch-cpu-test.lock"
[pytest]
install_from_resolve = "torch-cpu-test"

However this will error with

stderr:
Failed to resolve compatible artifacts from lock 3rdparty/python/default.lock for 1 target:
1. /usr/bin/python3.10:
    Failed to resolve all requirements for cp310-cp310-manylinux_2_35_x86_64 interpreter at /usr/bin/python3.10 from 3rdparty/python/default.lock:

which is weird because default.lock is irrelevant Setting

default_resolve = "torch-cpu-test"

seems to fix it, which makes me believe that the

install_from_resolve

is being ignored.

cc @happy-kitchen-89482 something you recognize?

can you show the rest of the stderr (which requirements it is not finding)?

The pytest resolve is only used for running the test, not any dependencies of the tested code.

16:04:26.96 [WARN] Pants cannot infer owners for the following imports in the target mlcore/models/cnn/litehrnet.py:

  * numpy (line: 11)

These imports are not in the resolve used by the target (`default`), but they were present in other resolves:

  * numpy: 'torch-cpu' from 3rdparty/python:default#numpy@resolve=torch-cpu, 'torch-cpu-test' from 3rdparty/python:default#numpy@resolve=torch-cpu-test

there are many lines of missing dependencies as default has very little in it. Here's an example of numpy not being available. It's here that you can again see it's looking in default not the

install_from_resolve

I have an idea, but let's go back one step. Why did you arrive with multiple resolves in the first place? The easiest solution is to just have a single resolve, which you've called default, which contains all your application dependencies. (i.e., excluding tools like pytest, black, ... -- things called dev-dependencies in other tools.) Re your other questions...

pytest

is just a program, from the perspective here. It doesn't have to have any relation to the code-under-test. What Pants does functionally is build one "venv" with pytest, and one "venv" with the code to test.

I'm installing torch, which has two versions, one for cpu one for gpu. I need to use different resolves for the two cases to ensure i build for gpu and non-gpu machines. Additionally, i was intending to have a resolve for dev-dependencies separately ontop of that. It sounds like the separation of dev-dependencies is not your recommendation? but i don't see a way around the cpu/gpu needs.

Ah, welcome the Torch-is-a-Pain club! When you say two versions, do you mean with

+cpu

and

+cu118

f.ex.? Do you also want to support mac?

Additionally, i was intending to have a resolve for dev-dependencies separately ontop of that.

you don’t need to keep that distinction with pants. It’s clever enough to not include any requirements you may have in your resolve that are not used by production code.

I'm popping away for dinner etc, but figured I could summarize how we've set it up. We have three resolves: •

gpu

which currently is

+cu118

-- used for local dev/algorithm development, and obviously cloud •

cpu

which currently is

+cpu

-- used for tests and Linux users without GPUs •

base

, which explicitly forbids either (

torch==1.x,!=1.x+cu118,!=1.x+cpu

). Since neither of the two above support mac, we need this for those developers. Each contains exactly the same dependencies except for torch and torch-related deps. We tried a few different setups for selecting resolves, but for now we do this:

__defaults__({
    (python_sources,python_tests): dict(resolve=parametrize("cpu", "gpu", "base))
})

Which in effect means any run or test has three variants:

pants test path/to/test.py@resolve=cpu
pants test path/to/test.py@resolve=gpu
pants test path/to/test.py@resolve=base

And same for

run

, etc. This ended up being easier to teach than using

--default-resolve

etc. Then we use filters (

pants test --filter-adress-regex='.*@resolve=cpu ::

) to only run the CPU variants or others when we do globbing.

Thanks for your answers! yes i think this is what we have setup as well. I wasn't aware of the

@resolve=cpu

syntax and can't get that to work for mine.

You'll need to combine it with parametrize, that's what generates the different targets.

Thank you! Will give it a try now 👍

Ah no, the

@resolve=cpu

goes on a single target. Unfortunately. If you want to filter with a glob-spec like

::

you need

--filter-adress-regex='.*@resolve=cpu'

.

Matt, do you plan to run tests against several resolves for the same

test

file?

Mostly my use case will be for running either the cpu OR the gpu tests at a given time. I'll try that glob based command. Thanks again for you help guys! 🙏