IF I parametrize

python_resolves

, I get the following error

ValueError: Only fields which will be moved to generated targets may be parametrized, so target generator path/to/proto/target (with type protobuf_sources) cannot parametrize the {'python_resolve'} field.

Indeed seems that

python_resolve

fields are installed via

register_plugin_field

and are not immediately moveable...

I think, after going down the path of multiple resolves for introducing cpu/gpu target arch for Torch, I eventually hit a roadblock. So what I did: • ✅ I created

***_custom

macros allowing me to automatically parametrize

python_sources

and

python_requirements

generators

def python_sources_custom(**kwargs):
    kwargs.pop("resolve", None)

    python_sources(resolve=parametrize("gpu", "cpu"), **kwargs)

• ✅ For torch requirement itself, I created two versions of it

python_requirement(
    name = "torch-cpu",
    requirements = ["torch @ <https://download.pytorch.org/whl/cpu/torch-1.10.2%2Bcpu-cp39-cp39-linux_x86_64.whl>"],
    resolve = "cpu",
)

# same for `gpu`

• ✅ Minor hiccup happened with the python generated grpc stubs where I had to create two

protobuf_sources

manually

protobuf_sources(
    name = "protos-cpu",
    grpc = True,
    python_resolve = "cpu",
)

# same for `gpu`

• 🛑 here happens the problem: since some of my

.proto

s include other protos, let's say

main.proto

includes

inc.proto

via

proto3

s

import "path/to/inc.proto"

directive, I started getting a warning:

15:17:12.27 [WARN] The target //path/to/main.proto:protos-cpu imports `path/to/inc.proto` in the file path/to/main.proto, but Pants cannot safely infer a dependency because more than one target owns this file, so it is ambiguous which to use: ['//path/to/inc.proto:protos-cpu', '//path/to/inc.proto:protos-gpu'].

In general, after many attempts to work out the "get torch of either cpu or gpu and make it a global switch" in various flavours (I happily lived with a custom generator for half a year until we introduced lockfiles to increase CI stability, now I made several attempts to solve it using multiple locks and resolves) it seems that the resolves mechanism does not accommodate this usecase well - solving each layer reveals more manual work on the next one and the problems pop up deeper. I wonder if I'm doing something fundamentally wrong, because the task of having a) "all packages except one to be the same" and b) "switch that single one depending on some environment, e.g. dev/prod" -- should be relatively easy to accomplish, yet I fail miserably 😅, meaning I likely don't wrap my mind around something simple here 🙈 Please help, would be a pity to just force everything to use GPU torch even for CI tests! 🙏