Hello folks! Question about "target arch". The usecase is: we have dependency on

pytorch

which can be installed with CPU or GPU support. For tests, and having consistent local dev env, we would obviously need

CPU

. For deployment to training infra, we would need

GPU

. With my limited knowldge of 👖 I see an imperfect solution: • have tags on literally every target (because we have a lot of things depending on

pytorch

potentially transitively)

cpu

or

gpu

and duplicate those targets' code, changing only dependency on

torch_cpu

vs

torch_gpu

Question: is it possible, without messing tags, jsut somehow define a conditional

python_requirement

, say (in pseudocode)

python_requirement(
    name = "torch_1.10",
    if (option == "cpu") {
        requirements = ["torch @ <https://download.pytorch.org/whl/cpu/torch-1.10.0%2Bcpu-cp38-cp38-linux_x86_64.whl>"]
    }
    else {
        requirements = ["torch @ <https://download.pytorch.org/whl/cu113/torch-1.10.0%2Bcu113-cp38-cp38-linux_x86_64.whl>"]
    }
)

maybe using options or smth like that. So that the choice is made at the level of the initial requirement. Something similar to Bazel's toolchain hosts... Any ideas are welcome! Thanks! Alex

Hi @acceptable-football-32760 ! We're currently massively reworking how we support multiple conflicting requirements (AKA "multiple lockfiles". @hundreds-father-404 is the expert on this, so I will defer to them.

Amazing! 🤩 Would it be possible to switch the requirement of prod targets to cpu (say for local development?) or the “type” is fixed per-target? Ideally, I seek the possibility to switch the whole deployment to either of “arch”s. Similar situation would arise say in Bazel world when building beyween x64 and x86.

Good question, I will defer to Eric. Possibly you can select an "active lockfile" on the cmd line.

Thank you @happy-kitchen-89482 👍 I'm looking forward to what @hundreds-father-404 tells! Maybe for such functionality, some kind of a custom plugin is possible

Hey Alexey, welcome! Indeed, 2.10 gained support for multiple "disjoint" resolves. https://pantsbuild.slack.com/archives/C046T6T9U/p1643922115275589?thread_ts=1643921413.397219&cid=C046T6T9U We're working on a new feature that will make multiple overlapping resolves ergonomic, where

utils.py

can claim to work with both the resolve

pytorch-cpu

and

pytorch-gpu

In the meantime, I think this might work? 1. Define two resolves with

[python].resolves

,

pytorch-cpu

and

pytorch-gpu

. Set

compatible_resolves

on each

python_requirement

target to declare which requirement belongs in which resolve, including if both. 2. Change

[python].default_resolve

to the one you want. You won't need to update any targets for first-party code like

python_sources

, they'll use the default. 3. When you want to toggle to the other resolve, simply update

[python].default_resolve

. All your first party targets will use that one without needing to change any BUILD files. FYI you can set this option via env var and CLI arg, too: https://www.pantsbuild.org/docs/options How does that sound?

Looks great @hundreds-father-404 (I already started implementing conditional dependency injection in 2.9.0 😅) - does your solution with resovles work in 2.9.0 or we should wait till 2.10.0?

it is only part of 2.10.0 - I'll be releasing 2.10.0rc0 today and our focus this week will be stabilizing that into 2.10.0 https://www.pantsbuild.org/docs/release-strategy