qq re requirements.txt. I'm trying to compile MacO...
# general
a
qq re requirements.txt. I'm trying to compile MacOS and Linux torch requirements to the same requirements.txt. I'd like to have something like
Copy code
torch==1.13.1; platform_system == "Linux" --index-url <https://download.pytorch.org/whl/cu116>
torchvision==0.14.1+cu116; platform_system == "Linux" --index-url <https://download.pytorch.org/whl/cu116>
torch==1.13.1; platform_system == "Darwin"
torchvision==0.14.1; platform_system == "Darwin"
but when running
pants generate-lockfiles
the
--index-url
is ignored. On the other hand, if i were to add the index globally via
pants.toml
, the macos installation breaks, because for some reasons the torch in https://download.pytorch.org/whl/cu116 is not compatible with macos. any ideas how I would solve this?
g
At my work, we use two different resolves, one with cuda and one without. (Actually three, we have one that is only CPU as well, for CI)
a
Interesting. Does that mean that you need to explicitly mention all the resolves on each
python_sources
/
python_tests
macros?
g
We use
__defaults__
to automate it, but in effect yes.
Copy code
# src/py/BUILD
__defaults__(
    {
        python_source: dict(resolve=parametrize("cpu", "gpu", "reqs")),
        python_sources: dict(resolve=parametrize("cpu", "gpu", "reqs")),
    }
)
We also have CLI aliases to help make this more ergonomic:
Copy code
# pants.toml
[cli.alias]
run-gpu = "--python-default-resolve=gpu run"
run-cpu = "--python-default-resolve=cpu run"
🙏 1
It isn't my desired workflow, but I haven't had time to fix it properly. See f.ex. this issue which would fix what you're trying to do, and this general tracking issue.
a
very helpful, thank you!