https://pantsbuild.org/ logo
Join Slack
Powered by
Context: I’m building a ML service and it has many...
# general
m
Context: I’m building a ML service and it has many complicated deps. I consider it somewhat of a miracle I got pants to lock — mainly due to the package 
xformers
; I got around it by pinning it to a specific wheel in pyproject.toml I have an exported 
symlinked_immutable_virtualenv
that I generally use for my IDE. I just noticed that 
xformers
never got installed (from wheel) even though it’s there in the lock. I just re-exported it as 
mutable_virtualenv
and 
xformers
seems to be there (yay) Inquiries: 1/ Is the different outcomes between the export-flags, a feature or a bug? 2/ The reason xformers is a pain to build is because of its build-time dependency on torch. I don’t know if there’s a good way to handle this with pants. I know we can specify runtime-overrides but since this is at build-time, I believe is irrelevant. I’ve also tried to leverage pyproject.toml 
[build-system.requires]
to include torch, without success.
Steps to repro: My BUILD looks like:
Copy code
python_requirements(
    name="ml-service",
    source="path/to/pyproject.toml",
    resolve="ml",
)
My pyproject.toml looks like:
Copy code
dependencies = [
    "torch==2.2.2",
    "torchaudio==2.2.2",
    "torchvision==0.17.2",
    "xformers @ <https://download.pytorch.org/whl/cu118/xformers-0.0.25.post1%2Bcu118-cp311-cp311-manylinux2014_x86_64.whl>",
]
Lock & export
Copy code
pants --pex-verbosity=9 generate-lockfiles --resolve=ml 
pants export --py-resolve-format=symlinked_immutable_virtualenv --resolve=ml
pants export --py-resolve-format=mutable_virtualenv --resolve=ml
h
Hi! Could you create a github repo that reproduces this, and whose README lists the steps to reproduce (including what you expect to happen vs what actually does)? Thanks
6 Views
Open in Slack
PreviousNext
Powered by