I'm trying to get my head around resolves, as a wa...
# general
w
I'm trying to get my head around resolves, as a way to split out large/complex dependencies (pytorch, et al) from the wider monorepo. https://www.pantsbuild.org/blog/2022/05/25/multiple-lockfiles-python was helpful, as was https://www.pantsbuild.org/2.20/docs/python/overview/lockfiles Am I right in thinking I can define a single requirements source (requirements.txt, or pyproject.toml) and use resolves to create subsets of the requirements? Essentially something like two resolves of
default
and
science
, and only loading science dependencies in source/builds that explicitly opt in to that resolve, and keep the default resolve as lean as possible?
d
Note that Pants infers dependencies, so it will pull only what the sources actually use (by analyzing import statements), therefore even with a single resolve and one monster requirements.txt, the resulting artifacts will end up as lean as possible.
Resolves are useful for more complex scenarios, where you have certain parts of monorepo needing different versions of the same library, or sorting out Pytorch challenges (separate resolve for cpu-only).
w
Gotcha. That makes sense with a bit more reading. Having dug into this more, I think what I want is closer to
--exclude
in PEX, to ignore large/complex dependencies that'll be provided elsewhere during deployment. In our case it's
pytorch
, which is enormous (especially with
cuda
deps). Requiring that to be installed and packages in dev/CI is really time consuming and complex. In production want plan to provide `pytorch`/`cuda` via a docker layer but ideally not actually install/load it,