https://pantsbuild.org/ logo
Join Slack
Powered by
Hi everyone, what’s the way to define dependency o...
# general
r
Hi everyone, what’s the way to define dependency on a distribution from local repo which uses a different resolve. I tried to use the associated target directly but it complains
All dependencies must work with the same 
resolve
.
These two packages have their own resolve and hence different lockfiles
h
Can you post the relevant BUILD file snippets? In particular, are you depending on a 
python_sources
or on a local 
python_distribution
? If the latter, then why?
Generally the behavior you're seeing is important, it prevents incompatible requirements from clashing
If you want A to depend on B then it's important that they have compatible requirements, so they must share a resolve. Otherwise how would you know which requirements to pull in for A+B?
r
I was using the 
python_distribution
, but I switched to 
python_sources
and yeah still the same issue. 
package_b
has 
package_a
as a dependency.
Copy code
# BUILD file for package_a 
python_sources(
    name="source_a", resolve="resolve_a"
)
Copy code
# BUILD file for package_b
python_sources(
    name="source_b",
    dependencies=["src/package-a/package_a:source_a"],
    resolve="resolve_b",
)
My understanding was when I generate lockfile corresponding to 
resolve_b
it would figure out what are the compatible dependency between the two packages. Isn’t that how it works when using third party dependencies? Like when you have some third party dependency with some common underlying dependency.
h
That's not how it works. The very purpose of resolves is to say "these two things are compatible in their 3rdparty deps".
If source_b depends on source_a and they have separate lockfiles, they are very unlikely to be compatible
unless one of the lockfiles is trivial
why is it necessary to have separate lockfiles here?
r
These are two separate packages with their own set of 3rd party dependencies. We have multiple such packages in our monorepo
h
Right, but those separate sets of 3rd party dependencies can still be in the same resolve. Pants will compute the relevant subsets as needed.
If these are two separate packages with truly independent deps in separate resolves, then B might not be able to depend on A because the 3rdparty deps are incompatible. So that's the problem that named resolves solves.
It's idiomatic to have a single resolve (or a small number of them, if you truly have conflicting deps in different parts of the codebase), you don't need a separate resolve for different packages, because Pants will take the relevant subset for you when you generate wheels or pexes or whatever.
Does that make sense?
r
From pants perspective it makes sense. But I am trying to understand why can’t I have multiple packages like we have with 3rd party dependencies. I can achieve something like this if I start publishing these packages in our private pypi server. The issue with having common resolve across multiple poetry packages is that we have multiple pyproject.toml which share some common 3rd party dependencies. When you have common 3rd party dependencies pants would complain. Something like this https://pantsbuild.slack.com/archives/C046T6T9U/p1658836119916989
Has anyone else run into this kind of issue i.e. managing multiple 1st party dependencies generated using poetry? I am bit stuck here. Without moving all 1st party packages inside one huge package I don’t see any other way. I am trying to avoid doing that.
h
Well, you don't need to move all 1stparty packages into one huge package. You shouldn't have to change any of that. What I am proposing is that all your separate 1stparty packages share a single resolve, which is just a logical name for a "universe" of possible 3rdparty dependencies, expressed via a lockfile.
Each package only uses a subset of that large resolve (as determined by inferred and explicit pants dependencies), so you don't get dependency bloat in each package.
But the single resolve guarantees that all these packages have compatible dependencies (because they are all subsets of that one big universe)
It sounds like you want each 1stparty package to have its own poetry lockfile, as a separate resolve. Can you explain what the advantage of that is in your use-case? You can do that with Pants, but then you can't have a 1stparty package B depend on another 1stparty package A directly as a source dependency, because they aren't known to be compatible. What you can do instead is publish A and have B depend on it as a published distribution (as if it were 3rdparty) rather than an in-repo dependency.
But nothing guarantees that will work, unless you manually ensure that all the requirements are compatible, which is what the single resolve would do for you automatically.
3 Views
Open in Slack
PreviousNext
Powered by