Hey All, Had a question - We have a monorepo structure & we were earlier packaging a single distribution which worked flawlessly, now we want to package another python-distribution from the same repo but when defining another

python_distribution

in the root I am observing

AmbiguousOwnerError

exception. I am quite certain that pants can package multiple distributions from the same repo, so maybe I am approaching this wrong ? The 2 distributions namely

rl

and

data_processing

share some common library code such as quitter, protos, dao etc and the expectation here is that they get packaged in both the distributions (i.e without a single distribution being the sole owner). I think in my case I essentially want the same code to be published in multiple distributions (the 2 distributions are isolated from each other and won’t ever run together so the associated risk is mitigated) Error snippet -

AmbiguousOwnerError: Found multiple sibling python_distribution targets that are the closest ancestor dependees of src/backend/abc/libs/service/quitter.py:quitter and are therefore candidates to own it: src/backend/abc:rl, src/backend/abc:data_processing. Only a single such owner is allowed, to avoid ambiguity. See <https://www.pantsbuild.org/v2.14/docs/python-distributions> for how python_sources targets are mapped to distributions. See <https://www.pantsbuild.org/v2.14/docs/python-distributions>.

I couldn’t find a lot of information on pants docs related to

AmbiguousOwnerError

& was wondering if this is a loose requirement and maybe there is a flag to tweak it such that >=2 distributions can live together in the root ? or is there any other standard process for packaging multiple distributions. Another way I could think of is - Package the common code as separate distribution (say

common_libs

) and mark it as dependency in the final distributions i.e

rl

and

data_processing

or manual setup.py (which I am trying to avoid) PS: Fwiw, our near-future plan is to leverage the pants

python_distribution

target for packaging multiple packages.

https://www.pantsbuild.org/docs/python-distributions#mapping-source-files-to-distributions

Naively, you might think that a

python_distribution

publishes all the code of all the

python_source

targets it transitively depends on. But that could easily lead to trouble if you have multiple distributions that share common dependencies. You typically don’t want the same code published in multiple distributions, as this can lead to all sorts of runtime import issues.

If you use a handwritten

setup.py

, you have to figure this out for yourself - Pants will bundle whatever the script tells it to.

Thanks @refined-addition-53644, I went through this while trying to understand how pants work for distributions. I was wondering if folks here had any ideas/suggestion on how to go around this (while trying to keep my monorepo future-proof from any potential pitfalls)? since this is a requirement in my case 😄

I am not entirely sure and would wait for input from someone from pants core team but one thing I see in the doc is that one distribution (D1) can depend on another (D2). So in your case you can package the common file only inside one of the distribution (D1) and then let pants figure it out how to package it for D2.

The generated

setup.py

will have its

install_requires

set to include the 3rdparty dependencies of the code bundled in the distribution, plus any other distributions from your own repo. For example, if distribution D1 contains code that has a dependency on some source file S, and that source file is published in distribution D2, then D1's requirements will include a dependency on D2. In other words, Pants does the right thing.

Or if you don't want to have D1 to depend on D2, you can extract the common parts as a separate library, used by D1 and D2, and you don't have to publish it - pants will include it in D1 and D2.

Yeah, this is indeed an option. But I will have to do a lot of changes for this since in the current setup we let pants auto-infer the dependencies when packaging targets, thus it’s automatically being inferred in both the places. 🤔

If you have the internal library available to pants, it will resolve it automatically. I don't know how your configuration is, so changes might need to be done.

The internal libs (including the proto) are indeed available to pants & it is able to resolve automatically but when creating the

python_distribution

it’s throwing ambiguity error since the common libs are to be shared in both the distributions and pants doesn’t allow multiple owners for same python_source. So I am stuck now 😕

I may not be entirely accurate here, but I think that if you create a separate python_distribution target for the common parts, and refer to it from each of the other distribution, then it will work. I have a similar situation, and no problems with it.