Hi community, I am new to pants. I read the pants docs and also try to search online including asking chatGPT. But I am still struggling to solve my distribution issue. I have the following structure for my python mono repo, and the git repo is here.

.
├── pants.toml
├── src
│   └── python
│       ├── lib1
│       │   ├── __init__.py
│       │   ├── BUILD
│       │   ├── lib1.py
│       │   └── pyproject.toml
│       ├── package1
│       │   ├── __init__.py
│       │   ├── BUILD
│       │   ├── package1.py
│       │   └── pyproject.toml
│       └── package2
│           ├── __init__.py
│           ├── BUILD
│           ├── package2.py
│           └── pyproject.toml
└── test
    └── python
        └── tests
            ├── __init__.py
            └── package1
                ├── __init__.py
                ├── BUILD
                ├── pyproject.toml
                └── test_package1.py

Basically both package1 and package2 depends on lib1. I want to publish package1 and package2 as lib, but keep lib1 as internal lib which won’t publish. And the package 1 in test folder is to test package 1. I can run

pants generate-lockfiles

and

pants test test/python/tests/package1:tests

without any problem. But when I do

pants package src/python/package1:dist

, I got the following error:

% pants package src/python/package1:dist
21:35:50.43 [ERROR] 1 Exception encountered:

Engine traceback:
  in `package` goal

NoOwnerError: No python_distribution target found to own src/python/lib1/__init__.py. Note that the owner must be in or above the owned target's directory, and must depend on it (directly or indirectly). See <https://www.pantsbuild.org/2.25/docs/python/overview/building-distributions> for how python_sources targets are mapped to distributions. See <https://www.pantsbuild.org/2.25/docs/python/overview/building-distributions>.

Could you pls let me know what the problem is and how to fix it? Also I am not quite sure for lib1, in package1 or package2, should I put lib1 dependency in the BUILD file as my current code, or I can put it in the pyproject.toml like below (tested, not work. Not sure if it is path issue)?

dependencies = [
  "tornado==6.4.2",
  "lib1 @ file:///${PROJECT_ROOT}/../lib1"
]

Pls help. thank you!

I don’t understand how this could work… How will the published versions of package1 and package2 consume lib1 if it’s not also published?

Hi, thanks @happy-kitchen-89482 . I want to use lib1 as internal lib for this repo, not want to publish it. In other word, I want it to be embedded inside package 1 and 2. How can we achieve this?

So for that you’ll have to write your own setup.py

I came from Java world and not an expert with python. So how we usually do in python if two places needs to reuse the same code like this case? And this piece of code is intended to be within this repo, and not want to be published.

I don’t see how this can be a common case in any language. If you’re publishing package1 then you must publish lib1 somehow, so that package1 can consume it. No? It can’t be “within this repo” if some published code, that may be used by someone with no access to your repo, is planning to use it, surely?

You are correct from accessibility perspective. But from code reusability, it is common to have some internal lib that needs to be embedded in both packages 1 and 2, and we do not want to release this lib which contains repo specific code (some special utility functions or classes only for this repo). So in this case, the way is to manually write setup.py?

I mean, effectively you are releasing it - twice - no? There is nothing to stop people using it from inside either package1 or package2 except that you’re not documenting it

yes, you are right. It is published twice as source code. I got what you are saying. I am not familiar of how to set it up using pyproject.toml setting. Do you mind if you give me some hint or some example? Thanks a lot!

Ah, so now you have to learn the wonderful world of Python packaging… This isn’t a Pants thing, and there are many resources online for this

I read some doc last night. I just feel python is very easy to get package or module collision. It is not like Java, which usually has much more verbose but reliable packaging naming convention, such as org.apache.commons.io.xxx

That’s true, there are no conventions for Python package naming, so it is wise to pick something unique. The reverse domain thing Java recommends is effective, and I wish Python had something similar. But you can still make good choices for your own packages, at least.