Hi, I am trying to use Pants with a Python monorepo using FastAPI The structure of the monorepo is as:

|--- alembic/ (for migrations, multi-db, before packaging, server will have multiple db)
...

|--- adapters (3rd parties)

|--- apps
|  |--- app1
|  |   |--- routers.py
|  |   |--- models.py
|  |--- app2
---

|--- core
|  |--- common config files, helpers

|--- certs (stores .pem, .key certificates needed by 3rdparty)

|--- services
|  |--- service1
|  |   |--- routes.py (imports from app, and adds to fastapi instance)
|  |   |--- alembic/ (per service migrations)

|--- requirements/
|  |--- base.txt
|  |--- dev.txt
|  |--- prod.txt

The apps will be separated as services. I have fixed dependencies per service by using

pants dependencies --transitive

. Now I need to package docker per service. As you can see, there are lots of dependencies involved here,

pants

is giving me

#no-infer-dep

warning for every imports. I have tried to fix them manually by doing

# services/service1/BUILD

python_source(
    name="routes.py",
    dependencies=[
        "//apps/app1"
    ]
)

However, it still gives me same warning. I have followed through the documentation of Pants. I have generated lockfiles using

requirements/base.txt

. Made pex_binary inside services/service1/BUILD having dependencies of resources sourcing -> requirements, certs, apps and everything (I think I don't need to source apps and everything as resources). Unzipped generated .pex file, and saw that only those explicitly specified resources (only certs & requirements) are being taken to pex file. And using that pex file as dependency, I am building docker image. Since the pex file is not packaging all my required directories i.e., apps, adapters, etc.,

pants run services/service1/:docker

is not working. Can anyone please guide me on how to proceed?

Hi! Please show the full services/BUILD file, my guess is that the dependencies for the docker image are not specified properly. You can check those with "pants peek", to see how Pants sees the situation.

@cold-vr-15232 Sure, here is the

services/authx/BUILD

file. Right now, Somehow, I made

pants run services/authx:authx-pex

work, but docker is not working, getting

OSError: [Errno 22] Invalid argument: '/proc/8/task/8/net'

I think, I am overdoing with the dependencies by specifying every directory. Doesn't pants infer the dependencies itself?

pex_binary(
    name="authx-pex",
    script="uvicorn",
    resolve="default",
    args=["services.authx.routes:app", "--reload"],
    dependencies=[
        "//:reqs",
        "//:base",
        "//adapters",
        "//core",
        "//services/authx:authx-migrations",
        "//apps",
        "//certs"
    ],
)

resources(
    name="authx-migrations",
    sources=[
        "alembic.ini",
        "alembic/*",
    ]
)

docker_image(
    name="authx-docker",
    dependencies=[
        ":authx-pex"
    ],
    instructions=[
        "FROM python:3.11.2-slim-buster",
        "COPY services.authx/authx-pex.pex /usr/src/app/authx.pex",
        "ENTRYPOINT [\"/usr/src/app/authx.pex\"]"
    ]
)

Pants will infer Python dependencies without needing to specifiy them. Maybe your roots are not ok? That's what tells Pants where the source code is. What do you get for "pants roots"? You should see every */src directory

[source]
root_patterns = [
  "/",
]

I have only this in my root_patterns. Do I need to specify every root folder I have in this list?

Yes, all source roots need to be specified. Check the docs, they are pretty good.

Hello @cold-vr-15232 when I specify the roots as

[source]
root_patterns = [
  "/",
  "apps",
  "adapters",
  "core",
  "certs",
  "services",
  "tests",
]

All my subdirectories get populated in the root directory itself causing circular imports, and server doesn't run. However upon specifying root_patterns as

[source]
root_patterns = [
  "/",
]

running

pants run services/authx:authx-binary

which is pex_binary runs fine. The problem I am facing now is, upon packaging pex binary as docker image, it doesn't work Here's my docker image BUILD

docker_image(
    name="authx-docker",
    dependencies=[
        ":authx-binary"
    ],
    instructions=[
    	"FROM python:3.11.2-slim",
        "COPY services.authx/authx-binary.pex /bin/app/pex",
        "ENTRYPOINT [ \"/bin/app/pex\" ]",
    ]
)

Running this docker image gives error First it gives

ModuleNotFoundError: No module named 'apps.user.models'

for apps module, then, the final error is returned as

OSError: [Errno 22] Invalid argument: '/proc/8/task/8/net'

Can you please guide me, what am I missing?

If you change the roots, I think you need to adjust the import statements too. I don't know how your code structure looks like, but in my example, where we have "src" as a root, the application's structure looks like "app->src>app->module.py" and the module is imported as "import app.module". If I were to use "app" ass root, then the import would have been "import app.src.app.module" (and all the directories would need to have init.py, I think). That is confusing, and I'm not sure how Pants puts things together in the pex binary in this case. So think about each app, and service, and library as a standalone entity that just happens to sit tn the same repo as the others, and look where the python code sits, and use that as root. I use "src" everywhere to make it easy. I hope thins makes sense.

Hi @ambitious-insurance-64047 - your source roots must be set to the roots of your package namespace, (what you would expect your PYTHONPATH to contain). This also affects your import statements. So for example, do you

from apps.app1.routers import Foo

or

from app1.routers import Foo

? In the first case, your source root is

/

, in the second case it is

apps

, and so on.

Hello @happy-kitchen-89482, I have made my docker image work, it's the first root for me, just "/" since I have

from apps.app1.models import Foo

However, my docker_image dependencies look too much cluttered

docker_image(
    ...
    dependencies=[
        "//apps",
        "//adapters",
        "//core",
        "//:base", # contains root alembic migrations (alembic.ini, alembic folder)
        ":authx-binary", # pex_binary
        ...
    ]
    ...
)

So, unless I specify apps, adapters, core inside dependencies, pex & the docker image respectively, don't pick up those directories transitively. I don't know whether this is a good practice or am I missing something?