Is there a way I can just declare a whole python directory/module, including its submodules as a dependency? Pants does not seem very good at the dependency inferring with our Django projects, and I don't want to have to list out every single submodule as a dependency to get an app to run

Like, as a glob?

python_sources(
    name="lib",
    sources=["**/*.py"],
)

As SJ says, I think if you define a target with a glob then

dependencies=[":lib"]

will depend on all of those files There's not a way to glob over targets, where there's separate

python_sources()

targets and one can write

dependencies=["path/to/**:*]

(or some other hypothetical syntax). (BTW, we're happy to help you get the dependency inference working more smoothly if you'd like, but no obligation of course!)

That glob isn't working unfortunately. I keep getting errors along the lines of

DifferingFamiliesError: Expected AddressMaps to share the same parent directory

Re: uselessness - you're referring to this example? https://github.com/pantsbuild/example-django

I've got several Django apps currently living under

src/

, i.e.

src/app1

src/app2

etc. Some of these are split futher like

src/app1/core

,

src/app1/admin

I've then got a separate dir atm for web services like

service/web_admin

. These contain my settings files with

INSTALLED_APPS

, which Pants cannot read so I'm having to manually specify the apps as dependencies and I'm having trouble working out the right way to do it. I've tried specifying the directory like

src/app1

, and I've tried adding the individual

apps.py

files like

src/app1/core/apps.py

, which I've then overriden the dependencies of again to specify

src/app1/core/migrations

as a dependency of

src/app1/core/apps.py

. This is all in an attempt to get

manage.py migrate

to actually run the migrations it's supposed to, but it usually fails saying a particular app isn't installed, which means Django couldn't find the AppConfig class for it

What do your BUILD files look like?

So in src/app1/core I've got

python_sources(
  dependencies=[
    './migrations',
  ],
)

In the migrations dir I also have this to include a directory of images, but I think that's by-the-by for my issue

python_sources(
  dependencies=[':files'],
)

resources(
  name='files',
  sources=['files/**/*'],
)

And right now the web service BUILD looks something like this

python_sources(
    name='lib',
    dependencies=[
        '//:reqs#django-cachalot',
        'src/barristan/apps.py',
        'src/app1/admin/apps.py',
        'src/app1/core/apps.py',
        'src/app2/apps.py',
    ],
    overrides={
        'manage.py': {
            'dependencies': [
              ':lib',
              '//:reqs#mysqlclient',
              './project/settings/local.py',
            ],
        },
    },
)

As mentioned above I've tried using

src/app1/core

inplace of the

apps.py

as well but no joy

Yeah, that's a bit of a confusing BUILD file. I didn't realize you could override something in the source to point to the source itself

This is based on https://github.com/pantsbuild/example-django/blob/main/helloworld/service/welcome/BUILD

👍

This is based on https://github.com/pantsbuild/example-django/blob/main/helloworld/service/welcome/BUILD

Yeah, it's locally qualified vs fully qualified to settings, but it just looks odd to me. Django seems to have it's own quirks. In general, @happy-kitchen-89482 might be a better person to talk about Django specifically, as he wrote a lot of the Django stuff/example. I use FastAPI and Alembic - and the way I structure everything is slightly different, but Django's "admin" capabilities are tightly coupled, so it makes sense there is some weirdness (from my perspective) in the build

It makes sense that Pants didn't infer the models dependency because in Django you can't directly import models in migrations, you have to use

apps.get_model()

to preserve history

Ahh, I didnt know that

That's a pretty common pattern in Django specifically and if you know how the app names map to Python modules it's possible to translate

get_model

to its relevant python file

Yeah, it's been like 12 years or something since I last touched Django 😆 I wonder if there is some reasonable way to "append" Django-specific dep inference on top of the existing python inference. As in, if this should be updated? https://github.com/pantsbuild/pants/blob/main/src/python/pants/backend/python/framework/django/dependency_inference.py

Oh hang on, that's a different framework…?

Hmm, you know, I'm not sure - the example doesnt have it

It's

pants.backend.experimental.python.framework.django

in the docs. 🧑‍🔬

https://www.pantsbuild.org/stable/docs/using-pants/key-concepts/backends#available-experimental-backends It's still experimental, so I'm not sure (though, there are a lot of our experimental backends which should be promoted to stable)

Django extension is picking up models and migrations correctly it seems. It still needs some help with my custom installed apps, but I suppose I can forgive that

We used Pants with Django at scale pretty effectively at Toolchain (RIP), the repo (now oss) should show this: https://github.com/toolchainlabs/toolchain-oss