Hi guys. We are trying to get Django running with ...
# general
i
Hi guys. We are trying to get Django running with pants. Is there a way to have pants scan all python files in a folder to determine dependencies instead of traversing the tree?
s
what do you mean by “scan all python files in a folder” vs. “traversing the tree”? 🤔
my company is also adopting pants in a django monorepo - there have been some rough edges, but I think we’re gradually figuring it out
we defined this macro to consolidate some of the django boilerplate / magic dependencies:
Copy code
def django_app_sources(
    *,
    name: Optional[str] = None,
    models_target: Optional[str] = "./models.py",
    with_migrations: bool = True,
    with_admin: bool = True,
    with_translation: bool = False,
    with_templates: bool = True,
    with_fixtures: bool = False,
    with_content_fixtures: bool = False,
    app_dependencies: Optional[List[str]] = None,
    **python_sources_kwargs
):
    """Target for the top-level sources of a Django app.

    Registers 2 targets in the directory:
        1. A `python_source` covering `apps.py`
        2. A `python_sources` covering all other `*.py` files at the top level

    The `apps.py` target is registered as the default in the directory, to help
    with pants inference based on references from Django settings. All other top-
    level sources are marked as dependencies of `apps.py`, to ensure `urls.py`,
    `signals.py`, etc. are pulled in when their `apps.py` is pulled.

    By default, the top-level `./migrations` directory is also marked as a dependency
    of `apps.py`. Set `with_migrations=False` for apps without migrations to prevent
    errors from pants.

    All other `kwargs` passed to the macro are passed through to the underlying
    `python_sources()` target.
    """

    if "sources" not in python_sources_kwargs:
        python_sources_kwargs["sources"] = []
    python_sources_kwargs["sources"].extend(["*.py", "!apps.py"])

    if not app_dependencies:
        app_dependencies = []
    if models_target:
        app_dependencies.append(models_target)
    if with_migrations:
        app_dependencies.append("./migrations")
    if with_admin:
        # `admin.py` is magically auto-loaded when `django.contrib.admin` is installed.
        app_dependencies.append("./admin.py")
    if with_translation:
        # `translation.py` is magically auto-loaded when `modeltranslation` is installed.
        app_dependencies.append("./translation.py")
    if with_templates:
        app_dependencies.append("./templates")
    if with_fixtures:
        app_dependencies.append("./fixtures")
    if with_content_fixtures:
        app_dependencies.append("./content_fixtures")

    if not name:
        name = "app"

    python_source(name=name, source="apps.py", dependencies=app_dependencies)
    python_sources(**python_sources_kwargs)
as part of this we mandated that all 1st-party apps define an
apps.py
with an
AppConfig
subclass, and updated our
INSTALLED_APPS
settings to refer to the
AppConfig
subclasses instead of using the short-hand and pointing at the app modules
i
Hi. Maybe I'm misunderstanding how pants works. Did you need to explicitly add all the requirements in 'installed_apps' to python sources?
b
Aside: I don't have an answer re the original question but just want to make sure you're aware of https://github.com/pantsbuild/example-django
s
setting
[python-infer].string_imports = true
in
pants.toml
makes inference work from
INSTALLED_APPS
-> the actual app code
I wouldn’t want to try adopting pants in a django repo w/o string inference enabled - too much dynamic magic from django 😬
i
Thanks. We do have string imports set to true. Unfortunately it's not recognizing at least some of the dependencies
s
are they first-party dependencies? or references to third-party apps pulled in via
python_requirements
?
i
Third party apps
s
ah, yeah I can think of 2 possible things going on: 1. the default value of 2 for
string_imports_min_dots
might be too high to pick up on some 3rd-party apps (i.e. the
django-impersonate
package provides the
impersonate
app, which won’t pass the default min-dots test). you could try setting the value to 0 to catch everything, but then pants will be checking every string in your code which I assume will be noticeably slow: https://www.pantsbuild.org/docs/reference-python-infer#section-string-imports-min-dots 2. if the apps have enough dots but the import path doesn’t match the package name, you might need to set the
module_mapping
on your
python_requirements
target: https://www.pantsbuild.org/docs/reference-python_requirements#codemodule_mappingcode
in our case we ended up manually setting some dependencies for 3rd-party apps
i
Hmm. Got it. Thanks for your help. I will give it a shot and see how far we get.
s
@sparse-lifeguard-95737 sorry for the late jump-in but where does the macro live? Or how do you actually use it with pants?
s
we have a
macros.py
file, registered using these instructions: https://www.pantsbuild.org/docs/macros#how-to-add-a-macro