https://pantsbuild.org/ logo
#general
Title
# general
h

handsome-sunset-98068

08/24/2022, 2:45 PM
Hey, now I am trying to configure psycopg2 in my django proj settings.py running
Copy code
./pants run projects/appsflyer/:manage -- createsuperuser
throws:
Copy code
django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.
my settings.py file:
Copy code
...
...
DATABASES = {
   'default': {
       'ENGINE': 'django.db.backends.postgresql',
       'NAME': '<database_name>',
       'USER': '<database_username>',
       'PASSWORD': '<password>',
       'HOST': '<database_hostname_or_ip>',
       'PORT': '<database_port>',
   }
}
BUILD
Copy code
python_sources()

resource(
    name="env",
    source=".env"
)

pex_binary(
    name="manage",
    entry_point="manage.py",
    dependencies=[
        "./appsflyer:appsflyer",
        ":env"
    ],
    restartable=True
)
e

enough-analyst-54434

08/24/2022, 2:47 PM
Pants auto-infers any
import ...
or
from ... import ...
(and some weirder cases), but it is not magic. If the settings aren't imported by your code, you may need to add a manual dependency.
I think that's the case with Django config magic?
So, does
./pants filedeps --transitive ./appsflyer:appsflyer
show it owning the
settings.py
file you expect?
Ah, reading the error more closely, I think I have the magic bit right and the magic file wrong. You do own settings.py but that refers to
django.db.backends.postgresql
in the ENGINE valiue. That is effectively a magic import.
You'll need to ensure you depend on whatever 3rdparty project provides that module.
And maybe you do and the magic is one layer deeper and you manually need to include a dependency on
psycopg2
So, those are some manual deps to try adding to solve this. ut the pattern is to ask yourself if the missing thing is likely not strongly imported by import statements in your code.
h

happy-kitchen-89482

08/24/2022, 3:45 PM
That makes Pants treat strings that look like dotted module names as possible dependencies
But the safer path is to add an explicit dependency
h

handsome-sunset-98068

08/24/2022, 4:45 PM
I tried this -
./pants filedeps --transitive ./appsflyer:appsflyer
and settings.py was not there. I tried adding a new source in projects/appsflyer/appsflyer/BUILD (this dir has settings.py)
Copy code
python_sources()

python_source(
    name="settings",
    source="settings.py",
)
Then added that as dependency in the projects/appsflyer/BUILD (this dir has manage.py)
Copy code
python_sources()

resource(
    name="env",
    source=".env"
)

pex_binary(
    name="manage",
    entry_point="manage.py",
    dependencies=[
        "./appsflyer:appsflyer",
        ":env",
        "./appsflyer:settings",
    ],
    restartable=True
)
But this doesn't add the settings.py as filedep in the toplevel appsflyer. How to do that?
r

refined-addition-53644

08/24/2022, 4:51 PM
You need to make your
python_sources
dependencies inside
appsflyer
on
:settings
This way you don’t need to explicitly mention it outside of the directory
appsflyer
h

handsome-sunset-98068

08/24/2022, 4:59 PM
I am not able to follow. Can you please guilde me a little more? For clarity I have update the BUILD file locations above.
r

refined-addition-53644

08/24/2022, 5:10 PM
This build file projects/appsflyer/appsflyer/BUILD
Copy code
python_sources(dependencies=[":settings"])

python_source(
    name="settings",
    source="settings.py",
)
then your other projects/appsflyer/BUILD
Copy code
python_sources()

resource(
    name="env",
    source=".env"
)

pex_binary(
    name="manage",
    entry_point="manage.py",
    dependencies=[
        "./appsflyer:appsflyer",
        ":env",
    ],
    restartable=True
)
Also I don’t think you need to put an explicit dependency inside
pex_binary
on
./appsflyer:appsflyer
. Pants will be able to infer the implicit dependency as long as you import the code from appsflyer somewhere inside
manage.py
.
h

handsome-sunset-98068

08/24/2022, 5:22 PM
This still doesn't fix it.
Copy code
> ./pants filedeps --transitive projects/appsflyer:appsflyer
BUILD
projects/appsflyer/BUILD
projects/appsflyer/manage.py
pyproject.toml
and even without adding the :settings target dependency I get settings.py in sources.
Copy code
> ./pants list projects/appsflyer/appsflyer/:

projects/appsflyer/appsflyer:appsflyer
projects/appsflyer/appsflyer/__init__.py
projects/appsflyer/appsflyer/asgi.py
projects/appsflyer/appsflyer/settings.py
projects/appsflyer/appsflyer/urls.py
projects/appsflyer/appsflyer/wsgi.py
And what does not having settings.py in filedep mean. We also don't have urls.py and wsgi.py in filedeps.
r

refined-addition-53644

08/24/2022, 5:25 PM
Can you try?
Copy code
./pants dependencies --transitive projects/appsflyer:appsflyer
https://www.pantsbuild.org/docs/project-introspection#dependencies---find-a-targets-dependencies
filedeps
outputs all of the files belonging to a target, based on its
sources
field. I would read this a bit https://www.pantsbuild.org/docs/project-introspection
Also good to understand source vs dependency https://www.pantsbuild.org/docs/targets
h

handsome-sunset-98068

08/24/2022, 6:00 PM
Copy code
> ./pants dependencies --transitive projects/appsflyer:appsflyer
//:poetry#Django
//pyproject.toml:poetry
projects/appsflyer/manage.py
btw what should I expect here? moreover, I see settings.py here.
Copy code
> ./pants filedeps --transitive projects/appsflyer:manage   
BUILD
projects/appsflyer/.env
projects/appsflyer/BUILD
projects/appsflyer/appsflyer/BUILD
projects/appsflyer/appsflyer/__init__.py
projects/appsflyer/appsflyer/asgi.py
projects/appsflyer/appsflyer/settings.py. <----HERE
projects/appsflyer/appsflyer/urls.py
projects/appsflyer/appsflyer/wsgi.py
projects/appsflyer/manage.py
pyproject.toml
Since the command run is actually
./pants run projects/appsflyer/:manage -- createsuperuser
then why do we check the
projects/appsflyer:appsflyer
target? I feel like I am missing something conceptual here.
r

refined-addition-53644

08/25/2022, 11:32 AM
Hey you are correct that we should be more concerned about the
pex_binary
and its dependencies. Whatever I had suggested was for debugging purpose. I also realised that the
python_sources
should automatically include
settings.py
inside the
projects/appsflyer/appsflyer/BUILD
since it’s just a glob of all
*.py
files inside the directory. So creating a settings using
python_source
wasn’t needed. At the end it’s
pex_binary
which figures out from the universe of all these sources and dependencies what it should package. Hence you do need to provide an explicit dependency on
settings.py
based target if
pex_binary
can’t figure it out itself.
h

handsome-sunset-98068

08/25/2022, 12:16 PM
Alright, so point taken that if in case some dependency is not being picked up automatically, we need to mention it explicitly. But it is still not working. I changed as you suggested projects/appsflyer/appsflyer/BUILD
Copy code
python_sources(
    dependencies=[
        ":settings"
    ]
)

python_source(
    name="settings",
    source="settings.py",
    dependencies=["//:poetry#psycopg2-binary"]
)
Since the error has not even reached the psycopg2 dependency level, it not even able to find the engine value in the first place. So it looks like that even settings.py is not avl for the binary, though the output of the following cmds is:
Copy code
> ./pants dependencies --transitive projects/appsflyer:manage
//:poetry#Django
//:poetry#apig-wsgi
//:poetry#psycopg2-binary
//:poetry#python-decouple
//pyproject.toml:poetry
projects/appsflyer/appsflyer/__init__.py
projects/appsflyer/appsflyer/asgi.py
projects/appsflyer/appsflyer/settings.py
projects/appsflyer/appsflyer/urls.py
projects/appsflyer/appsflyer/wsgi.py
projects/appsflyer/appsflyer:settings <--- HERE
projects/appsflyer/manage.py
projects/appsflyer:env

> ./pants filedeps --transitive projects/appsflyer:manage
BUILD
projects/appsflyer/.env
projects/appsflyer/BUILD
projects/appsflyer/appsflyer/BUILD
projects/appsflyer/appsflyer/__init__.py
projects/appsflyer/appsflyer/asgi.py
projects/appsflyer/appsflyer/settings.py <--- HERE
projects/appsflyer/appsflyer/urls.py
projects/appsflyer/appsflyer/wsgi.py
projects/appsflyer/manage.py
pyproject.toml
r

refined-addition-53644

08/25/2022, 2:41 PM
I am not really a django expert. From my minimal googling, I am trying to guess. How does django figure out location of settings.py ? I guess you aren’t explicitly importing it, right? In that case django reads some path. You need to make sure that path is available inside
pex_binary
I feel like you need to add this import somewhere
Copy code
from django.conf import settings
https://docs.djangoproject.com/en/4.0/internals/contributing/writing-code/coding-style/#use-of-django-conf-settings
e

enough-analyst-54434

08/25/2022, 2:57 PM
@handsome-sunset-98068 I cannot repro. Here is a worked example using `django-admin startproject appsflyer`: https://github.com/jsirois/tarun-django-debug
The commits are maybe useful to see my progression setting this up.
The db accesses fail, but they otherwise work using correct drivers in each stage.
Maybe you can clone that repo and get it to fail like your project fails? Or else maybe it sheds light on why your project fails when you look at it. I hope that helps.
It looks like I missed some pushes. Everything is now pushed though to a tip of https://github.com/jsirois/tarun-django-debug/commit/d946bf8c52cf93de870bdb012ff5c0b4fdead309
h

handsome-sunset-98068

08/25/2022, 6:21 PM
OMG thanks for this repo that I could observe the diff...I can't believe I copied the wrong key in the DATABASES dictionary, hence it was saying ENGINE was not present. This issue is closed. And super thanks for the effort.
❤️ 1
9 Views