most-advantage-18294
03/22/2023, 2:10 PMsrc
folder, and then slowly breaking out the shared modules/apps to get rid of all the duplication. I've moved the first repo under the src
folder, set my PYTHONPATH=src
env var, and in the pants.toml
file I've set the source root as root_patterns = ["/src"]
.
However, this doesn't seem to be working. When I try to run the Django dev server using the command pants run src/djangoproj1/manage.py -- runserver
I get the error ModuleNotFoundError: No module named 'djangoproj1.settings'
. Fully aware I could be holding it wrong and missed a configuration setting while following the docs, but I would appreciate any help.
I've setup a minimum reproduction using the example-django
repo from the Pants org trying to best emulate what I did in my private repo, here's the relevant commit with all the changes and a comment containing the traceback -- https://github.com/joshuadavidthomas/example-django/commit/6e8436eb26e15964e7ef1f6366404adfc06d4dffhappy-kitchen-89482
03/22/2023, 2:14 PMPYTHONPATH=src
root_patterns
are the key thing hereimport
statements. But Django is funky, it loads some deps at runtime by naming conventions or reading env vars. So Pants can't infer those deps, and you have to add them manually via dependencies
fields on your targets. (although I am noodling at a Django plugin that will automate some of this)./src
with src
in your root patterns...enough-analyst-54434
03/22/2023, 2:17 PMhappy-kitchen-89482
03/22/2023, 2:18 PM/
is a special symbol that means "the repo root is a source root"enough-analyst-54434
03/22/2023, 2:18 PMhappy-kitchen-89482
03/22/2023, 2:19 PM/src
looks suspiciousmost-advantage-18294
03/22/2023, 2:19 PMexample-django
repo:
pants-example-django on ļ src-source-root ī v3.11.1
ā pants roots
09:18:00.45 [INFO] Initializing scheduler...
09:18:01.20 [INFO] Scheduler initialized.
src
pants-example-django on ļ src-source-root [!] ī v3.11.1 took 4s
ā pants run src/helloworld/service/admin/manage.py -- runserver
09:18:56.77 [INFO] Completed: Building 5 requirements for lib.pex from the lockfiles/python-default.lock resolve: django-stubs>=1.10.0, django<4,>=3.2.13, gunicorn>=20.1.0, requests>=2.25.1, types-requests>=2.25.1
Traceback (most recent call last):
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 61, in execute
super().execute(*args, **options)
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
output = self.handle(*args, **options)
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 68, in handle
if not settings.DEBUG and not settings.ALLOWED_HOSTS:
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 82, in __getattr__
self._setup(name)
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 69, in _setup
self._wrapped = Settings(settings_module)
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 170, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/home/josh/.pyenv/versions/3.9.15/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'helloworld.service.admin.settings'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/pants-sandbox-8eU8st/./.cache/pex_root/venvs/34b83b0c74ebbfa316d0b4e9693282495ce26847/881dadbb13bd88f9751379a568a51e037d0e6994/pex", line 243, in <module>
runpy.run_module(module_name, run_name="__main__", alter_sys=True)
File "/home/josh/.pyenv/versions/3.9.15/lib/python3.9/runpy.py", line 225, in run_module
return _run_module_code(code, init_globals, run_name, mod_spec)
File "/home/josh/.pyenv/versions/3.9.15/lib/python3.9/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/home/josh/.pyenv/versions/3.9.15/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/tmp/pants-sandbox-8eU8st/src/helloworld/service/admin/manage.py", line 7, in <module>
Service(__file__).run_manage()
File "/tmp/pants-sandbox-8eU8st/src/helloworld/util/service.py", line 46, in run_manage
execute_from_command_line(args)
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
utility.execute()
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/core/management/base.py", line 367, in run_from_argv
connections.close_all()
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/db/utils.py", line 208, in close_all
for alias in self:
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/utils/connection.py", line 73, in __iter__
return iter(self.settings)
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/utils/functional.py", line 48, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/utils/connection.py", line 45, in settings
self._settings = self.configure_settings(self._settings)
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/db/utils.py", line 144, in configure_settings
databases = super().configure_settings(databases)
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/utils/connection.py", line 50, in configure_settings
settings = getattr(django_settings, self.settings_name)
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 82, in __getattr__
self._setup(name)
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 69, in _setup
self._wrapped = Settings(settings_module)
File "/home/josh/.cache/pants/named_caches/pex_root/venvs/s/295d9e5d/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 170, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/home/josh/.pyenv/versions/3.9.15/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'helloworld.service.admin.settings'
src/helloworld/service/admin:lib
to the dependencies
, it works, e.g. the BUILD
file looks like this now:
# src/helloworld/service/admin/BUILD
# Copyright 2021 Pants project contributors.
# Licensed under the Apache License, Version 2.0 (see LICENSE).
python_sources(
name="lib",
dependencies=[
"src/helloworld/service/admin:lib",
"src/helloworld/greet",
"src/helloworld/person",
"src/helloworld/translate",
],
)
pex_binary(
name="gunicorn",
entry_point="gunicorn.py",
dependencies=[
":lib",
],
restartable=True,
)
pex_binary(
name="manage",
entry_point="manage.py",
dependencies=[
":lib",
],
restartable=True,
)
enough-analyst-54434
03/22/2023, 2:28 PMmost-advantage-18294
03/22/2023, 2:35 PMdependencies
list.ModuleNotFoundError
enough-analyst-54434
03/22/2023, 2:39 PMjsirois@Gill-Windows:~/dev/joshuadavidthomas/example-django (src-source-root *) $ pants --keep-sandboxes=on_failure run src/helloworld/service/admin/manage.py -- runserver
Traceback (most recent call last):
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 61, in execute
super().execute(*args, **options)
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
output = self.handle(*args, **options)
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 68, in handle
if not settings.DEBUG and not settings.ALLOWED_HOSTS:
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 82, in __getattr__
self._setup(name)
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 69, in _setup
self._wrapped = Settings(settings_module)
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 170, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'helloworld.service.admin.settings'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/tmp/pants-sandbox-uFvszq/./.cache/pex_root/venvs/34b83b0c74ebbfa316d0b4e9693282495ce26847/108a3ddc84230ab282ea6312e06cb68f51008ce5/pex", line 243, in <module>
runpy.run_module(module_name, run_name="__main__", alter_sys=True)
File "/usr/lib/python3.9/runpy.py", line 225, in run_module
return _run_module_code(code, init_globals, run_name, mod_spec)
File "/usr/lib/python3.9/runpy.py", line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File "/usr/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/tmp/pants-sandbox-uFvszq/src/helloworld/service/admin/manage.py", line 7, in <module>
Service(__file__).run_manage()
File "/tmp/pants-sandbox-uFvszq/src/helloworld/util/service.py", line 46, in run_manage
execute_from_command_line(args)
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
utility.execute()
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/core/management/base.py", line 367, in run_from_argv
connections.close_all()
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/db/utils.py", line 208, in close_all
for alias in self:
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/utils/connection.py", line 73, in __iter__
return iter(self.settings)
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/utils/functional.py", line 48, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/utils/connection.py", line 45, in settings
self._settings = self.configure_settings(self._settings)
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/db/utils.py", line 144, in configure_settings
databases = super().configure_settings(databases)
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/utils/connection.py", line 50, in configure_settings
settings = getattr(django_settings, self.settings_name)
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 82, in __getattr__
self._setup(name)
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 69, in _setup
self._wrapped = Settings(settings_module)
File "/home/jsirois/.cache/pants/named_caches/pex_root/venvs/s/065716be/venv/lib/python3.9/site-packages/django/conf/__init__.py", line 170, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/usr/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'helloworld.service.admin.settings'
08:15:40.61 [INFO] Preserving local process execution dir /tmp/pants-sandbox-uFvszq for interactive process
jsirois@Gill-Windows:~/dev/joshuadavidthomas/example-django (src-source-root *) $ ls -l /tmp/pants-sandbox-uFvszq/src/helloworld/service/admin/
total 8
-rw-r--r-- 1 jsirois jsirois 0 Mar 22 08:15 __init__.py
-rwxr-xr-x 1 jsirois jsirois 217 Mar 22 08:15 manage.py
drwxr-xr-x 2 jsirois jsirois 4096 Mar 22 08:15 __pycache__
jsirois@Gill-Windows:~/dev/joshuadavidthomas/example-django (src-source-root *) $ git diff src/helloworld/service/admin/BUILD
diff --git a/src/helloworld/service/admin/BUILD b/src/helloworld/service/admin/BUILD
index c4eb766..404f2b7 100644
--- a/src/helloworld/service/admin/BUILD
+++ b/src/helloworld/service/admin/BUILD
@@ -24,6 +24,7 @@ pex_binary(
entry_point="manage.py",
dependencies=[
":lib",
+ "src/helloworld/service/admin/settings.py:lib",
],
restartable=True,
)
@happy-kitchen-89482 hopefully you've been working on this too. I'm out of ideas for the short time I have left at the keyboard this morning.pants run src/helloworld/service/admin:manage -- runserver
, so this is the newish pants run path/to/file.py
stuff failing.pex_binary
target address src/helloworld/service/admin:manage
. It fails when running the source file directly, probably because settings.py is not imported from that file.most-advantage-18294
03/22/2023, 3:37 PMenough-analyst-54434
03/22/2023, 3:39 PMmost-advantage-18294
03/22/2023, 3:43 PMexample-django
README be updated then? https://github.com/pantsbuild/example-django#to-view-the-frontendIn three terminals run:
pants --concurrent run helloworld/service/frontend/manage.py -- runserver
pants --concurrent run helloworld/service/user/manage.py -- runserver
pants --concurrent run helloworld/service/welcome/manage.py -- runserver
And visit this URL in a browser: <http://127.0.0.1:8000/?person=sherlock&lang=es> . You will have to first set up a database and run migrations, of course.
happy-kitchen-89482
03/22/2023, 4:47 PMpython_sources(
name="lib",
dependencies=[
"helloworld/ui",
],
overrides={
"manage.py": {"dependencies": ["helloworld/service/frontend/settings.py:lib"]}
}
)
Your coarser dep effectively said "everything in this lib depends on everything else in this lib", but all that is actually needed is for manage.py
to depend on settings.py
(since this is not inferrable)most-advantage-18294
03/22/2023, 11:39 PM