Hello all I m a Pants noob trying it as a way to get my mult

Question

Hello all I m a Pants noob trying it as a way to get my multiple Django projects under control and have run in to an roadblock onboarding it My goal is to slowly introduce the build system by initially putting all of my projects under a single `src` folder and then slowly breaking out the shared modules apps to get rid of all the duplication I ve moved the first repo under the `src` folder set my `PYTHONPATH=src` env var and in the `pants toml` file I ve set the source root as `root patterns = src ` However this doesn t seem to be working When I try to run the Django dev server using the command `pants run src djangoproj1 manage py runserver` I get the error `ModuleNotFoundError No module named djangoproj1 settings ` Fully aware I could be holding it wrong and missed a configuration setting while following the docs but I would appreciate any help I ve setup a minimum reproduction using the `example django` repo from the Pants org trying to best emulate what I did in my private repo here s the relevant commit with all the changes and a comment containing the traceback <https github com joshuadavidthomas example django commit 6e8436eb26e15964e7ef1f6366404adfc06d4dff>

happy-kitchen-89482 · Answer

This isn t hurting you but FYI you shouldn t need `PYTHONPATH=src`

happy-kitchen-89482 · Answer

the `root patterns` are the key thing here

happy-kitchen-89482 · Answer

A hypothesis on what s going on is this Pants infers dependencies automatically by looking at `import` statements But Django is funky it loads some deps at runtime by naming conventions or reading env vars So Pants can t infer those deps and you have to add them manually via `dependencies` fields on your targets although I am noodling at a Django plugin that will automate some of this

happy-kitchen-89482 · Answer

Oh but the issue might be a lot simpler Try replacing ` src` with `src` in your root patterns

enough-analyst-54434 · Answer

Is this the right repo <https github com joshuadavidthomas example django tree 6e8436eb26e15964e7ef1f6366404adfc06d4dff src|https github com joshuadavidthomas example django tree 6e8436eb26e15964e7ef1f6366404adfc06d4dff src>

happy-kitchen-89482 · Answer

in this context ` ` is a special symbol that means the repo root is a source root

enough-analyst-54434 · Answer

There is no djangoproj1 where you claim

happy-kitchen-89482 · Answer

At least in your modified example django that ` src` looks suspicious

most-advantage-18294 · Answer

Ah no there wouldn t be That was just a dummy stand in name for my actual project

most-advantage-18294 · Answer

Hm removing the forward slash is still giving me this error in the `example django` repo ```pants example django on src source root v3 11 1 pants roots 09 18 00 45 INFO Initializing scheduler 09 18 01 20 INFO Scheduler initialized src pants example django on src source root v3 11 1 took 4s pants run src helloworld service admin manage py runserver 09 18 56 77 INFO Completed Building 5 requirements for lib pex from the lockfiles python default lock resolve django stubs gt =1 10 0 django lt 4 gt =3 2 13 gunicorn gt =20 1 0 requests gt =2 25 1 types requests gt =2 25 1 Traceback most recent call last File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django core management base py line 354 in run from argv self execute args cmd options File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django core management commands runserver py line 61 in execute super execute args options File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django core management base py line 398 in execute output = self handle args options File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django core management commands runserver py line 68 in handle if not settings DEBUG and not settings ALLOWED HOSTS File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django conf init py line 82 in getattr self setup name File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django conf init py line 69 in setup self wrapped = Settings settings module File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django conf init py line 170 in init mod = importlib import module self SETTINGS MODULE File home josh pyenv versions 3 9 15 lib python3 9 importlib init py line 127 in import module return bootstrap gcd import name level package level File lt frozen importlib bootstrap gt line 1030 in gcd import File lt frozen importlib bootstrap gt line 1007 in find and load File lt frozen importlib bootstrap gt line 984 in find and load unlocked ModuleNotFoundError No module named helloworld service admin settings During handling of the above exception another exception occurred Traceback most recent call last File tmp pants sandbox 8eU8st cache pex root venvs 34b83b0c74ebbfa316d0b4e9693282495ce26847 881dadbb13bd88f9751379a568a51e037d0e6994 pex line 243 in lt module gt runpy run module module name run name= main alter sys=True File home josh pyenv versions 3 9 15 lib python3 9 runpy py line 225 in run module return run module code code init globals run name mod spec File home josh pyenv versions 3 9 15 lib python3 9 runpy py line 97 in run module code run code code mod globals init globals File home josh pyenv versions 3 9 15 lib python3 9 runpy py line 87 in run code exec code run globals File tmp pants sandbox 8eU8st src helloworld service admin manage py line 7 in lt module gt Service file run manage File tmp pants sandbox 8eU8st src helloworld util service py line 46 in run manage execute from command line args File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django core management init py line 419 in execute from command line utility execute File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django core management init py line 413 in execute self fetch command subcommand run from argv self argv File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django core management base py line 367 in run from argv connections close all File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django db utils py line 208 in close all for alias in self File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django utils connection py line 73 in iter return iter self settings File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django utils functional py line 48 in get res = instance dict self name = self func instance File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django utils connection py line 45 in settings self settings = self configure settings self settings File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django db utils py line 144 in configure settings databases = super configure settings databases File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django utils connection py line 50 in configure settings settings = getattr django settings self settings name File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django conf init py line 82 in getattr self setup name File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django conf init py line 69 in setup self wrapped = Settings settings module File home josh cache pants named caches pex root venvs s 295d9e5d venv lib python3 9 site packages django conf init py line 170 in init mod = importlib import module self SETTINGS MODULE File home josh pyenv versions 3 9 15 lib python3 9 importlib init py line 127 in import module return bootstrap gcd import name level package level File lt frozen importlib bootstrap gt line 1030 in gcd import File lt frozen importlib bootstrap gt line 1007 in find and load File lt frozen importlib bootstrap gt line 984 in find and load unlocked ModuleNotFoundError No module named helloworld service admin settings ```

most-advantage-18294 · Answer

Aha if I add `src helloworld service admin lib` to the `dependencies` it works e g the `BUILD` file looks like this now ``` src helloworld service admin BUILD Copyright 2021 Pants project contributors Licensed under the Apache License Version 2 0 see LICENSE python sources name= lib dependencies= src helloworld service admin lib src helloworld greet src helloworld person src helloworld translate pex binary name= gunicorn entry point= gunicorn py dependencies= lib restartable=True pex binary name= manage entry point= manage py dependencies= lib restartable=True ```

most-advantage-18294 · Answer

Seems odd that I would essentially need to add itself as a dependency of itself Am I doing this right

enough-analyst-54434 · Answer

Aha I think you found a hermeticity leak in the pre refactor PYTHONPATH= was implicit then and there was no src directory to screw that up

enough-analyst-54434 · Answer

The way you have things now is correct I think both pre and post src dir insertion

enough-analyst-54434 · Answer

IOW you may have found a silent bug in the example django repo

enough-analyst-54434 · Answer

Ah no you mean the 1st line That is odd Something else is going on here

most-advantage-18294 · Answer

Yep the first item in the `dependencies` list

most-advantage-18294 · Answer

FWIW adding that to my actual project fixes the `ModuleNotFoundError`

enough-analyst-54434 · Answer

Yeah that line is totally whacky and a sign nothing is understood at all yet On the face of it that line should be rejected by Pants and is probably only allowed due to support for circular deps added for totally different reasons

enough-analyst-54434 · Answer

Definitely super weird ```jsirois Gill Windows ~ dev joshuadavidthomas example django src source root $ pants keep sandboxes=on failure run src helloworld service admin manage py runserver Traceback most recent call last File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django core management base py line 354 in run from argv self execute args cmd options File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django core management commands runserver py line 61 in execute super execute args options File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django core management base py line 398 in execute output = self handle args options File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django core management commands runserver py line 68 in handle if not settings DEBUG and not settings ALLOWED HOSTS File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django conf init py line 82 in getattr self setup name File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django conf init py line 69 in setup self wrapped = Settings settings module File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django conf init py line 170 in init mod = importlib import module self SETTINGS MODULE File usr lib python3 9 importlib init py line 127 in import module return bootstrap gcd import name level package level File lt frozen importlib bootstrap gt line 1030 in gcd import File lt frozen importlib bootstrap gt line 1007 in find and load File lt frozen importlib bootstrap gt line 984 in find and load unlocked ModuleNotFoundError No module named helloworld service admin settings During handling of the above exception another exception occurred Traceback most recent call last File tmp pants sandbox uFvszq cache pex root venvs 34b83b0c74ebbfa316d0b4e9693282495ce26847 108a3ddc84230ab282ea6312e06cb68f51008ce5 pex line 243 in lt module gt runpy run module module name run name= main alter sys=True File usr lib python3 9 runpy py line 225 in run module return run module code code init globals run name mod spec File usr lib python3 9 runpy py line 97 in run module code run code code mod globals init globals File usr lib python3 9 runpy py line 87 in run code exec code run globals File tmp pants sandbox uFvszq src helloworld service admin manage py line 7 in lt module gt Service file run manage File tmp pants sandbox uFvszq src helloworld util service py line 46 in run manage execute from command line args File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django core management init py line 419 in execute from command line utility execute File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django core management init py line 413 in execute self fetch command subcommand run from argv self argv File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django core management base py line 367 in run from argv connections close all File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django db utils py line 208 in close all for alias in self File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django utils connection py line 73 in iter return iter self settings File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django utils functional py line 48 in get res = instance dict self name = self func instance File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django utils connection py line 45 in settings self settings = self configure settings self settings File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django db utils py line 144 in configure settings databases = super configure settings databases File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django utils connection py line 50 in configure settings settings = getattr django settings self settings name File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django conf init py line 82 in getattr self setup name File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django conf init py line 69 in setup self wrapped = Settings settings module File home jsirois cache pants named caches pex root venvs s 065716be venv lib python3 9 site packages django conf init py line 170 in init mod = importlib import module self SETTINGS MODULE File usr lib python3 9 importlib init py line 127 in import module return bootstrap gcd import name level package level File lt frozen importlib bootstrap gt line 1030 in gcd import File lt frozen importlib bootstrap gt line 1007 in find and load File lt frozen importlib bootstrap gt line 984 in find and load unlocked ModuleNotFoundError No module named helloworld service admin settings 08 15 40 61 INFO Preserving local process execution dir tmp pants sandbox uFvszq for interactive process jsirois Gill Windows ~ dev joshuadavidthomas example django src source root $ ls l tmp pants sandbox uFvszq src helloworld service admin total 8 rw r r 1 jsirois jsirois 0 Mar 22 08 15 init py rwxr xr x 1 jsirois jsirois 217 Mar 22 08 15 manage py drwxr xr x 2 jsirois jsirois 4096 Mar 22 08 15 pycache jsirois Gill Windows ~ dev joshuadavidthomas example django src source root $ git diff src helloworld service admin BUILD diff git a src helloworld service admin BUILD b src helloworld service admin BUILD index c4eb766 404f2b7 100644 a src helloworld service admin BUILD +++ b src helloworld service admin BUILD 24 6 +24 7 pex binary entry point= manage py dependencies= lib + src helloworld service admin settings py lib restartable=True ``` < happy kitchen 89482> hopefully you ve been working on this too I m out of ideas for the short time I have left at the keyboard this morning

enough-analyst-54434 · Answer

Aha yeah this works fine `pants run src helloworld service admin manage runserver` so this is the newish `pants run path to file py` stuff failing

enough-analyst-54434 · Answer

So to be clear < most advantage 18294> your migration commit works fine IFF you run the `pex binary` target address `src helloworld service admin manage` It fails when running the source file directly probably because settings py is not imported from that file

enough-analyst-54434 · Answer

I think it used to be the case running a source py would use the pex binary target if owned by one Newish work has changed that though for better or worse

most-advantage-18294 · Answer

Gotcha that makes sense So it was a little bit of me holding it wrong I figured smile thanks for your help

enough-analyst-54434 · Answer

Well yeah this is new behavior IIUC maybe since 2 14 and its totally confusing to me as well and I supposedly know what I m doing somewhat in this space I m not sure whether this is considered expected behavior or is a bug

most-advantage-18294 · Answer

Should the `example django` README be updated then <https github com pantsbuild example django to view the frontend>

most-advantage-18294 · Answer

```In three terminals run pants concurrent run helloworld service frontend manage py runserver pants concurrent run helloworld service user manage py runserver pants concurrent run helloworld service welcome manage py runserver And visit this URL in a browser <http 127 0 0 1 8000 person=sherlock amp lang=es> You will have to first set up a database and run migrations of course ```

most-advantage-18294 · Answer

Also thank you so much for your help < enough analyst 54434> and < happy kitchen 89482> I really appreciated you both taking your time to help me through this issue grin

happy-kitchen-89482 · Answer

I ll take another look when I m back in front of a keyboard to be sure I understand what s going on here

happy-kitchen-89482 · Answer

OK so this should be sufficient this is in the unmodified example django repo ```python sources name= lib dependencies= helloworld ui overrides= manage py dependencies helloworld service frontend settings py lib ``` Your coarser dep effectively said everything in this lib depends on everything else in this lib but all that is actually needed is for `manage py` to depend on `settings py` since this is not inferrable

happy-kitchen-89482 · Answer

happy-kitchen-89482 · Answer

Thanks for the heads up on the bug in the example repo

most-advantage-18294 · Answer

Awesome thanks for the help