https://pantsbuild.org/ logo
#general
Title
# general
b

big-fall-51153

05/27/2021, 2:04 PM
Got a bit stuck again with trying to get django's
manage.py
running under the
run
without chroot solution
most of the commands seem to work, but the
runserver
command isn't - I'm seeing import errors that I don't see under other operations, even
manage.py shell
I am seeing a really weird behavior with how that script appears to be executed - I'm seeing it loaded twice
this might be some weird django-specific behavior though
Untitled.py
Given the above version of
manage.py
, these commands illustrate the difference in behavior I'm seeing:
Copy code
λ › ./pants django-run tests:manage -- check                                                                                             odl/ol-django nl/pants-t2
LOAD
RUN
System check identified no issues (0 silenced).
Copy code
λ › ./pants django-run tests:manage -- runserver                                                                                         odl/ol-django nl/pants-t2
LOAD
RUN
LOAD
RUN
ah, ok, answered my own question - django uses two processes to support code reloading
so disabling reloading, it works:
Copy code
λ › ./pants django-run tests:manage -- runserver --noreload                                                                              odl/ol-django nl/pants-t2
LOAD
RUN
Performing system checks...
I'm guessing that this is due to
PYTHONPATH
being blank for the second process, so it's not inheriting it from pants
and I'm not sure if this is also a contributor, but I found I had to add
execution_mode="venv"
to
pex_binary
for
manage.py
the problem
venv
fixed, was getting an
ImportError
for
manage.py
, probably because of this same
PYTOHNPATH
issue
b

busy-vase-39202

05/27/2021, 2:35 PM
Thanks for documenting your fix!
b

big-fall-51153

05/27/2021, 2:39 PM
I think it only papered over the problem unfortunately. The forked process still can't import anything that pants is adding to the path
Ok, so I was able to find the source for the django command where it forks, which I believe is here: https://github.com/django/django/blob/02c59b7a4355fda8c99224b5de9c0a3929bffe22/django/utils/autoreload.py#L260
I've been able to hack a working solution by futzing with
sys.path
in
manage.py
, i'm trying to debug it a bit to figure out why I need to do this
ok, so a few things going on here, if I don't run in
venv
mode, then it can't find the
manage.py
package.
once I turn that mode on, then it fails because it can't find certain packages that are in another source root, because that second source root isn't in
sys.path
for the forked process
so for debugging purposes (this is also my hack to fix it), I put this at the top of `manage.py`:
Copy code
print("=============================")
print(sys.path)
for path in os.environ.get("PEX_EXTRA_SYS_PATH", "").split(":"):
    if path not in sys.path:
        sys.path.append(path)
print(sys.path)
This is the output from that, you can see several things are dropped from the forked process, but as long as I add at least
src/
back in, my app will initialize.
this is probably the limits of my knowledge of python paths and threading, but the hack works for now
h

happy-kitchen-89482

05/28/2021, 1:12 AM
I have been able to get runserver working with --noreload, as you have, without issues, but I will look over your logs and see if anything stands out.
The reloading mechanism does indeed cause the script to reenter itself, as you deduced
And I've had issues in the past with getting that working in a sandbox
b

big-fall-51153

05/28/2021, 1:28 AM
Amending the path seems to work
Obviously not ideal though. Fortunately this is a test app, not a production one
h

happy-kitchen-89482

05/28/2021, 6:23 PM
Yeah, I don't see a way around having all your source roots on sys.path, if you want to run directly from those sources.
Can you do this via PYTHONPATH?
b

big-fall-51153

05/28/2021, 6:30 PM
I probably could, although for the use case I have, this script will only ever be run for local development purposes on this particular project, so hacking
sys.path
inline is doable
6 Views