https://pantsbuild.org/ logo
Join Slack
Powered by
Hi everyone, I’m struggling to make my `flask` `p...
# general
a
Hi everyone, I’m struggling to make my 
flask
pex
works with 
gunicorn
, having tried with 
entry_points
and 
PEX_SCRIPT
./pants run xxx
works perfectly fine Once I try to have it running with 
gunicorn
(from local or from a docker image, which is the ultimate goal), I can’t seem to find the right syntax for the module/app
Copy code
COPY packages.hello_world/main.pex packages/hello_world/main.pex

CMD ./packages/hello_world/main.pex packages.hello_world.main:app
Will give me the following error:
Copy code
/pants run packages/hello_world    
/Users/quentin/.cache/pants/setup/bootstrap-Darwin-arm64/2.11.0_py39/bin/pants
15:34:54.78 [INFO] Canceled: Building docker image hello_world:latest
15:34:56.01 [INFO] Completed: Building docker image hello_world:latest
[2022-05-20 19:34:57 +0000] [8] [INFO] Starting gunicorn 20.1.0
[2022-05-20 19:34:57 +0000] [8] [INFO] Listening at: <http://127.0.0.1:8000> (8)
[2022-05-20 19:34:57 +0000] [8] [INFO] Using worker: sync
[2022-05-20 19:34:57 +0000] [11] [INFO] Booting worker with pid: 11
[2022-05-20 19:34:57 +0000] [11] [ERROR] Exception in worker process
Traceback (most recent call last):
  File "/root/.pex/venvs/s/4323ba33/venv/lib/python3.9/site-packages/gunicorn/arbiter.py", line 589, in spawn_worker
    worker.init_process()
  File "/root/.pex/venvs/s/4323ba33/venv/lib/python3.9/site-packages/gunicorn/workers/base.py", line 134, in init_process
    self.load_wsgi()
  File "/root/.pex/venvs/s/4323ba33/venv/lib/python3.9/site-packages/gunicorn/workers/base.py", line 146, in load_wsgi
    self.wsgi = self.app.wsgi()
  File "/root/.pex/venvs/s/4323ba33/venv/lib/python3.9/site-packages/gunicorn/app/base.py", line 67, in wsgi
    self.callable = self.load()
  File "/root/.pex/venvs/s/4323ba33/venv/lib/python3.9/site-packages/gunicorn/app/wsgiapp.py", line 58, in load
    return self.load_wsgiapp()
  File "/root/.pex/venvs/s/4323ba33/venv/lib/python3.9/site-packages/gunicorn/app/wsgiapp.py", line 48, in load_wsgiapp
    return util.import_app(self.app_uri)
  File "/root/.pex/venvs/s/4323ba33/venv/lib/python3.9/site-packages/gunicorn/util.py", line 359, in import_app
    mod = importlib.import_module(module)
  File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'packages.hello_world.main'
[2022-05-20 19:34:57 +0000] [11] [INFO] Worker exiting (pid: 11)
[2022-05-20 19:34:57 +0000] [8] [INFO] Shutting down: Master
[2022-05-20 19:34:57 +0000] [8] [INFO] Reason: Worker failed to boot.
I’ve been trying multiple syntax, and I either get: 
Failed to find attribute 'app' in 'packages.hello_world'.
or 
ModuleNotFoundError: No module named 'packages.hello_world.main'
which tells me that • the package 
packages.hello_world
exists • the package `packages.hellow_world.main`doesn’t exist • there is no 
app
attribute in the package • there is no 
main
attribute in the package folder structure is:
Copy code
/packages/hello_word
|- Build
|- Dockerfile
|- main.py
You can find a repo to reproduce the issue here: https://github.com/qventura/pants-flask/tree/main/packages/hello_world Thanks in advance, if anyone can help me finding the right way to start my app, I’ll be grateful!
👀 1
I actually can build directly with 
pex
, with:
Copy code
pex -vv -D . -o ./main.pex -r requirements.txt
PEX_SCRIPT=gunicorn ./main.pex main:app
I looks like I’m just struggling with the naming…
w
Just reading this quickly, I'm having a hard time understanding which part isn't working as expected? Do you want "dist/packages.hello_world/main.pex" to run? Or is it specifically inside the docker container that is causing the problems?
Also, are you adding your 
:src
sources as a dependency into your pex_binary?
a
both, actually. if I do:
Copy code
./pants package packages/hello_world:main-non-docker
./dist/packages.hello_world/main-non-docker.pex main:app
or
Copy code
./pants run package packages/hello_world:docker
They both don’t run, with error 
ModuleNotFoundError: No module named 'main'
w
Well, for starters, when you unzip your pex - are the appropriate files there?
And under what package structure?
So, like, unrelated to gunicorn - I have this running the flask dev server
Copy code
pex_binary(
  name="main",
  execution_mode="venv",
  entry_point="hello_world.main",
  dependencies=[":src"]
a
Also, are you adding your 
:src
sources as a dependency into your pex_binary?
Hum, no. So, when I use 
entry_point="main.py",
the file is included automatically, but if my entry point is 
gunicorn
I need to add the files?
I’ll give it a try
w
Well, I know there is some level of dependency inference and all that, but personally, I like seeing explicit deps, to avoid any potential confusion. Also, make sure you have appropriate 
__init__.py
files where needed
You can use 
./pants dependencies
or 
./pants peek
to see what will be included, but personally, I'm always explicit over implicit
If you're still struggling when I come back in a few hours, I can whip up a working example from one of my flask projects
h
Ignoring the Docker aspect for now, are you able to 
./pants package packages/hello_world:main
and run the resulting pex?
w
Okay, sorry for the delay here - was just out of commission. How's about this?
Copy code
# BUILD
python_sources(
    name="src"
)

pex_binary(
    name="main",
    script="gunicorn",
    dependencies=[":src", "//:root#gunicorn"]
)
And then 
./pants package ::
And then from the correct dir:
Copy code
> ./main.pex hello_world.main:app

[2022-05-20 20:43:01 -0400] [78898] [INFO] Starting gunicorn 20.1.0
[2022-05-20 20:43:01 -0400] [78898] [INFO] Listening at: <http://127.0.0.1:8000> (78898)
[2022-05-20 20:43:01 -0400] [78898] [INFO] Using worker: sync
[2022-05-20 20:43:01 -0400] [78899] [INFO] Booting worker with pid: 78899
There may be other, better ways to do it - with more flexibility. In the past when I needed to do something similar to this, I used the 
PEX_TOOLS=1
exported to a venv and then called that from 
nginx unit
@enough-analyst-54434 would know better though
If you unzip the 
main.pex
and check out 
PEX_INFO
that should give you a better idea of what's going on
h
Looking at the repo you posted (thanks for that, it's very helpful for debugging), I see that 
packages/helloworld:main
has 
gunicorn
as an entry point, and gunicorn expects an argument
So 
./dist/packages.hello_world/main.pex hello_world.main
would be the right way to invoke that
(Your 
main.pex
is really a pex that contains gunicorn and your main code, but the entry point is generic gunicorn doesn't know about the main)
When I run that I get an error 
ModuleNotFoundError: No module named 'hello_world'
That is because 
main.py
has no imports from 
hello_world
so pants has nothing to infer a dependency from
So I add 
dependencies=[":src"]
to the pex_binary target
Now I get a Gunicorn error: 
Failed to find attribute 'application' in 'hello_world.main'.
Which I think is legit
renaming 
app
to 
application
in 
main.py
solved that and the server works for me
And re the problem of having to specify the entry point to the pex: At Toolchain we solve that by having a wrapper file that runs gunicorn against the appropriate entry point, which allows the pex to run without arguments. This in turn allows us to have a single Docker layer stack for all our pexes, with just one final layer for each service that differs by just one file - the pex
You could also maybe do this with config files, since gunicorn now lets you specify 
wsgi_app =
in gunicorn.conf.py
Anyway, @agreeable-oyster-28981 let us know if this sorts things out!
a
It totally does
renaming 
application
to 
app
is not 100% necessary, as the attribute name can be passed directly when calling unicorn with 
hello_world.main:app
Thanks both of you @wide-midnight-78598 and @happy-kitchen-89482 it was 100% the sources not being included in the pex file -_-
👍 1
7 Views
Open in Slack
PreviousNext
Powered by