https://pantsbuild.org/ logo
Join Slack
Powered by
I keep getting this warning about PEX_ROOT inside ...
# general
r
I keep getting this warning about PEX_ROOT inside a lambda docker image. This is after I have configured a custom location and provided read/write permissions. Not sure what I am missing
Copy code
/function/service.pex/.bootstrap/pex/variables.py:582: PEXWarning: PEX_ROOT is configured as /pex_root but that path is un-writeable, falling back to a temporary PEX_ROOT of /tmp/tmpxcb25rwu which will hurt performance.
✅ 1
At the end it leads to PEX extracting packages multiple times and then server runs out of disk space.
b
If I remember correctly, I fixed this by setting the environment variable within the lambda: 
PEX_ROOT=/tmp/.pex
👀 2
r
yeah that worked. Thank you! Lost whole day after it 😞
e
"configured a custom location" - that's either using 
--runtime-pex-root
on the Pex CLI or setting that env var somewhere, in the docker image using ENV instructions or in the lambda config as @billions-keyboard-33102 suggests. There are no other ways I'm aware of. How did you try to configure this @refined-addition-53644?
r
Yeah I was setting custom location inside docker
Copy code
ENV PEX_ROOT=/pex_root
e
Hrm. Should work.
r
But even after giving all permissions pex was complaining about this location. Only thing worked was 
/tmp/.pex
as suggested by Jake.
e
No clue what's up there.
r
yeah agree.
e
Do you know what user AWS runs the image with?
Probably not root.
I'm sure they have doc specifying this.
r
Yeah I couldn’t figure that out.
e
You could also figure it out with a few lines of code in your lambda.
Good thing to know for future. Voodoo works is always bad. Better to have solid works.
r
So from the docs preq section • The container image must be able to run on a read-only file system. Your function code can access a writable 
/tmp
directory with between 512 MB and 10,240 MB, in 1-MB increments, of storage. • The default Lambda user must be able to read all the files required to run your function code. Lambda follows security best practices by defining a default Linux user with least-privileged permissions. Verify that your application code does not rely on files that other Linux users are restricted from running. I don’t know how to fetch this default Lambda user. https://docs.aws.amazon.com/lambda/latest/dg/images-create.html
e
Well bullet point 1 is enough.
Says it right there, use /tmp
👍 1
Soo, @refined-addition-53644 and @billions-keyboard-33102 if you're using a container you should definitely do something like this instead: https://pex.readthedocs.io/en/v2.1.117/recipes.html#pex-app-in-a-container That pre-installs the PEX and pre-compiles it's .pyc. Lower latency cold start and no need for writable PEX_ROOT or any Pex anything at runtime.
r
e
Aha, OK. I missed that while afk for a ~week. I responded on that thread and filed https://github.com/pantsbuild/pex/issues/2001
👍 1
b
One extra thing to add here. I needed to use the non-bundled versions of boto3/botocore to use more recent versions. I had to use the 
PEX_INHERIT_PATH=fallback
- I couldn’t remove the 
/var/runtime
from the python path 😞 https://pex.readthedocs.io/en/latest/api/vars.html#PEX_INHERIT_PATH
e
@billions-keyboard-33102 I want to make sure I track the disappointment. Is there something Pex isn't doing here you think it should or are you disappointed you can't remove /var/runtime from the 
sys.path
? If its the latter, I think you can, you just don't use - I forget the terminology exactly - the standard AWS Lambda executor. They have an API to plug that.
So, if it is the latter, just build the PEX with a 
-D <dir>
where that dir includes an executable 
bootstrap
file meeting their specs. If that proves difficult, perhaps a feature can be added to ... Pants I think, to do that for you. This feels 1 level higher than Pex, Pants knows AWS Lambda, Pex does not - currently.
b
Yes, I agree it’s at the pantsbuild level. I’m using this target: https://www.pantsbuild.org/docs/reference-python_awslambda
e
So, I'm getting lost, but, IIUC @billions-keyboard-33102 you chimed into this thread up above because you too are using a docker container for your lambda. If that's correct then I'd suggest not using `python_awslambda`and using 
pex_binary
instead. That supports 
PEX_INHERIT_PATH
config (https://www.pantsbuild.org/docs/reference-pex_binary#codeinherit_pathcode). You really only need 
python_awslambda
for the lambda zip deployment scheme and even that is only historically true. You can now use a plain old PEX (
pex_binary
again) for that too; you just prefix your lambda entry point configuration value out in AWS with 
__pex__.
to make PEX files as lambda zips work.
b
Ah, sorry - no we’re using the zip deployment. We’ll try to migrate - thank you!
e
Ok, well, probaly best to migrate to containers, but zip works fine! Just switch to 
pex_binary
like I said and ensure a few things like a 
platforms
or 
complete_platforms
that matches AWS (I can provide more info there is needed), 
inherit_path
to solve your problem without having to set 
PEX_*
env vars out in AWS and a handler spec like 
__<http://pex__.my|pex__.my>_<http://package.my|package.my>_module:my_handler_func
.
b
great, thank you - I can use the lambda docker to get the complete_platforms - let me try now!
✅ 1
e
I will be killing the pantsbuild/lambdex project FWIW at some point in the new year since it is no longer needed. That's what currently powers the 
python_awslambda
target / rules.
👍 1
6 Views
Open in Slack
PreviousNext
Powered by