https://pantsbuild.org/ logo
#general
Title
# general
b

broad-processor-92400

11/22/2022, 6:11 AM
With this potential switch from Lambdex/`python_awslambda` to PEX/`pex_binary` , is there a way to replace the behaviour of
lambdex_handler.handler
? That is, a universal entrypoint that runs whatever is configured in a
BUILD
file. Previously, with
python_awslambda
, we set
handler="path/to/whatever.py:handler"
which both defines the 'root' of the packaging and also the handler function, and then our CDK construct for a pants lambda just needs to be told about the zip file and we're good to go (the construct can just set
handler: 'lambdex_handler.handler'
because that always works). With PEX, it seems like we'll have to set
entry_point="path/to/whatever.py:handler"
(or just the file) to define the packaging, and then duplicate that into the CDK construct by also specifying
path.to.whatever:handler
. This is annoying to write twice, and risks inconsistencies.
I suppose we could do this ourselves in CDK by reading
PEX-INFO
from the zip file but that's... not so nice! For example, a
__pex__.something.configured_entry_point
magic path would allow us to set
handler: '__pex__.something:configured_entry_point'
h

happy-kitchen-89482

11/22/2022, 1:20 PM
Interesting. I had always assumed that having a fixed handler at the CDK level was a nuisance, not a benefit. I think the plan (when someone gets around to it) is to modify the existing backend to produce pexes with a .zip extension and set the handler in the same way, so you can switch to the new implementation without having to change anything, and then let users opt out of the fixed handler. But it sounds like that might not even be desirable, in your case!
b

broad-processor-92400

11/22/2022, 2:57 PM
Yeah, I’m just feeling this out as I go (investigating switching to using PEXes directly), and I think you’re right that that wouldn’t be desirable for us: the handler function is intimately connected to the packaging for ~all lambdas, and specifying that twice info seems annoying. (The only case we might want to override is multiple lambdas that use different handlers within one file, but that’s pretty rare for us.)
e

enough-analyst-54434

11/22/2022, 3:05 PM
It would be easy enough to keep the interface as is. Along with growing support for
__pex__
Pex also finally got support for closing over environments. Both were needed to free itself from API consumers to allow cleanly cutting over to Pex 3.x. The closing over environments thing would mean lambdex can change to shipping as just another distribution that was a main-delegator ~like uvicorn. So lambdex then just has a fixed lamdex_handler:handle but where that delegates to can be controlled from env vars. On the Pants side we'd just use Pex's ~new:
Copy code
--inject-env INJECT_ENV
                        Environment variables to freeze in to the application environment. (default: [])
To freeze in your handler from the BUILD file target metadata as-is today.
So, in short, lambdex will still be ~gutted, but we can make it so you don't know the difference.
Hopefully that works for you @broad-processor-92400, it will be a smaller step then transitioning to a bespoke zip, which I think we determined has a single cold-start overhead as its only advantage.
b

broad-processor-92400

11/22/2022, 8:51 PM
Thanks, I think that sounds reasonable. I take from the discussion that this isn't currently supported. (Faster cold starts would be handy, e.g. ours got noticably slower when we switched to using Lambdex-driven lambdas)
6 Views