Has anyone had success building a python aws lambd...
# general
l
Has anyone had success building a python aws lambda docker image and getting it to work with the built in Runtime Interface Emulator?
r
We have. I am not sure if there is something special you are asking for. But we use a custom lambda image in combination with PEX generated using
pex_binary
l
@refined-addition-53644 I'm struggling to find a good dev workflow with
python_awslambda
. Seems difficult to do local invocations and testing and doesn't allow for providing a pex source. Would you mind providing a brief summary of the workflow you're using?
r
This is more or less what we are doing using what’s suggested by aws in order to build a custom image.
Copy code
# BUILD
python_sources(name="sources")

pex_binary(
    name="service",
    entry_point="awslambdaric",
    dependencies=["3rdparty/python:default#awslambdaric", ":sources"],
)
Copy code
# Dockerfile
ARG build_for=linux/x86_64
ARG FUNCTION_DIR="/function"
ARG BASE_IMAGE=public.ecr.aws/docker/library/python:3.10.10-slim-bullseye
FROM --platform=$build_for $BASE_IMAGE as base

# Keeps Python from generating .pyc files in the container
ENV PYTHONDONTWRITEBYTECODE 1

# Turns off buffering for easier container logging
ENV PYTHONUNBUFFERED 1

RUN apt-get -qq update && \
    apt-get -y install gcc curl && \
    rm -rf /var/lib/apt/lists/* && \
    python -m ensurepip

WORKDIR /build

# FIXME: Get rid of copying local repository and settings and installing pants.
# From pants 2.15, we can build PEX outside of docker image on MacOS also.
# Install pants
COPY get-pants.sh .
COPY pants.toml .
RUN ["./get-pants.sh", "-d", "/usr/local/bin", "-V", "0.5.4"]

COPY src ./src
RUN pants --no-watch-filesystem --no-pantsd package src/lambda-service:service

# final image
FROM --platform=$build_for $BASE_IMAGE as final

# Include global arg in this stage of the build
ARG FUNCTION_DIR

# Set working directory to function root directory
WORKDIR ${FUNCTION_DIR}

RUN groupadd --gid 1000 lambda \
  && useradd --uid 1000 --gid lambda --shell /bin/bash --create-home lambda

COPY --from=base --chown=lambda:lambda /build/dist/src.lambda-service/service.pex ${FUNCTION_DIR}/service.pex

ENV PEX_ROOT=/tmp/.pex
# FIXME: verify if still needed or not
RUN chmod o+rX ./service.pex
EXPOSE 8080 9000
USER lambda

CMD ["./service.pex", "service.main.lambda_handler"]
You don’t need to build PEX inside docker as we are doing it but we were working with some M1 users and it was bit annoying pre pants 2.15. You can use
docker_image
target and have dependency on the generated
pex_binary
which you can further run inside
docker_environment
We got rid of
python_awslambda
as you can see. Soon even pants is supposed to get rid of it and suggests using
pex_binary
l
Thanks for the example. Appreciated. That should definitely get me going.
👍 1
r
There are more optimizations you can do in order to build the PEX itself inside the docker https://pex.readthedocs.io/en/v2.1.131/recipes.html#pex-app-in-a-container But just start with the initial thing and see how it goes. On the other hand, testing lambda locally still is bit annoying. This is the case even without pants.