Hello - I'm trying to create an AWS Lambda zip file. Im running my command to create the zip on an EC2 ubuntu instance since my local machine is a Mac. However, I'm getting this error when I run

~/bin/pants package project/lambda_example.py

. Any ideas?

21:23:38.17 [ERROR] 1 Exception encountered:

Engine traceback:
  in `package` goal
  in Create Python AWS Lambda - project:lambda

ProcessExecutionFailure: Process 'Building lambdex.pex from lambdex_default.lock' failed with exit code 1.
stdout:

stderr:
There were 2 errors downloading required artifacts:
1. lambdex 0.1.9 from <https://files.pythonhosted.org/packages/20/89/05c73da172c510299869e904543825222ffa990b719ffaba4bce2d1be0c4/lambdex-0.1.9-py2.py3-none-any.whl>
    ModuleNotFoundError: No module named 'distutils.util'
2. pex 2.1.111 from <https://files.pythonhosted.org/packages/5d/08/89438cc626ec77a6f7dc3ab17cad581d14882a2f51a37d24c8a4c127e7bc/pex-2.1.111-py2.py3-none-any.whl>
    ModuleNotFoundError: No module named 'distutils.util'

You likely need to install

python3-distutils

with apt.

I've run

sudo apt-get install python3-distutils

Great context! What about

apt install python3.9-distutils

? 🙂

let me try that

That likely means your pants itself is running with 3.8... Are you running the new "get-pants.sh" pants or the old

./pants

? I assume the latter, but just to double-check.

I had curled the install script > get-pants.sh, then installed it. this was before i installed python 3.8 or python 3.9 (back when only 3.10 was installed by default)

Interesting. I'm a bit out of my depth re: that then, I thought that the new one would always run 3.9. Maybe the pex builder uses the host Python still.

Also - I updated my lambda function using the lambda.zip and tried to trigger an invocation using the "Test" button. I get the error

"Unable to import module 'lambda_function': No module named 'lambda_function'"

Am I supposed to name my lambda handler file and handler function a certain name? I'm just following the instructions here: https://www.pantsbuild.org/docs/awslambda-python and the file is named "lambda_example.py" and the function is named "example_handler.py" ; seems like my Lambda function couldn't find those or know about those?

By the way I would have used Ubuntu standard:5.0 since Ubuntu standard:6.0 only supports 3.10. I am using standard:5.0 and since it has pyenv, I manually installed python 3.10 inside it. Can you write here how you are defining your

python_awslambda

target? Your handler should look like

path.to.module:handler_func

as described here https://www.pantsbuild.org/docs/reference-python_awslambda#codehandlercode

I copied and pasted the BUILD and lambda_example.py here: https://www.pantsbuild.org/docs/awslambda-python

It will as long as you define it using one of the

_requirements

function like

python_requirements()

and you should have

requirements.txt

with your python packages in same directory where this is defined

is it possible to not define a requirements.txt or have pants auto-infer and generate that for me?

No, I don’t think so. Pants only does auto-infer for the local imports. It still needs to know what 3rd party dependencies it needs from outside. Although you can give it 100 3rd party packages but it will only pick those which you have imported

^ how would you parse that above statement?

Read this https://www.pantsbuild.org/docs/targets#dependencies-field

A target’s dependencies determines which other first-party code and third-party requirements to include when building the target.

Usually, you leave off the

dependencies

field thanks to dependency inference. Pants will read your import statements and map those imports back to your first-party code and your third-party requirements. You can run

pants dependencies path/to:target

to see what dependencies Pants infers.

hm. what is the point of the auto-infer for local imports then? im still not quite sure what local imports is referring to

So if you have modules or packages locally called

local_module

import local_module #1st party
import pandas as pd #3rd party

i see. so a way to get around this would be to call

pants dependencies

to get the list of 3rd party modules that are inferred, add that to the dependencies in my BUILD, then call

pants package

?

No. It’s different than that. You should definitely read the docs. I feel like I might put something wrong in your head if I go too deep explaining it.

I went through the docs, and things i ended up tweaking were: • added a requirements.txt and an entry in BUILD:

python_requirements( source="requirements.txt"

• In my Lambda runtime settings, I had to set the handler to the auto-generated file and function name:

lambdex_handler.handler

Still a bit confused about the 3rd party inference part for inferring pandas and numpy, as the docs makes it sound like those are auto-inferred

I think there's two different concepts of dependency here that are confusing: • In terms of a target/build it depends on other things in Pants • What do you depend on from outside Pants The first kind is inferred, the second one is not.

python_requirements

bridges these, making Pants know about more things that it can infer from. So: • add

pandas

to `requirements.txt`: Pants learns that pandas exists, where it comes from, etc • add

python_sources()

to a directory: Pants learn those files exist • when looking at a new file it'll see what it imports and try to find those from the things it knows about from before