Hi all! Before any question, a little bit of context: Monorepo using Python, split into “libraries” and “lambdas” folders. A “lambda” folder can have a library dependency from “libraries” folder. Each lambda folder is meant to be zipped and used as, guess what: aws lambda package. In order to create each lambda package, we developped a tool that looks for dependency file and gather everything in a dist folder. This folder is then zipped and meant to be used as a aws lambda package. So, Pants looks like the perfect tool to replace our internal one. I already managed to have something working on a test repository (with dummy files, but with the same folder organization), but I have some questions: • Using python_awslambda seemed perfect but not really... As it using PEX, it needs to have a runtime defined. This means that I have to define it in the Pants target, but also again in my IaC tool (Using CDK). Is there any way to avoid it ? My guess is no. • My idea was then to generate a simple archive, without any PEX related stuff. After trying the python_awslambda target, I tried the archive, but I can’t get archive target to work with python_sources. Is it normal? I have a python_sources target, with dependencies. It works great when used in python_awslambda target, but don’t work at all with archive. It seems to consider that there are no files to zip/tar. Is such a thing possible? Feel free to ask for more details if needed! Thanks.

I tried the archive, but I can’t get archive target to work with python_sources. Is it normal?

yea, that’s correct: currently when sources are “typed” as python via

python_sources

, they are not treated as loose files which can be packaged into an

archive

files

targets are typed as loose files, and so can be… but then they are not typed as python, and so don’t have dependency inference, for example

runtime -> According to the doc: ” The

runtime

should be one of the values from https://docs.aws.amazon.com/lambda/latest/dg/lambda-python.html. ” So basically, it’s the Python version. Also, It is required to define the lambda handler, which, as the runtime, I have to redefine in my IaC tool.

How are you specifying it in your IaC tool? If the violation of DRY was offensive enough, you could write a target generator that reads in the IaC to auto-set the target's fields 🙂

Regarding the archive thing: part of the reason why type information is necessary is that there is no well defined layout for sources and third-party wheels that could go into the archive. You'd need type information like "this should be a portable virtualenv" (which isn't really a thing), at which point you'd probably not call it

archive

anymore

That’s interesting. To answer @hundreds-father-404: I’m using aws cdk. So basically, it’s typescript stuff and for a Lambda, that’s how it works:

const fn = new lambda.Function(this, 'MyFunction', {
  runtime: lambda.Runtime.NODEJS_12_X,
  handler: 'index.handler',
  code: lambda.Code.fromAsset(path.join(__dirname, 'lambda-handler')),
});

It requires a runtime, a handler and an asset. Currently in my case, the asset is the archive file generated by our internal tool. My idea was to replace this asset by one generated by Pants. The idea of a target generator seems interesting. I’ll have to work more on the doc. I just started playing with Pants some hours ago. Do you have any useful resources for this ?

We don't have dedicated examples of target generators yet, which we have discussed. But w/ Pants, core Pants uses the exact same plugin api It would look a bit like the target generator

pex_binaries

, but even simpler because you are generating 1->1 rather than 1->n so no need for things like the

overrides

field https://github.com/pantsbuild/pants/blob/cf172080a5de8140b4e244e15a2f6265498e957e/src/python/pants/backend/python/target_types_rules.py#L77-L124