Hey all, Im trying to use Pants to build an AWS La...
# general
Hey all, Im trying to use Pants to build an AWS Lambda function. Ive been able to pretty much get it all working, except for one catch. Our code relies transiently on the
library, and as I understand it, AWS Lambdas require a specific binary for that (and probably other) libraries. I was able to resolve that pre-Pants by following the instructions on this Github issue, but Im struggling to figure out how to get that working with the
target in Pants. Pre-Pants I can run this pip command to fetch the correct binary
pip install --platform manylinux2014_x86_64 --implementation cp --python 3.9 --only-binary=:all: --upgrade --target ./ cryptography==38.0.1
. But I dont understand how to translate this information to Pants and Pex. Im doing my best to understand, but pretty much everything Im doing is a stab in the dark 😂 I assume I need to do something with the
argument in the
target. So I created a
target to a
file, and provided that to the
argument. The json looks like this:
Copy code
  "marker_environment": {
    "platform_release": "manylinux2014_x86_64",
    "implementation_name": "cp",
    "python_version": "3.9"
  "compatible_tags": [
The package goal prints the correct platform file, however if I look at the cryptography dist info in the generated lambda, the
file differs from what is generated when I run the pip command, and it doesnt seem to include the
platform. So it doesnt appear to have worked. Does anyone have any advice or thoughts here? Am I on the right track? (Side note, changing the lambda_platform.json contents doesnt trigger a new package. I would've thought Pants would re-build if that dependency changed)
Also I apologize if this is more of a general Python/Pex question. Im somewhat green to python, and not sure of a better place to ask
I don't know if it's the cause of the issue you're seeing here, but we package lambdas (for Python 3.9) that include
, and it works fine for us on macOS and Linux. We use a complete platform file that's generated by running the appropriate pex command within the lambda environment itself, which ends up including many more elements in
, presumably all tags that are compatible (as well as different
values). Specifically, we set up a lambda containing the following, invoked it manually, and then copying the output into the
Copy code
import subprocess

def lambda_handler(event, context):
        pip install --target=/tmp/subdir pex
        PYTHONPATH=/tmp/subdir /tmp/subdir/bin/pex3 interpreter inspect --markers --tags
    return {
        'statusCode': 200,
        'body': "{}",
I'm imagining the lambda environment is pretty similar everywhere, so here's the file we use, for convenience/comparison (generated many months ago, so there may be a more up-to-date one).
We also hit https://github.com/pantsbuild/pants/issues/17621 in our CI builders, which, I think, can be worked around by forcing pants (before 2.16) to use a newer version of pex.
Ah nice! Thanks for the code snippet. I was able to grab the environment we are using. It looks pretty similar at a first glance. Unfortunately trying both your platform json, as well as the one I created results in the same error. Ive included in the above message the whole exception
It's late here now, gonna head off. But I'll be back at this again relatively soon. Will post here if I find a solution
I wonder if you’re hitting issue #17621. I vaguely recall seeing similar errors before adding a workaround for that (see the comment at the end of the issue for one possibility). Are you running from a Linux machine with cpython 3.9 installed?
Ah! That worked!! Thanks so much! I have no idea how long it would've taken to figure that out on my own
Woohoo! No worries I'm glad the hours of confusion I went through saved someone else that experience 😅
This thread saved me a lot of work and improved my understanding of wheel compatibility. Thank you so much ❤️
❤️ 2