green-tiger-17672
01/08/2024, 4:00 AMpants publish ::
and pants package ::
We are getting this error now when attempting to package certain AWS lambdas .
stderr:
56
No pre-built wheel was available for fuzzysearch 0.7.3.
57
Successfully built the wheel fuzzysearch-0.7.3-cp39-cp39-linux_x86_64.whl from the sdist fuzzysearch-0.7.3.tar.gz but it is not compatible with the requested foreign target complete platform cp311-cp311-manylinux_2_26_x86_64.
58
You'll need to build a wheel from fuzzysearch-0.7.3.tar.gz on the foreign target platform and make it available to Pex via a --find-links repo or a custom --index.
59
these lambdas are packaged as docker images, and we followed the documentation for the necessary build file changes from 2.16 -> 2.18. Nothing else has changed with these specific lambdas.
My assumption is that this must be related to the updates around 'complete platforms' , Im curious if there is anything I can add/pass in the build file to have the package/publish behavior operate like it did before the update?
here is what the BUILD file looks like for example.
python_sources(name="lib")
python_aws_lambda_function(
name="referral_lambda",
runtime="python3.11",
handler="referral_lambda.py:lambda_handler",
dependencies=["//:reqs#psycopg2-binary", "//:reqs#jmespath"],
)
docker_image(
name="referral",
dependencies=[":referral_lambda"],
image_tags=["{build_args.GIT_COMMIT}"],
repository="referral",
)
And here is the relevant portion of the python-default lockfile for that specific library/dependency.
{
"artifacts": [
{
"algorithm": "sha256",
"hash": "d5a1b114ceee50a5e181b2fe1ac1b4371ac8db92142770a48fed49ecbc37ca4c",
"url": "<https://files.pythonhosted.org/packages/f7/28/3e9e4e55fd35356f331a22976694e151eb0214b68d3cd471936f9c09deba/fuzzysearch-0.7.3.tar.gz>"
}
],
"project_name": "fuzzysearch",
"requires_dists": [
"attrs>=19.3"
],
"requires_python": null,
"version": "0.7.3"
},
Thanks in advance!broad-processor-92400
01/08/2024, 4:29 AMpants.toml
?green-tiger-17672
01/08/2024, 4:39 AM[GLOBAL]
pants_version = "2.18.0"
backend_packages.add = [
"pants.backend.python",
"pants.backend.python.lint.docformatter",
"pants.backend.python.lint.black",
"pants.backend.python.lint.flake8",
"pants.backend.python.lint.isort",
"pants.backend.python.typecheck.mypy",
"pants.backend.awslambda.python",
"pants.backend.experimental.python.lint.autoflake",
"pants.backend.shell",
"pants.backend.docker",
]
[anonymous-telemetry]
enabled = false
[source]
# The Python source root is the repo root. See <https://www.pantsbuild.org/docs/source-roots>.
root_patterns = ["/", "sailebot", "sailestack"]
[python]
# The default interpreter constraints for code in this repo. Individual targets can override
# this with the `interpreter_constraints` field. See
# <https://www.pantsbuild.org/docs/python-interpreter-compatibility>.
# Modify this if you don't have Python 3.9 on your machine.
# This can be a range, such as [">=3.8,<3.11"], but it's usually recommended to restrict
# to a single minor version.
interpreter_constraints = [">=3.11.4,<3.12"]
# Enable the "resolves" mechanism, which turns on lockfiles for user code. See
# <https://www.pantsbuild.org/docs/python-third-party-dependencies>. This also adds the
# `generate-lockfiles` goal for Pants to generate the lockfile for you.
enable_resolves = true
resolves = { python-default = "python-default.lock", flake8 = "3rdparty/python/flake8.lock" }
[docformatter]
args = ["--wrap-summaries=100", "--wrap-descriptions=100"]
[test]
use_coverage=true
# open_coverage=true
report=true
[coverage-py]
fail_under=67
global_report=true
# report=["html"]
[flake8]
install_from_resolve="flake8"
[docker]
build_args = ["GIT_COMMIT"]
[docker.registries.saile-dev-registry]
address = "<our aws ecr address>"
default = true
extra_image_tags = ["latest"]
green-tiger-17672
01/08/2024, 4:39 AMbroad-processor-92400
01/08/2024, 4:41 AMpants.toml
?
[lambdex]
layout = "lambdex"
green-tiger-17672
01/08/2024, 4:57 AMbroad-processor-92400
01/08/2024, 4:58 AMgreen-tiger-17672
01/08/2024, 4:58 AMbroad-processor-92400
01/08/2024, 4:58 AMpython_aws_lambda_function
targets to be packaged using the old style approach, rather than the new better (mostly) one.green-tiger-17672
01/08/2024, 4:59 AMRemove the whole [lambdex] section, as Lambdex is deprecated and its functionality will be removed. If you have `layout = "zip"`, no further action is required, as you are already using the recommended layout.
If you have `layout = "lambdex"`, removing the section will switch any `python_aws_lambda_function` (formerly `python_awslambda`) and `python_google_cloud_function` targets to using the `zip` layout, as recommended by cloud vendors. (If you are using `python_aws_lambda_function`, you will need to also update the handlers configured in the cloud from `lambdex_handler.handler` to `lambda_function.handler`.)
See the docs for more details:
broad-processor-92400
01/08/2024, 4:59 AMgreen-tiger-17672
01/08/2024, 5:00 AMgreen-tiger-17672
01/08/2024, 5:17 AMinterpreter_constraints
--lambdex-interpreter-constraints="['<str>', '<str>', ...]"
PANTS_LAMBDEX_INTERPRETER_CONSTRAINTS
default:
[
"CPython>=3.7,<3.12"
]
Python interpreter constraints for this tool.
Im assuming then , if we were not specifying this before then it would have been defaulting to this interpreter constraint, correct?
If so, would that explain why it was able to package before? The library in question only officially supports up to 3.10, so it would have accepted the wheel built for 3.9 as that falls within the interpreter constraints? (Im a little out of my element here so forgive me if I am way off base)broad-processor-92400
01/08/2024, 5:24 AM[lambdex] layout
field is new, as part of the switch from the Lambdex wrapper, to the new AWS-recommended layout, to allow staging a roll-out across 2.17 & 2.18 better without breaking people (or at least, when there is a breaking problem, give more options/workarounds).
The interpreter constraints there are a bit different, they say that the Lambdex tool itself needs that an interpreter in that range to run the tool, not so much what the output of the tool is.broad-processor-92400
01/08/2024, 5:25 AMDockerfile
? In particular, what's the base image FROM ...
?green-tiger-17672
01/08/2024, 5:37 AMFROM public.ecr.aws/lambda/python:3.11
RUN yum install unzip-6.0-57.amzn2.0.1.x86_64 -y
COPY sailebot.referral/referral_lambda.zip .
RUN unzip referral_lambda.zip -d ${LAMBDA_TASK_ROOT}
CMD ["lambdex_handler.handler"]
Noticing now that this is still referencing "lambdex_handler.handler" which I am assuming will need to be updated.broad-processor-92400
01/08/2024, 5:37 AMbroad-processor-92400
01/08/2024, 5:37 AMgreen-tiger-17672
01/08/2024, 5:41 AMbroad-processor-92400
01/08/2024, 6:14 AMfuzzysearch
library available for it.
Can you confirm the lambda runs successfully in the old style? (not just builds successfully)