Running into a weird issue after upgrading from pants 2.18.0 -> 2.19.1 where I can no longer build

python_google_cloud_function

using docker. Previously my working setup was as follows. When building locally (on an M1 Mac), fall back to using a

--platform=linux/amd64

docker image.

local_environment(
    name="linux",
    compatible_platforms=["linux_x86_64"],
    fallback_environment="docker",
)

docker_environment(
    name="docker",
    platform="linux_x86_64",
    image="multiply_base",
    python_bootstrap_search_path=["<PATH>"],
)


docker_image(
    name="multiply_base",
    instructions=[
        "FROM --platform=linux/amd64 python:3.11-slim",
        "RUN apt update && apt install -y gcc libpq-dev",
    ],
)

the

complete_platforms=["//:platforms"]

references a file created by doing

pex3 interpreter inspect --markers --tags

as detailed here: https://docs.pex-tool.org/buildingpex.html#complete-platform

docker run --rm -it multiply_base /bin/bash
pip install pex
pex3 interpreter inspect --markers --tags

python_google_cloud_function(
    name="cloud_function",
    handler="main.py:example",
    type="http",
    complete_platforms=["//:platforms"],
    environment="docker",
)

Previously (I’ve reverted my version bump and checked), this approach worked fine. But running the same build command on

2.19.1

now fails with

ProcessExecutionFailure: Process 'Build python_google_cloud_function artifact for functions/example:cloud_function' failed with exit code 1.
stdout:

stderr:
A distribution for psycopg2 could not be resolved for /pants-named-caches/python_build_standalone/c12164f0e9228ec20704c1aba97eb31b8e2a482d41943d541cc8e3a9e84f7349/bin/python3.9.
Found 1 distribution for psycopg2 that do not apply:
1.) The wheel tags for psycopg2 2.9.9 are cp311-cp311-linux_x86_64 which do not match the supported tags of /pants-named-caches/python_build_standalone/c12164f0e9228ec20704c1aba97eb31b8e2a482d41943d541cc8e3a9e84f7349/bin/python3.9:
cp39-cp39-manylinux_2_36_x86_64
... 767 more ..

Any ideas?

Sorry for the trouble. I think pants is attempting to run some “pex” commands with its internal Python, rather than the one appropriate for your code… which is info that doesn’t help you fix it much. Related to https://github.com/pantsbuild/pants/issues/19514 Potentially a fix would be upgrading the pex cli that 2.19.1 uses, because supposedly 19514 is improved in pex 2.1.154 but 2.19.1 uses pex 2.1.148 The way to do this is to customise the

[pex-cli]

version

and

known_versions

options https://www.pantsbuild.org/2.19/reference/subsystems/pex-cli by downloading the latest artifacts from https://github.com/pex-tool/pex/releases to compute its length and sha256 hash

Thanks for the response I'll have a play around with the PEX cli versions. Strangely I don't have this issue when building regular flask/gunicorn pex_binary targets for the same function (presumably it's the same PEX version being used?) It's only the google_cloud_function target that fails this way? Makes me think the problem potentially might be something else

The FaaS targets like GCF do a bit more/different work with pex, to lay out the “expected” zip file with code and dependencies, so it’s not implausible that it’s hitting different pex behaviour… but also not implausible that this doesn’t help, as you say!

@dazzling-elephant-33766 did you ever find a solution to this problem? i think i'm running into something similar

I didn't find a solution and reverted to deploying my cloud functions the old fashioned way. This is helpful for future people thanks!

Ah, that's no good. If you either of you could file an issue with a stand-alone reproducer of the error message, that would be great! Thanks Justin for the copy-pasteable section