Hi all, running into difficulties recently building

google_cloud_function

targets on linux. I have a reproducible example here: https://github.com/Jackevansevo/pants-gcloud-function I'm testing

2.18.0rc5

python_sources()

pex_binary(
    name="flask",
    entry_point="flask",
    dependencies=["./main.py", ":reqs#python-dotenv"],
)

python_google_cloud_function(
    name="cloud_function",
    handler="main.py:handler",
    type="http",
    runtime="python311",
)

python_requirements(name="reqs")

When I attempt to package my

cloud_function

target I get the following error:

Failed to resolve requirements from PEX environment @ /home/jack/.cache/pants/named_caches/pex_root/unzipped_pexes/80c564f4a9a2dcbfaa138d96e1e6eba82636a03a.
Needed cp311-cp311-linux_x86_64 compatible dependencies for:
 1: greenlet!=0.4.17; platform_machine == "aarch64" or (platform_machine == "ppc64le" or (platform_machine == "x86_64" or (platform_machine == "amd64" or (platform_machine == "AMD64" or (platform_machine == "win32" or platform_machine == "WIN32")))))
    Required by:
      SQLAlchemy 2.0.23
    But this pex had no ProjectName(raw='greenlet', normalized='greenlet') distributions.

I've tried adding

greenlet

into my

requirements.txt

and being explicit about the dependency

python_requirements(
    name="reqs",
    overrides={'sqlalchemy': {"dependencies": [":reqs#greenlet"]}}
)

Then I just end up with x2 errors

1: greenlet!=0.4.17; platform_machine == "aarch64" or (platform_machine == "ppc64le" or (platform_machine == "x86_64" or (platform_machine == "amd64" or (platform_machine == "AMD64" or (platform_machine == "win32" or platform_machine == "WIN32")))))
    Required by:
      SQLAlchemy 2.0.23
    But this pex had no ProjectName(raw='greenlet', normalized='greenlet') distributions.
 2: greenlet
    But this pex had no ProjectName(raw='greenlet', normalized='greenlet') distributions.

I've also tried swapping out the

runtime="python311"

section and replacing it with

complete_platforms=[":platforms"]

where

resource(name="platforms", source="platforms.json")

Which is the result of

pex3 interpreter inspect --markers --tags | python -m json.tool > functions/platforms.json

But this again gives me an error

A distribution for greenlet could not be resolved for /home/jack/.cache/nce/f3ff38b1ccae7dcebd8bbf2e533c9a984fac881de0ffd1636fbb61842bd924de/cpython-3.9.18+20231002-x86_64-unknown-linux-gnu-install_only.tar.gz/python/bin/python3.9.
Found 1 distribution for greenlet that do not apply:
1.) The wheel tags for greenlet 3.0.1 are cp311-cp311-manylinux_2_24_x86_64, cp311-cp311-manylinux_2_28_x86_64 which do not match the supported tags of /home/jack/.cache/nce/f3ff38b1ccae7dcebd8bbf2e533c9a984fac881de0ffd1636fbb61842bd924de/cpython-3.9.18+20231002-x86_64-unknown-linux-gnu-install_only.tar.gz/python/bin/python3.9:
cp39-cp39-manylinux_2_35_x86_64

I'm confused as to why this is only happening for my

cloud_function

target, when I'm able to build my

:flask

target completely fine. They appear to share the exact same set of transitive dependencies when I run

pants dependencies --transitive functions:cloud_function

and

pants dependencies --transitive functions:flask

Possibly this is related: https://github.com/pantsbuild/pex/issues/1488 Any ideas?

Thanks for testing! Did this work on a prior release? Which one? I suspect this is a platform issue, since the GCF target is building for an explicit external platform while the flask PEX is just being built for your local one. And pants doesn’t have full precise info about the GCF platform.

Hi managed to revisit this today: https://github.com/Jackevansevo/pants-cloud-function-reproducable-bug I'm getting a similar error with pants 2.19.2 When I said I thought this was working with a prior release, I don't think that's correct.

Thanks for the repo. Unfortunately I cannot reproduce, likely because I'm on macOS, not Linux. I suggest these next steps: 1. Go back to using complete platforms, because that'll be most reliable (the estimated platform based on just the runtime is always an approximation that may not work). a. Just re-reading just now, I think the complete platform JSON file should be loaded as a

file

target, not

resource

. 2. See if you can reduce it even further (e.g. given this is a packaging problem, you probably don't need the files to do anything useful, even just an empty

def handler(): pass

in single

.py

file, and then using

dependencies=["path/to#dep"]

, this will allow getting rid of all the code and BUILD files other than the targets to package (having the

pex_binary

as a comparision seems handy. It seems likely that the docker backend isn't required either) 3. Open an issue in Pants with the as-small-as-possible reproducer

a. Just re-reading just now, I think the complete platform JSON file should be loaded as a

file

target, not

resource

.

Yep this was absolutely the issue. With:

file(name="platforms", source="platforms.json")

It builds but with:

resource(name="platforms", source="platforms.json")

I get the original error

Found 1 distribution for greenlet that do not apply:
1.) The wheel tags for greenlet 3.0.3 are cp311-cp311-manylinux_2_24_x86_64, cp311-cp311-manylinux_2_28_x86_64 which do not match the supported tags of /home/jackevans/.cache/nce/67912efc04f9156d8f5b48a0348983defb964de043b8c13ddc6cc8a002f8e691/cpython-3.9.18+20240107-x86_64-unknown-linux-gnu-install_only.tar.gz/python/bin/python3.9:
cp39-cp39-manylinux_2_38_x86_64
... 809 more ...

I feel like

complete_platforms

should throw an error if

platforms

is a

resource

and not a

file

but perhaps there's no way of knowing this. I might have a look Cheers, for the help! That's been a blocker for ages.

Phew! Sorry for not noticing that the first time around. Definitely agree about diagnostics; please at least file an issue about it, and then feel free to jump in to try to fix

upon revisiting this there's plenty of warnings all over the docs that I missed https://www.pantsbuild.org/2.20/reference/targets/python_google_cloud_function#complete_platforms Nonetheless I've filed an issue/request https://github.com/pantsbuild/pants/issues/20805 describing/reproducing the exact behaviour I ran into. Will see if I can get some time to look into it further this weekend. Thanks for your help

Thank you! No-one reads docs 🙂 we should definitely aspire to having Pants catch problems and offer solutions, rather than assume humans understand the whole system full and never make mistakes