Hi all, I’ve got some google cloud functions that ...
# general
d
Hi all, I’ve got some google cloud functions that rely on the
psycopg2
package. I’d like the ability to build + deploy my cloud functions from my MacOS machine, however I’m running into current limitations of pants:
Copy code
16:44:33.76 [WARN] Google Cloud Functions built on macOS may fail to build. If your function uses any third-party dependencies without binary wheels (bdist) for Linux available, it will fail to build. If this happens, you will either need to update your dependencies to only use dependencies with pre-built wheels, or find a Linux environment to run ./pants package. (See <https://realpython.com/python-wheels/> for more about wheels.)
It looks like the only way to get a wheel is via
psycopg2-binary
(which works) but the documentation recommends avoiding this in production and instead using the source distribution. I’m wondering if anyone has any recommendations/advice to work around this limitation? Is having a Linux pipeline to build/deploy cloud functions really my only option? Or is it possible to temporarily override/replace psycopg2->psycopg2-binary for one-off deployments?
h
Hmmm, so this isn't exactly a limitation of Pants itself per se.
psycopg2
is an sdist, so it has to be built on the platform you're going to deploy it on. That would be true regardless of which build system was orchestrating the building. However in Pants 2.15.0rc7 you can use the new Environments feature to run the necessary processes in a Docker container on macOS, so you end up with a linux artifact.
In fact, we should probably wire that up automatically for cloud functions, since they will always have to be built for linux
But for now you should be able to set it up manually?
Is this macOS system running on ARM?
d
Yep, its arm
h
So you'll need to build it for linux x86 on macos ARM...
I assume the cloud function runtime platform is x86?
Fortunately you can do this via Docker, as it can emulate
d
Yep
h
but it will be slow...
d
I can have a look at doing it in docker thanks
What’s confused me slightly, was that previously I’ve not had issues. I used to upload a
requirements.txt
explicitly to GCLOUD, and presumably
psycopg2
would be compiled/build at the point of deployment instead. https://cloud.google.com/functions/docs/writing/specifying-dependencies-python#python38
h
Ah! That's the difference
yeah, I guess Google would do all this for you at deployment, which is late in the workflow. Pants will build a hermetic binary up front and you deploy that.
And if that works we should probably integrate it into the cloud function plugin automatically