https://pantsbuild.org/ logo
#general
Title
# general
g

gorgeous-eve-12553

10/31/2020, 10:27 PM
What is the recommended way to put compiled code in a pex using pants, take for example: psycopg2
f

flat-zoo-31952

10/31/2020, 10:31 PM
if you use
psycopg2-binary
you can just add it as a requirement like anything else
g

gorgeous-eve-12553

10/31/2020, 10:33 PM
Psycopg specifically recommends not using binary in production, but instead compiling from source.
Also, in my particular case, I'm trying to put turbulette in my pex, which has a dependency 4 layers deep on the compiled versiion of psycopg, and I've been unable to add in the binary version.
f

flat-zoo-31952

10/31/2020, 10:35 PM
then i think the only way is to build psycopg in something resembling the target platform
what's your target platform?
g

gorgeous-eve-12553

10/31/2020, 10:41 PM
Everything
😄 2
f

flat-zoo-31952

10/31/2020, 10:42 PM
then you'll need to build wheels for everything
g

gorgeous-eve-12553

10/31/2020, 10:42 PM
So, I plan on just using my CI pipeline to do the building, but how do I get this into pants to start with?
Or do I just need the build and runtime deps on my host system prior to running pants?
f

flat-zoo-31952

10/31/2020, 10:43 PM
same way as normal, you just add
psycopg2
as a dep, but you'll have to install the system deps on the host
yeah
g

gorgeous-eve-12553

10/31/2020, 10:43 PM
In which case, I would build a docker image with the buildtime deps, and let pants / pex gather my runtime deps.
f

flat-zoo-31952

10/31/2020, 10:44 PM
under the hood, pex calls pip (which it vendors in), which calls setup.py
g

gorgeous-eve-12553

10/31/2020, 10:44 PM
Sound about right?
f

flat-zoo-31952

10/31/2020, 10:44 PM
yep
g

gorgeous-eve-12553

10/31/2020, 10:44 PM
Not ideal, but I'll work with it. Thanks!
f

flat-zoo-31952

10/31/2020, 10:44 PM
welcome to compiled code and python
g

gorgeous-eve-12553

10/31/2020, 10:44 PM
I love Python as a language, but it is so difficult to package and distribute code.
f

flat-zoo-31952

10/31/2020, 10:45 PM
for pure python stuff it's pretty easy
but you have to keep an eye on your dependency chain
for linux images you should use these as bases: https://github.com/pypa/manylinux
you can go with the 2014 series these days pretty safely
these contain old versions of linux system libs, so things should be forward compatible
6 Views