https://pantsbuild.org/ logo
#welcome
Title
# welcome
i

icy-hair-30586

05/11/2022, 8:10 AM
Can I have a pex running in a docker container use a package that’s not contained in the pex ,but built from source in the container? I’ll provide more context in the thread.
It’s for a FastAPI app that needs to connect to sql server with pyodbc. I’m building a linux container on mac, and have to provide a platform string for pex. No pyodbc matches that platform string, so we want to build from source in the image and have that available when the pex runs, hoping that solves the problem
c

curved-television-6568

05/11/2022, 8:15 AM
Hi! Ought to be doable. Either by augmenting
PYTHONPATH
, or installing your
pex
in the container as a venv, and then build and install pyodbc into that, may work.. https://pex.readthedocs.io/en/latest/recipes.html#pex-app-in-a-container
i

icy-hair-30586

05/11/2022, 8:18 AM
the second option seems to be working on my teammate’s machine, but he’s running linux (I’m on mac) and he got it to work by removing the explicit platform string for pex. That will break the build on my machine, though. Is there a way to ignore the platform string only on the pyodbc dependency?
c

curved-television-6568

05/11/2022, 8:20 AM
That’s beyond my knowledge unfortunately.
@enough-analyst-54434 or someone else from the maintainer team will be able to guide you here when they wake up.
e

enough-analyst-54434

05/11/2022, 2:18 PM
Andreas had this approximately right. PEX ignores
PYTHONPATH
as part of its quest to provide isolated environments though; so you must either set
PEX_INHERIT_PATH
in your environment to tell your PEX it is OK to let the
PYTHONPATH
leak in (See: https://pex.readthedocs.io/en/v2.1.85/api/vars.html#PEX_INHERIT_PATH) or else you can set
PEX_EXTRA_SYS_PATH
with the path element to add in directly (See: https://pex.readthedocs.io/en/v2.1.85/api/vars.html#PEX_EXTRA_SYS_PATH).
As to the issue with platform @icy-hair-30586 that is almost certainly orthogonal. Either the
PEX_*
technique ot the installing the PEX in a container approach should work without modifying platform strings. Do not though that in the PEX in a container approach you need to change the 1st example
RUN
line from
RUN PEX_TOOLS=1 /usr/local/bin/python3.10 /my-app.pex venv --scope=deps --compile /my-app
to
RUN PEX_TOOLS=1 /usr/local/bin/python3.10 /my-app.pex venv --pip --scope=deps --compile /my-app
. Without adding that
--pip
then created venv will not include
pip
to use to install more dependencies with.
More direct answers:
Is there a way to ignore the platform string only on the pyodbc dependency?
No. A PEX is a fully enclosed environment so Pex demands it is able to include all dependencies requested. As such, you can't say, "except for FastAPI". Instead you must take the work you did to build FastAPI from source in the container and save the resulting Linux platform specific FastAPI wheel outside the container. You then make that wheel availale to your build using
[python-repos] repos|indexes
https://www.pantsbuild.org/docs/reference-python-repos (whichever is easier for you to host internally, for most this is a
repo
which can be a simple directory with wheels in it that is shared over NFS or sshfs or some other network filesystem or else a simple webserver with index.html generation enabled for the directory with wheels in it). Finally, your
platforms
PEXes will now be buildable via Pants and the will include FastAPI since you pre-built the Linux wheel and made it available outside of PyPI. Its a bit of work to be sure, but does that make sense @icy-hair-30586?
i

icy-hair-30586

05/11/2022, 3:23 PM
That makes sense, thanks! What doesn't make sense is why we're trying to use odbc and sql server with all the other dbs out there, but that's another story 🙂
🤣 1