https://pantsbuild.org/ logo
Join Slack
Powered by
Can I have a pex running in a docker container use...
# welcome
i
Can I have a pex running in a docker container use a package that’s not contained in the pex ,but built from source in the container? I’ll provide more context in the thread.
It’s for a FastAPI app that needs to connect to sql server with pyodbc. I’m building a linux container on mac, and have to provide a platform string for pex. No pyodbc matches that platform string, so we want to build from source in the image and have that available when the pex runs, hoping that solves the problem
c
Hi! Ought to be doable. Either by augmenting 
PYTHONPATH
, or installing your 
pex
in the container as a venv, and then build and install pyodbc into that, may work.. https://pex.readthedocs.io/en/latest/recipes.html#pex-app-in-a-container
i
the second option seems to be working on my teammate’s machine, but he’s running linux (I’m on mac) and he got it to work by removing the explicit platform string for pex. That will break the build on my machine, though. Is there a way to ignore the platform string only on the pyodbc dependency?
c
That’s beyond my knowledge unfortunately.
@enough-analyst-54434 or someone else from the maintainer team will be able to guide you here when they wake up.
e
Andreas had this approximately right. PEX ignores 
PYTHONPATH
as part of its quest to provide isolated environments though; so you must either set 
PEX_INHERIT_PATH
in your environment to tell your PEX it is OK to let the 
PYTHONPATH
leak in (See: https://pex.readthedocs.io/en/v2.1.85/api/vars.html#PEX_INHERIT_PATH) or else you can set 
PEX_EXTRA_SYS_PATH
with the path element to add in directly (See: https://pex.readthedocs.io/en/v2.1.85/api/vars.html#PEX_EXTRA_SYS_PATH).
As to the issue with platform @icy-hair-30586 that is almost certainly orthogonal. Either the 
PEX_*
technique ot the installing the PEX in a container approach should work without modifying platform strings. Do not though that in the PEX in a container approach you need to change the 1st example 
RUN
line from 
RUN PEX_TOOLS=1 /usr/local/bin/python3.10 /my-app.pex venv --scope=deps --compile /my-app
to 
RUN PEX_TOOLS=1 /usr/local/bin/python3.10 /my-app.pex venv --pip --scope=deps --compile /my-app
. Without adding that 
--pip
then created venv will not include 
pip
to use to install more dependencies with.
More direct answers:
Is there a way to ignore the platform string only on the pyodbc dependency?
No. A PEX is a fully enclosed environment so Pex demands it is able to include all dependencies requested. As such, you can't say, "except for FastAPI". Instead you must take the work you did to build FastAPI from source in the container and save the resulting Linux platform specific FastAPI wheel outside the container. You then make that wheel availale to your build using 
[python-repos] repos|indexes
https://www.pantsbuild.org/docs/reference-python-repos (whichever is easier for you to host internally, for most this is a 
repo
which can be a simple directory with wheels in it that is shared over NFS or sshfs or some other network filesystem or else a simple webserver with index.html generation enabled for the directory with wheels in it). Finally, your 
platforms
PEXes will now be buildable via Pants and the will include FastAPI since you pre-built the Linux wheel and made it available outside of PyPI. Its a bit of work to be sure, but does that make sense @icy-hair-30586?
i
That makes sense, thanks! What doesn't make sense is why we're trying to use odbc and sql server with all the other dbs out there, but that's another story 🙂
🤣 1
Open in Slack
PreviousNext
Powered by