Can I have a pex running in a docker container use...
# welcome
Can I have a pex running in a docker container use a package that’s not contained in the pex ,but built from source in the container? I’ll provide more context in the thread.
It’s for a FastAPI app that needs to connect to sql server with pyodbc. I’m building a linux container on mac, and have to provide a platform string for pex. No pyodbc matches that platform string, so we want to build from source in the image and have that available when the pex runs, hoping that solves the problem
Hi! Ought to be doable. Either by augmenting
, or installing your
in the container as a venv, and then build and install pyodbc into that, may work..
the second option seems to be working on my teammate’s machine, but he’s running linux (I’m on mac) and he got it to work by removing the explicit platform string for pex. That will break the build on my machine, though. Is there a way to ignore the platform string only on the pyodbc dependency?
That’s beyond my knowledge unfortunately.
@enough-analyst-54434 or someone else from the maintainer team will be able to guide you here when they wake up.
Andreas had this approximately right. PEX ignores
as part of its quest to provide isolated environments though; so you must either set
in your environment to tell your PEX it is OK to let the
leak in (See: or else you can set
with the path element to add in directly (See:
As to the issue with platform @icy-hair-30586 that is almost certainly orthogonal. Either the
technique ot the installing the PEX in a container approach should work without modifying platform strings. Do not though that in the PEX in a container approach you need to change the 1st example
line from
RUN PEX_TOOLS=1 /usr/local/bin/python3.10 /my-app.pex venv --scope=deps --compile /my-app
RUN PEX_TOOLS=1 /usr/local/bin/python3.10 /my-app.pex venv --pip --scope=deps --compile /my-app
. Without adding that
then created venv will not include
to use to install more dependencies with.
More direct answers:
Is there a way to ignore the platform string only on the pyodbc dependency?
No. A PEX is a fully enclosed environment so Pex demands it is able to include all dependencies requested. As such, you can't say, "except for FastAPI". Instead you must take the work you did to build FastAPI from source in the container and save the resulting Linux platform specific FastAPI wheel outside the container. You then make that wheel availale to your build using
[python-repos] repos|indexes (whichever is easier for you to host internally, for most this is a
which can be a simple directory with wheels in it that is shared over NFS or sshfs or some other network filesystem or else a simple webserver with index.html generation enabled for the directory with wheels in it). Finally, your
PEXes will now be buildable via Pants and the will include FastAPI since you pre-built the Linux wheel and made it available outside of PyPI. Its a bit of work to be sure, but does that make sense @icy-hair-30586?
That makes sense, thanks! What doesn't make sense is why we're trying to use odbc and sql server with all the other dbs out there, but that's another story 🙂
🤣 1