seeking some basic advice on python project struct...
# general
c
seeking some basic advice on python project structure and code sharing: currently, we have around 15 deployable units in our mono repo, and the way we prepare the docker containers for deployments is that each project’s dockerfile copies over at least 5-8 of the other projects according to dependencies, then there is a slew of pip installs in the docker container
thats just the pattern that has been used over and over, and we are about to add 3-4 more deployable units
as you may guess, because everything depends on everything else, we have to deploy nearly all these these units together to make sure there aren’t any issues
so the first step we are taking is, try to package internal libraries that get referenced in nearly all the projects, but moving them out of the repo has proved difficult and jarring for the devs
any tips? I am just getting started with experimenting with pants to incrementally layer it in
r
pants recommended way of packaging 1st party (+ any 3rd party) dependencies is
pex_binary
. Not sure if you already read about it https://www.pantsbuild.org/docs/python-package-goal#creating-a-pex-file-from-a-pex_binary-target You can combine this with docker. Worth going though the docs to understand the flow. https://www.pantsbuild.org/docs/docker
c
Yes I need to study up on this and introduce it, but first I gotta see how you reference code inside of a pex binary, like shared configuration etc etc
r
What do you mean “reference code” inside pex binary? It just packages all of your python code and its underlying dependencies
The PEX file will contain all the code needed to run the binary, namely:
• All Python code and resources the binary transitively depends on.
• The resolved 3rd-party Python dependencies (sdists and wheels) of all targets the binary transitively depends on.
c
ah apologies I might be using the wrong terminology since I am newer to python if I copy a PEX binary into a docker container that used to copy the whole project directory, I need to see how that affects the imports in the code base
r
you don’t need to change any import. It’s like it does all the copying you were doing earlier when building docker. As long as your imports are valid locally it should work inside pex and docker
c
ah ok! I just need to stand up reference repo and just see how it works, but thanks for giving me some direction
👍 1