I have question that's maybe a feature request -- I'm looking for something in between a pex_binary and a python_distribution. pex_binary is a nice neatly packaged executable all dependencies installed, but there are some environments (weird systems like GCP Dataflow, strange CI/CD setups) where I don't want an executable per se, but the ability to install an environment with all the needed dependencies. A python_distribution would get us that ability, but its intention is for publishing code, so it doesn't publish overlapping source files -- we don't actually want to publish any of the code, so overlapping source files isn't a problem for us. Two concrete examples: Example 1: We have a test that uses playwright, and when run in CI/CD the docker container that we run our test in needs to have a particular version of chromium installed. This means the docker container needs to: •

pip install

the same version of playwright depended on by the test • Run as root

playwright install-deps chromium

• Run as the test runner user

playwright install chromium

It's possible we could get the

playwright install chromium

to work via a

shell_command

dependency, but the fact that

playwright install-deps chromium

needs to run as root makes doing everything strictly through pants dependencies tough. If we could do something like

COPY /playwright_test.py:<http://thirdparty_deps.xyz|thirdparty_deps.xyz>
RUN pip install <http://thirdparty_deps.xyz|thirdparty_deps.xyz> && playwright install-deps chromium

it would be super convenient Example 2: GCP Dataflow (an Apache Beam executor) has a weird execution model where it pickles python code from the launcher to the workers and expects the workers to have an environment that can run them. I haven't had a chance to try using a PEX yet, but I'm a bit nervous. However, they do provide support to supply the docker container for the worker, which typically people

pip install

the necessary deps and then either

pip install -e

or

PYTHONPATH

the source code. If we could instead package up our deps and source code and install them into the worker, it would be super convenient. I get the sense that the Multi-Stage Build in the python docker blog post is doing something kind of similar, but I couldn't quite grok what's going on there.

Given a PEX and a Dockerfile, you can do something like:

RUN --mount=type=bind,target=/tmp/bound.pex,source=$PEX_FILE,ro PEX_TOOLS=1 python3.11 /tmp/bound.pex venv --compile $VENV_DIR

to get a "regular" venv in the docker image. The blog post is about optimizations on top of that

Ah awesome! And then if we only had one venv that we cared about in this dockerfile we could just

RUN $VENV_DIR/bin/activate

after that?

yeah, or just RUN some script in bin that should have all the paths/shebangs setup already

Amazing, thank you so much!