Hello, everyone... I'm in the process of adding Pu...
# general
Hello, everyone... I'm in the process of adding Pulumi to our project (https://www.pulumi.com), and using the Python SDK for it. The
CLI is a Go binary, but to execute our infrastructure program written in Python, it needs to be invoked from inside a virtualenv with the right dependencies. Is there some kind of way for Pants to create such an environment for me, and then "get out of my way" and let me execute arbitrary
commands in that environment? I see there's a
pants run
command, but that looks like it's for executing Pex binaries. Can that be adapted for my usecase, or is there another way? Thanks 🙇
You could create a plugin that would basically be
pants run
but with the final step being to kick off
. The plugin would create the virtual env based on your target(s). I am not familiar with
but if it is triggered by a simple command, this should be fairly simple to create.
My guess is it would be similar to the run test plugin: https://www.pantsbuild.org/docs/plugins-test-goal
But maybe @hundreds-father-404 could suggest a better starting point.
hrm... interesting idea; I'll give that a look. Thanks @jolly-midnight-72759
@brash-baker-91190 I'm not sure I completely follow, but if you're on Pants 2.3.0rc1 you can specify your
wants `execution_mode=venv`: https://www.pantsbuild.org/v2.3/docs/reference-pex_binary#codeexecution_modecode
@enough-analyst-54434 I'm not sure that'll do what I need, though it sounds like it's getting closer. The binary I want to execute isn't Python, but in the course of it running, it will end up invoking Python code. I just need the right Python code to be present in the environment. Without Pants, I just activate a venv in my terminal and then execute the
binary. I'm just looking for a way for Pants to handle that part so engineers don't have to remember that, while the rest of the Python code in our repo is handled by Pants, this one special bit of Python code isn't, and you'll have to manually curate your environment.
(Longer term, Pulumi has an automation API that allows you to integrate into larger Python programming without having to go through the Go binary at all... we may end up using that for other reasons, but we're just starting out, so I'm trying to ease the adoption and usage path as much as possible.)
So a PEX file that executes in venv mode can also be told to build a venv of itself here like so:
PEX_TOOLS=1 ./my.pex venv right/here
. That will turn
into a venv
. Does that work minus the need to run
./pants package
and then run
PEX_TOOLS=1 dist/.../my.pex ...
And ...
source right/here/bin/activate
Hrmm... that may work. If I'm understanding correctly, this is basically declaring my environment as an artifact that Pants knows how to "build", and prevents me from having to have a stray
file for this corner of code, correct? (in addition to allowing me to pull in arbitrary dependencies from elsewhere in our repo, which may be required soon). It sounds like about the same amount of work to get the environment set up, but is at least "Pants-native", so that seems like a nice incremental improvement. Do I have that all about right?
Yup, I think so.
Neat, I'll give that a try in a bit. Thanks!
The side benefits of venvs stored in a PEX are multiplatform support and shippability. You can have a multiplatform PEX so you can build a linux venv on a mac and ship it over to a linux box, PEX_TOOLS "install it" and go.
🚀 1
Thanks for raising this issue, I've got a Pulumi repo that I've been working on for about a year now and have been thinking about introducing Pants to it.