https://pantsbuild.org/ logo
#general
Title
# general
r

ripe-gigabyte-88964

04/27/2023, 6:27 PM
I have a venv pex that needs to call an application via subprocess that is also included in the pex. I see the
VIRTUAL_ENV
environment variable is set and also that the
PATH
is inheriting from my terminal. Is there a way I can easily include the
VIRTUAL_ENV
path in my
PATH
variable or do I need to manually export the venv first?
e

enough-analyst-54434

04/27/2023, 6:36 PM
How did you create the venv PEX? With pex you'd say
--venv append|prepend
when building it.
Copy code
$ pex cowsay --venv prepend -o cowsay.pex
$ ./cowsay.pex -c 'import subprocess, sys; subprocess.check_call(["cowsay", *sys.argv[1:]])' 'Moo!'
  ____
| Moo! |
  ====
    \
     \
       ^__^
       (oo)\_______
       (__)\       )\/\
           ||----w |
           ||     ||
r

ripe-gigabyte-88964

04/27/2023, 6:39 PM
Copy code
dagster_pex_request = await Get(
            VenvPexRequest,
            PexRequest(
                output_filename="dagster.pex",
                internal_only=True,
                requirements=pex_requirements,
                inject_env={"DAGSTER_ENVIRONMENT": "LOCAL_DEV"},
                interpreter_constraints=InterpreterConstraints(
                    python_setup.resolves_to_interpreter_constraints[dagster_subsystem.resolve]
                ),
                main=EntryPoint("dagster"),
            ),
        )
        dagster_pex = await Get(VenvPex, VenvPexRequest, dagster_pex_request)
Building it as part of a plugin
e

enough-analyst-54434

04/27/2023, 6:40 PM
IIRC internal_only is the rabbit hole to go down, but just grepping "--venv" probably works.
Out of luck:
Copy code
$ git grep -C10 '"\--venv"' src/python/pants/backend/python/util_rules/pex.py
src/python/pants/backend/python/util_rules/pex.py-    # named caches store might be pruned at any time. To guard against that case we introduce a shim
src/python/pants/backend/python/util_rules/pex.py-    # bash script that checks to see if the `pex` venv script exists in the PEX_ROOT and re-creates
src/python/pants/backend/python/util_rules/pex.py-    # the PEX_ROOT venv if not. Using the shim script to run Python tools gets us down to the ~1ms
src/python/pants/backend/python/util_rules/pex.py-    # of overhead we currently enjoy.
src/python/pants/backend/python/util_rules/pex.py-
src/python/pants/backend/python/util_rules/pex.py-    pex_request = request.pex_request
src/python/pants/backend/python/util_rules/pex.py-    seeded_venv_request = dataclasses.replace(
src/python/pants/backend/python/util_rules/pex.py-        pex_request,
src/python/pants/backend/python/util_rules/pex.py-        additional_args=pex_request.additional_args
src/python/pants/backend/python/util_rules/pex.py-        + (
src/python/pants/backend/python/util_rules/pex.py:            "--venv",
src/python/pants/backend/python/util_rules/pex.py-            "--seed",
src/python/pants/backend/python/util_rules/pex.py-            "verbose",
src/python/pants/backend/python/util_rules/pex.py-            pex_environment.venv_site_packages_copies_option(
src/python/pants/backend/python/util_rules/pex.py-                use_copies=request.site_packages_copies
src/python/pants/backend/python/util_rules/pex.py-            ),
src/python/pants/backend/python/util_rules/pex.py-        ),
src/python/pants/backend/python/util_rules/pex.py-    )
src/python/pants/backend/python/util_rules/pex.py-    venv_pex_result = await Get(BuildPexResult, PexRequest, seeded_venv_request)
src/python/pants/backend/python/util_rules/pex.py-    # Pex verbose --seed mode outputs the absolute path of the PEX executable as well as the
src/python/pants/backend/python/util_rules/pex.py-    # absolute path of the PEX_ROOT.  In the --venv case this is the `pex` script in the venv root
So Pants passes
--venv
but not
--venv prepend
or
--venv append
.
So looks like you need to break out a precursor PR that either just always does that or else plumbs a parameter to toggle it.
r

ripe-gigabyte-88964

04/27/2023, 7:16 PM
Should I create an issue along with the PR?
As a temp workaround passing
"PEX_VENV_BIN_PATH": "prepend"
as an env var to my process worked!
e

enough-analyst-54434

04/27/2023, 8:14 PM
Aha, I forgot I added that. Well, that's probably the right thing to stick with.
2 Views