https://pantsbuild.org/ logo
#development
Title
# development
w

wide-midnight-78598

10/15/2022, 5:08 PM
Is it possible to get the absolute path to these venvs? I see that
mypy
uses the shim script as the python executable, but I'm wondering if we can just get the absolute venv location out of this data structure?
Copy code
requirements_venv_pex = await Get(
        VenvPex,
        PexRequest(
            output_filename="requirements_venv.pex",
            internal_only=True,
            pex_path=[requirements_pex],
            interpreter_constraints=InterpreterConstraints(["==3.9.15"]),
        ),
    )

requirements_venv_pex.venv_rel_dir
1
Alternatively, is there a way to activate this venv and run any given Process in that activated env?
b

bitter-ability-32190

10/15/2022, 6:42 PM
I'm sure there is through PEX_TOOLS=1 but I wouldn't know the incantation. Try copying it out of the sandbox and executing it with that car and poke around?
h

happy-kitchen-89482

10/16/2022, 4:57 AM
Does
VenvPexProcess
not do what you want?
b

bitter-ability-32190

10/16/2022, 11:19 AM
I suspect SJ is asking because pyright needs to be told a venv path
w

wide-midnight-78598

10/16/2022, 12:40 PM
Yep, mypy grabs the python executable shim, but pyright needs either an activated venv, or the path to the venv directory (through a strange split API, which is a bit annoying)
https://github.com/microsoft/pyright/blob/main/docs/configuration.md#main-pyright-config-options
venvPath
and
venv
are what I need to populate, For testing, I copied this from mypy
Copy code
# See `requirements_venv_pex` for how this will get wrapped in a `VenvPex`.
    requirements_pex = await Get(
        Pex,
        RequirementsPexRequest(
            (fs.address for fs in request.field_sets),
            # hardcoded_interpreter_constraints=partition.interpreter_constraints,
        ),
    )
    requirements_venv_pex = await Get(
        VenvPex,
        PexRequest(
            output_filename="requirements_venv.pex",
            internal_only=True,
            pex_path=[requirements_pex],
            interpreter_constraints=InterpreterConstraints(["==3.9.15"]),
        ),
    )
I recall there was a function parameter that would give me the cache location (
NamedCachesDirOption
)?, but I didn't know if it was the correct approach
b

bitter-ability-32190

10/16/2022, 12:50 PM
So the named cache would be the PEX root, which the venv should be somewhere under. But that only gets you the first part of the path...
w

wide-midnight-78598

10/16/2022, 12:50 PM
Yeah, so, here's what I have
errr...
b

bitter-ability-32190

10/16/2022, 12:50 PM
(also don't forget pyright is written with VS Code LSP support in mind. So the config is very IDE centric on existing configurations)
w

wide-midnight-78598

10/16/2022, 12:51 PM
Copy code
dummy_config_digest = await Get(Digest, CreateDigest([FileContent("pyrightconfig.json", f'{{ "venv": "{requirements_venv_pex.venv_rel_dir}" }}'.encode())]))

 NpxProcess(
            npm_package=pyright.default_version,
            args=(
                f"--venv-path={named_cache_dir.val}/pex_root/",
                *pyright.args,  # User-added arguments
                *source_files.snapshot.files,
            ),
So, this works correctly - with the two workarounds being the dummy config (not the end of the world, as we need to support real configs at some point). And then needing to append "pex_root" on the venv-path
via
async def pyright_typecheck(request: PyrightRequest, pyright: Pyright, named_cache_dir: NamedCachesDirOption) -> CheckResults:
I just tried the NamedCachesDir this morning - as I recall it being added to NodeJS, and it looks good - would be nice if there was a fully qualified path to the venv though, so I don't need that manual
pex_roots
- I'm assuming (hoping) this venv path is stable though
b

bitter-ability-32190

10/16/2022, 1:33 PM
Ah yeah that all looks great
I thinkn pex_root is hardocded elsewhere? Grep around. And yes the venv path is stable. PEX is big on stability 😉
w

wide-midnight-78598

10/16/2022, 5:52 PM
For sure the venv path is, I was just wondering about the fully qualified combo - I'll take a look for pex_roots though, maybe I missed it
Copy code
@dataclass(frozen=True)
class CompletePexEnvironment:
    _pex_environment: PexEnvironment
    pex_root: PurePath
    _working_directory: PurePath | None
    append_only_caches: FrozenDict[str, str]

    _PEX_ROOT_DIRNAME = "pex_root"
But this looks interesting:
Copy code
def in_sandbox(self, *, working_directory: str | None) -> CompletePexEnvironment:
        pex_root = PurePath(".cache") / self._PEX_ROOT_DIRNAME
        return CompletePexEnvironment(
            _pex_environment=self,
            pex_root=pex_root,
            _working_directory=PurePath(working_directory) if working_directory else None,
            append_only_caches=FrozenDict({self._PEX_ROOT_DIRNAME: str(pex_root)}),
        )
and
Copy code
@rule
async def setup_pex_process(request: PexProcess, pex_environment: PexEnvironment) -> Process:
    pex = request.pex
    complete_pex_env = pex_environment.in_sandbox(working_directory=request.working_directory)
b

bitter-ability-32190

10/16/2022, 10:00 PM
For the config, is there cmdline flags for these? Cmd line usually trumps config file. I'm thinking if someone has it pointing to an exported venv for editor support, we don't wanna box them out because that's the "ideal" setup IMO
w

wide-midnight-78598

10/17/2022, 12:31 AM
So, none of this touches config yet - that's another can of worms I'm trying to avoid for the moment. Not all of the config have command line checks. Right now, just ensuring it "works" in the degenerate cases with 3rd party code and whatnot
Was going to do interpreter partitions - but pyright also has it's own config for that, so gotta be careful in what overlaps
b

bitter-ability-32190

10/17/2022, 12:34 AM
Yeah I'm thinking a long-term strategy for optimal success would be to use the multi-partition support for per-resolve codebases. But I'm also getting ahead of myself
w

wide-midnight-78598

10/17/2022, 12:34 AM
https://github.com/microsoft/pyright/blob/main/docs/configuration.md#execution-environment-options 🤷 Not exactly the same, so maybe that gets overridden - the config stuff will be interesting
b

bitter-ability-32190

10/17/2022, 12:39 AM
Oh yeah huh. Not as powerful as I thought 😔
w

wide-midnight-78598

10/17/2022, 12:40 AM
🤷 The typechecker is dope, the config is definitely vs-code workflow centric. Not gonna hate on that, because of the checker's inherent dopeness. But it does mean that we'll end up with some whacky config/overrides/subsystems
h

happy-kitchen-89482

10/17/2022, 3:23 AM
The old/new export-by-symlinking functionality grabs a full path to a VenvPex
w

wide-midnight-78598

10/17/2022, 3:40 AM
Ahhh, I was so close - I used
in_sandbox(working_directory=None)
and it didnt work