https://pantsbuild.org/ logo
Join Slack
Powered by
Is it possible to get the absolute path to these v...
# development
w
Is it possible to get the absolute path to these venvs? I see that 
mypy
uses the shim script as the python executable, but I'm wondering if we can just get the absolute venv location out of this data structure?
Copy code
requirements_venv_pex = await Get(
        VenvPex,
        PexRequest(
            output_filename="requirements_venv.pex",
            internal_only=True,
            pex_path=[requirements_pex],
            interpreter_constraints=InterpreterConstraints(["==3.9.15"]),
        ),
    )

requirements_venv_pex.venv_rel_dir
✅ 1
Alternatively, is there a way to activate this venv and run any given Process in that activated env?
b
I'm sure there is through PEX_TOOLS=1 but I wouldn't know the incantation. Try copying it out of the sandbox and executing it with that car and poke around?
h
Does 
VenvPexProcess
not do what you want?
b
I suspect SJ is asking because pyright needs to be told a venv path
w
Yep, mypy grabs the python executable shim, but pyright needs either an activated venv, or the path to the venv directory (through a strange split API, which is a bit annoying)
https://github.com/microsoft/pyright/blob/main/docs/configuration.md#main-pyright-config-options 
venvPath
and 
venv
are what I need to populate, For testing, I copied this from mypy
Copy code
# See `requirements_venv_pex` for how this will get wrapped in a `VenvPex`.
    requirements_pex = await Get(
        Pex,
        RequirementsPexRequest(
            (fs.address for fs in request.field_sets),
            # hardcoded_interpreter_constraints=partition.interpreter_constraints,
        ),
    )
    requirements_venv_pex = await Get(
        VenvPex,
        PexRequest(
            output_filename="requirements_venv.pex",
            internal_only=True,
            pex_path=[requirements_pex],
            interpreter_constraints=InterpreterConstraints(["==3.9.15"]),
        ),
    )
I recall there was a function parameter that would give me the cache location (
NamedCachesDirOption
)?, but I didn't know if it was the correct approach
b
So the named cache would be the PEX root, which the venv should be somewhere under. But that only gets you the first part of the path...
w
Yeah, so, here's what I have
errr...
b
(also don't forget pyright is written with VS Code LSP support in mind. So the config is very IDE centric on existing configurations)
w
Copy code
dummy_config_digest = await Get(Digest, CreateDigest([FileContent("pyrightconfig.json", f'{{ "venv": "{requirements_venv_pex.venv_rel_dir}" }}'.encode())]))

 NpxProcess(
            npm_package=pyright.default_version,
            args=(
                f"--venv-path={named_cache_dir.val}/pex_root/",
                *pyright.args,  # User-added arguments
                *source_files.snapshot.files,
            ),
So, this works correctly - with the two workarounds being the dummy config (not the end of the world, as we need to support real configs at some point). And then needing to append "pex_root" on the venv-path
via 
async def pyright_typecheck(request: PyrightRequest, pyright: Pyright, named_cache_dir: NamedCachesDirOption) -> CheckResults:
I just tried the NamedCachesDir this morning - as I recall it being added to NodeJS, and it looks good - would be nice if there was a fully qualified path to the venv though, so I don't need that manual 
pex_roots
- I'm assuming (hoping) this venv path is stable though
b
Ah yeah that all looks great
I thinkn pex_root is hardocded elsewhere? Grep around. And yes the venv path is stable. PEX is big on stability 😉
w
For sure the venv path is, I was just wondering about the fully qualified combo - I'll take a look for pex_roots though, maybe I missed it
Copy code
@dataclass(frozen=True)
class CompletePexEnvironment:
    _pex_environment: PexEnvironment
    pex_root: PurePath
    _working_directory: PurePath | None
    append_only_caches: FrozenDict[str, str]

    _PEX_ROOT_DIRNAME = "pex_root"
But this looks interesting:
Copy code
def in_sandbox(self, *, working_directory: str | None) -> CompletePexEnvironment:
        pex_root = PurePath(".cache") / self._PEX_ROOT_DIRNAME
        return CompletePexEnvironment(
            _pex_environment=self,
            pex_root=pex_root,
            _working_directory=PurePath(working_directory) if working_directory else None,
            append_only_caches=FrozenDict({self._PEX_ROOT_DIRNAME: str(pex_root)}),
        )
and
Copy code
@rule
async def setup_pex_process(request: PexProcess, pex_environment: PexEnvironment) -> Process:
    pex = request.pex
    complete_pex_env = pex_environment.in_sandbox(working_directory=request.working_directory)
b
For the config, is there cmdline flags for these? Cmd line usually trumps config file. I'm thinking if someone has it pointing to an exported venv for editor support, we don't wanna box them out because that's the "ideal" setup IMO
w
So, none of this touches config yet - that's another can of worms I'm trying to avoid for the moment. Not all of the config have command line checks. Right now, just ensuring it "works" in the degenerate cases with 3rd party code and whatnot
Was going to do interpreter partitions - but pyright also has it's own config for that, so gotta be careful in what overlaps
b
Yeah I'm thinking a long-term strategy for optimal success would be to use the multi-partition support for per-resolve codebases. But I'm also getting ahead of myself
w
https://github.com/microsoft/pyright/blob/main/docs/configuration.md#execution-environment-options 🤷 Not exactly the same, so maybe that gets overridden - the config stuff will be interesting
b
Oh yeah huh. Not as powerful as I thought 😔
w
🤷 The typechecker is dope, the config is definitely vs-code workflow centric. Not gonna hate on that, because of the checker's inherent dopeness. But it does mean that we'll end up with some whacky config/overrides/subsystems
h
The old/new export-by-symlinking functionality grabs a full path to a VenvPex
w
Ahhh, I was so close - I used 
in_sandbox(working_directory=None)
and it didnt work
Open in Slack
PreviousNext
Powered by