Please check my thinking on pants virtualenv expor...
# development
p
Please check my thinking on pants virtualenv exports and PEP 660 editable wheel installs: 🧵
pants export works by creating a requirements.pex and then use pex.tools venv to extract the pex into a dist/export/... directory. GOAL: I want that exported virtualenv to have editable installs of all my python_distribution targets (in the exported resolve).
PEP660 defines a special editable wheel that PEP517 backends can build (if they support PEP660). That wheel contains: a minimal metadata directory (.dist-info), and files needed to redirect module usage to the developer's copy of source files (could be as simple as a .pth file to add them to sys.path). This editable wheel gets installed by PEP 517 frontends (pip, pants) just like any other wheel.
So, I started going down the rabbit hole to allow pants to act as the frontend and call the new PEP660 backend methods. So, like any other dist: populate the sandbox with sources and the generated setup.py; then create a build_backend.pex and a shim script that calls the backend methods. Then all those wheels get installed in a PEX that can be used to populate the export virtualenv with the editable installs (via pex.tools venv). Here's where it gets tricky: if we delegate building the PEP660 wheel to the PEP517 backend from within a sandbox, the editable wheel will point to the sandbox instead of the actual developer sources. The PEP660 interface doesn't have a way to pass in the path to the source files, as that is implicitly the current module's directory. So, I don't think it makes sense for pants to call PEP660
build_editable
method.
Instead, what if pants provided its own PEP660 backend? Or a wrapper that is more than just a shim? That wrapper could delegate building the dist-info directory to the user-specified PEP517 build backend (via
prepare_metadata_for_build_editable
or similar), and then build the editable wheels itself. Or it could break with PEP660 and skip building wheels and just create a PEX or even just a digest with the relevant files that can be added to the export virtualenv.
e
What is the use case again? Editable installs tend to be about live edits; so local webservers that pick up edits on the fly. We try to address that use case in `pants run`; so I'm guessing that support is deficient or else your use case is different.
Is this all just to get generated entry point metadata?
p
Yes. So that dev tools outside of pants have access to: • The locked requirements • The editable sources on the python path • the entry points (and any other package metadata that can be loaded from dist-info. Entry points is the biggest most impactful example) I need to point all the dev tooling at a virtualenv, but recreating that env for every dev change wouldn't be a good dev experience. I can investigate
run
to see if I can somehow use that to run nosetest or other legacy dev tooling. I didn't know that could pick up edits on the fly.
e
Ok, thanks for the summary. That helps me loosen up the lines of thought.
c
Sorry to bump this really old threads by I have not been able to find anything on github: Did "editable installs of `python_distribution`s" become possible in the intervening months?