Please check my thinking on pants virtualenv expor...
# development
Please check my thinking on pants virtualenv exports and PEP 660 editable wheel installs: 🧵
pants export works by creating a requirements.pex and then use venv to extract the pex into a dist/export/... directory. GOAL: I want that exported virtualenv to have editable installs of all my python_distribution targets (in the exported resolve).
PEP660 defines a special editable wheel that PEP517 backends can build (if they support PEP660). That wheel contains: a minimal metadata directory (.dist-info), and files needed to redirect module usage to the developer's copy of source files (could be as simple as a .pth file to add them to sys.path). This editable wheel gets installed by PEP 517 frontends (pip, pants) just like any other wheel.
So, I started going down the rabbit hole to allow pants to act as the frontend and call the new PEP660 backend methods. So, like any other dist: populate the sandbox with sources and the generated; then create a build_backend.pex and a shim script that calls the backend methods. Then all those wheels get installed in a PEX that can be used to populate the export virtualenv with the editable installs (via venv). Here's where it gets tricky: if we delegate building the PEP660 wheel to the PEP517 backend from within a sandbox, the editable wheel will point to the sandbox instead of the actual developer sources. The PEP660 interface doesn't have a way to pass in the path to the source files, as that is implicitly the current module's directory. So, I don't think it makes sense for pants to call PEP660
Instead, what if pants provided its own PEP660 backend? Or a wrapper that is more than just a shim? That wrapper could delegate building the dist-info directory to the user-specified PEP517 build backend (via
or similar), and then build the editable wheels itself. Or it could break with PEP660 and skip building wheels and just create a PEX or even just a digest with the relevant files that can be added to the export virtualenv.
What is the use case again? Editable installs tend to be about live edits; so local webservers that pick up edits on the fly. We try to address that use case in `pants run`; so I'm guessing that support is deficient or else your use case is different.
Is this all just to get generated entry point metadata?
Yes. So that dev tools outside of pants have access to: • The locked requirements • The editable sources on the python path • the entry points (and any other package metadata that can be loaded from dist-info. Entry points is the biggest most impactful example) I need to point all the dev tooling at a virtualenv, but recreating that env for every dev change wouldn't be a good dev experience. I can investigate
to see if I can somehow use that to run nosetest or other legacy dev tooling. I didn't know that could pick up edits on the fly.
Ok, thanks for the summary. That helps me loosen up the lines of thought.
Sorry to bump this really old threads by I have not been able to find anything on github: Did "editable installs of `python_distribution`s" become possible in the intervening months?