I’ve got a change that makes pants do editable ins...
# development
I’ve got a change that makes pants do editable installs of all relevant `python_distribution`s when exporting a mutable virutalenv (which essentially turns pants into a PEP-660 frontend and backend - albeit delegating a few pieces of that to pip and to the configured PEP-517 backend). Or at least I’ve got all of the export tests passing now, including integration tests. I haven’t used this on the st2 repo yet 🙂 Scope of my change: This is only possible fore mutable venvs, so symlinked venvs will not get this feature. Tool resolves (which are deprecated in 2.16) do not ever have dists, so this does not apply there either. And I’m not touching the legacy export codepath (where you do
export <addresses>
) that will stay as is; this will only apply when using
export --resolve=
). So, here’s my question: Should we have a new export option to enable/disable editable installs? If so, should it default to enabled or disabled (disabled is the backwards compatible option)? Or should we always do editable install of dists when exporting a resolve that includes dists?
@curved-television-6568 I believe you mentioned in an issue that you tried to do something like this - would you want this to always be enabled?
what’s the downside to having it always enabled? It’s been quite a while since I needed this, and it may come up again but not something I’d need on a regular basis for sure.
does not have a
field. So figuring out which resolve a
belongs to is expensive: calculating the owned deps of all distributions, and for each distribution, look through those deps until one of them has a resolve field, and use that for that dist’s resolve. Plus there’s the cost of building the PEP-660 wheels - if the configured PEP-517 build backend doesn’t support the PEP-660 methods, then it falls back to a method that is, sadly, optional in PEP-517. If that method isn’t there, then it falls back to telling that backend to build the whole wheel, and then it extracts just the dist-info directory from it and discards the rest. So, installing these editable wheels isn’t free. It’ll slow down the export, though I’m not sure by how much.
hmm… feels like an option is a good idea then; possibly defaulting to
based on the relatively few times we’ve seen this feature requested..?
Yeah. I would enable that option in pants.toml for the st2 project, but 🤷 others apparently don’t need it as much?
👍 1
What should the option name be? •
• something else?
😬 1
@happy-kitchen-89482 wdyt?
oh. Should this be one of those
options? I’m thinking yes. hmm. that’s more complex.
Is it a property of the resolve though, or of the python_distribution?
oh. I hadn’t thought about adding a field to
to control inclusion.
The resolve gets exported atomically (basically), so the option has to be set at the resolve level. But I suppose individual
targets could opt out of such inclusion.
Oh, right, this is when exporting an entire resolve
Oh wait, but why do we even need python_distributions? can we not "export" the relevant source roots as editable installs? Or does that require a setup.py?
I have no idea how editable installs work
We need to be able to generate the
directory with the PEP-517 backend, so we need the
An editable install consists of the
directory and a
file that injects the source directory into the PYTHONPATH (or more accurately, into
Technically, we could also inject a
file that just adds all of the source roots (that contain code in the exported resolve). But people can set PYTHONPATH for that as well. The major value of editable installs is to get the
@happy-kitchen-89482 and @curved-television-6568 Could I get some feedback on https://github.com/pantsbuild/pants/pull/18639 before I mark it ready for review?
I’m waiting to mark it ready for review because: one part of it, the refactoring in
should probably go in a separate PR. If it does, then it could be backported to 2.16 as a behavior-less refactor. Doing that means that I can copy a lot of the code in the PR into a plugin for use in the st2 repo while I wait for 2.17 to stabilize enough to upgrade.
But I would like feedback on the approach first (before splitting off the refactor PR and before creating the plugin).
I marked it as ready for review. I'm very excited about this change. 😄
👀 1