From rule code, I would like to modify the exporte...
# development
From rule code, I would like to modify the exported venv whenever someone does
./pants export --resolve=st2
. Or perhaps just add another
option. @happy-kitchen-89482 @curved-television-6568 what would you think of modifying this code to do the following: • allow other rules to inject things into the digest (probably via a new union) ◦ Like this rule which could inject
files for the `python_distribution`s in the current resolve: ▪︎ ◦ Or a new rule that could inject
files for each of the `python_distribution`s in the resolve (which would mean we don't have to mess with
and is essentially the solution described in PEP 660 for editable wheels. • make
more configurable somehow (maybe add more post processing commands before removing the reqs+pex pexes) ◦ like something that modifies the
in the
script to include the first party source roots included in that resolve (Instead of adding the .pth files above) ◦ adjust the requirements pex venv command line args (or maybe the env) so that I can specify the version of pip/setuptools By manipulating
and adding
, we could use this to simulate editable installs of the first party sources.
Note that none of this can be allowed for symlinked venvs, so I assume this is all for newly-build venvs?
correct. only for
If there's a way to do this somewhat streamlined? I fear turning this into a swiss army knife, but if we focus on 1-2 use cases here (e.g., simulating editable installs) to start?
Yeah. simulating editable installs is my focus.
Studying this, I think the best extension point might be to allow rules to add files to the digest before the post-process command runs. Assumption I need to validate: I think we’ll be able to add files into the virtualenv before the virtualenv is actually created. If not, then the rules will have to add the files to a separate temp directory with another post-processing command that copies them into the virtualenv after it gets created.
As far as editing
doesn’t have very many options for the
sub command. I mentioned editing
because I thought it would be something like the
command where I need to specify pip/setuptools/wheel versions. But, that’s not necessary - I’m adding those to my requirements instead (since they are runtime deps for st2), and then will allow my pex-provided versions to be used instead of whatever default
versions happen to get included in the venv thanks to the
flag. ps: I really want whatever version of pex includes that feature to make it into pants 2.16 if at all possible These pex-tools venv flags might also need to be configurable somehow: •
(maybe we just need to set this and not make it configurable. I think it should have the resolve name instead of defaulting to the python version (the dir name of
like it does now) •
(not sure if the dev env might need this at some point) •
(this is only needed if some libs in the venv don’t work with the symlinks. I have encountered this in the past, though I don’t think I need it now) And then looking at the pex sources, this option could be interesting in pants if it gets exposed via pex-tools
subcommand: •
(not sure if that would make sense for the dev environment) ◦ `Virtualenv.create(…, system_site_packages=False, …)`: ◦ which gets called in pex-tools
We could just optionally add all the source roots, no?
With no configurability?
I don't understand what you mean by configurability. That other rules should not be able to inject things into the export? Or that the user shouldn't have to use more CLI flags?
I mean that letting users configure arbitrary post_processing_cmds seems like overkill? We can give you one option that adds all the source roots as "editable installs"?
Yeah. Making that configurable doesn't provide as much value as I expected. My comments are, afaict, an exhaustive list of what value could be extracted from it (ie not much)
Creating editable installs is the way to go. Do we do that per source root or per
? Somehow it should be filtered to only include the source roots or `python_distribution`s if they contain something in the target resolve.
If a source root does not include a
, then we do not have enough info to add
directories of an editable install. We could add the source roots to the
files, but we risk adding source roots that are irrelevant, or, worse, that mask module names in an unrelated source root. So, I don't want to just add all the source roots to the path.
Extracting entry_point details from anything other than a
(like a
) seems very problematic, because we would not have a package name for where to put the metadata.