@happy-kitchen-89482, @hundreds-father-404, @ancient-vegetable-10556, @enough-analyst-54434: i think that last time https://github.com/pantsbuild/pants/pull/14050 occurred (plugin with an incompatible requirement), i had hoped that moving to releasing a PEX (via #12397) would prevent the issue from recurring. but thinking about it more, anything that doesn’t do an on-the fly resolve at use-time (PEX, a lockfile, a rust binary, etc) would all potentially still suffer from this issue

Yeah, we need a lockfile for plugins to resolve this iiuc

so: changing tacks (and trying to tackle this more aggressively this time… “fool me once”): it occurred to me that we might want to skip using the code paths in `pkg_resources`/`setuptools` that actually check for version conflicts, and add to the

sys.path

regardless of conflicts

@hundreds-father-404 Moreso than that, it’s probably-invalid for a plugin to specify a dependency that’s in Pants’ own transitive dependencies

@ancient-vegetable-10556: it’s ok as long as they’re compatible… the dependency that’s often in question is

requests

@witty-crayon-22786 Compatibility means that plugins need to move in lockstep with mainline, so that counts to me as “eventually invalid”, and should be discouraged for that reason

“should be discouraged”, sure. but it’s awkward for either party to need to avoid the most common packages in the ecosystem

The alternative is that the resolve for a given dependency of Pants is actually part of Pants’ API, and we make sure that the plugin itself is invalid for a given version if it conflicts with Pants’ API

@ancient-vegetable-10556: range/floating resolves do actually mostly work, except for the two resolves problem, which is the main thrust of this thread

As long as there isn’t a plugin that specifies an incompatible version of a dependency, sure

we either fail fast (as we did this morning) or slow (with an actual API incompatiblility) in that case

slow failure seems less likely to happen, these are stable libraries like requests

We should probably keep the result of pip freeze from the Pants venv as a constraints input for plugin resolves. That would both be truthful and allow plugin resolves to solve correctly.

Longer term, perhaps we shade and vendor Pants's requirements?

Yeah, or isolate them in a separate Python only used for rules. Right now the Pants deps are mainly outside of rules IIUC and in boot-time / tear-down-time code paths. They all happen to share a sys.path today but maybe they don't need to. Could be complex though.

We should probably keep the result of pip freeze from the Pants venv as a constraints input for plugin resolves.

ah: yes… this seems like an exact match for what we want to do. basically, take the current working set, and feed it to PEX while we’re resolving plugins!