This would be useful: <https://discuss.python.org/...
# random
b
f
Definitely need to read that whole PEP. Very relevant to some of the stuff I'd like to do with Python deps in Pants
b
If you wanted to chat, I have something that already simulates this, in a way
f
You have a way of modeling external Python deps in Pants?
b
High-level: • Model system packages just like Python ones ◦ Input
requirements.txt
◦ Install it in a completely clean docker image thats the same as a docker image you want to ship using ◦ Script out a lockfile (what got installed, of what version, and why) • Now you can model
python_requirement
but with system packages • And now you can assign any target a "system package requirement" target • Then when you want to build a docker image, collect those from the transitive walk, and presto you have pinned, transitive system package dependencies (We take this even further by downloading the debs on the host and
dpkg install
-ing them, but thats the core)
The end goal is an ongoing (almost a year now?!) experiment to see if Pants can "provide" the sandboxed system packages, and make an almost-install-nothing dev experience
f
Hmmm interesting. That's more ambitious that what I was even going for. I was just looking for a way to at least refer to things that are installed with system-level package requirements, and potentially also validate them. I've done some exploratory work on this (https://github.com/pantsbuild/pants/issues/19552) and there's a lot of assumptions in the Python backend code about stuff coming from pip, not to mention that I'm not sure how to make pex happy with all this unless https://github.com/pantsbuild/pex/issues/2097 does it. Abstracting how systems can provide dependencies is super hard though, because there's a lot of different ways deps can be provided, and a lot of different systems.
There's also annoying (but understandable) things that downstream packagers do like backporting fixes to earlier versions of packages, which makes pip versions vs external versions not always line up correctly. For example if foo 3.1.1 has a bug that they fix in foo 3.2.0 but the debhat distribution packager sees that foo 3.2.0 is incompatible with the bar 2.7 they need in debhat 30. So they cherry-pick the bugfix and release foo 3.1.1-2. It mucks with the ability to map requirements "semantically" across ecosystems