#22396 RFC: add an experimental `bindeps` backend New discussion created by cognifloyd Would anyone be interested in a

bindeps

backend that can analyze binaries to list which shared libs it is linked with? ## The

bindeps

backend The

bindeps

backend would be functionally similar to

dpkg-shlibdeps

in debian, and `rpmdeps`+`elfdeps` in EL distros. But, it would not require the native packaging tooling to run it. I imagine it would be most useful when using the `nfpm` backend to create native system packages (

.deb

,

.rpm

, ...) because

nfpm

does not analyze included binaries like the native deb/rpm tooling does. But it could also be useful to audit/lint generated binaries before distributing them. ### What I have so far: So far, I have code that can • take a

pex_binary

, • extract its wheels (using pex-tools of course), and • use `elfdeps` :package: (the python lib, not the native rpmbuild binary) to get the list of SONAMEs used by any binaries in the package. I also have code that can take the list of SONAMEs and map them to deb package names using the debian package search API (for debian or ubuntu) instead of querying via native packaging tool(s) like

dpkg -S

. This is important because deb `depends` expects package names, not SONAMEs. This package lookup is not needed for rpm where `requires` takes a list of SONAMEs in addition to package names. For the rules I've written so far, the only python-specific bits handle extracting wheels from a

pex_binary

and using zipfile to make the wheel contents available for

elfdeps

to inspect. The actual

elf

dependency inspection logic and the deb package search are not python-specific and could easily be used to inspect binaries from other pants package targets ### Potential scope of the

bindeps

backend: I'm calling this a

bindeps

backend, because it does not need to be specific to

elf

binaries/libs. In figuring out how best to analyze linux wheels, I also found these projects which would fit nicely under a

bindeps

backend (though I haven't written the code to make them part of a pants backend) - perhaps as

check

or

fix

tools that take packaged binaries as input: • These 3 can analyze a wheel's included python extensions (binaries / shared libs) and "repair" the wheel by copying the shared lib(s) into a copy of the wheel so that no system deps are required. • `delocate` :package:: this is MacOS-specific, so it works with dynamic libraries. It can also combine an

arm64

wheel with an

x86_64

wheel to make a

universal2

wheel that supports both architectures. • `auditwheel` :package:: this is linux-specific, so it works with ELF binaries /

.so

files. It can also audits the wheels'

manylinux

tag, if any, to ensure that the wheel follows the system dep requirements to use that

manylinux

tag. • `delvewheel` :package:: this is Windows-specific, so it works with DLLs. • `repairwheel` :package:: This lib combines the 3 above libs, (so it is cross-platform) focusing on using pure python libs instead of system binaries to modify any libs in the wheel. It also promises to be more host-agnostic so that it can hermetically create the same changes on any host platform. • `abi3audit` :package:: Another wheel audit tool. This one audits a wheel's

abi3

tag to see if python extensions in the wheel only use python symbols stabalized in the target python version or earlier. ## Request for feedback 1. Does this sound generally useful? If so, which part(s) sound useful? 2. Does the

bindeps

name make sense for a backend of this scope? 3. Is there any interest in the creation of

pants.backend.experimental.bindeps

? pantsbuild/pants