There is another Python package import issue I came across I

Question

There is another Python package import issue I came across Is is possible to use system installed site packages and make them available inside pants More details in the thread

red-postman-3638 · Answer

The product uses the in order to use the Berkley Package Filter BPF This package cannot be installed by pip as it is deeply bound to the kernel and needs to be compiled as kernel module Thus it need to be system installed On system python it is usable with `from bcc import BPF` Note that this is not to be confused with the which With venv there is an option ` system site packages` to allow such packages to be used How would I be able to use this system installed package with Pants

red-postman-3638 · Answer

I am not sure whether local requirements <https www pantsbuild org docs python third party dependencies version control and local requirements|from docu> supports unpacked site packages The following requirement does not seem to work for me ```python requirement name= bcc requirements= bcc file usr lib python3 10 site packages bcc ```

enough-analyst-54434 · Answer

In short you can t The reason is this breaks a key principle in Pants repeatable builds Passes on your machine with that system site library installed not on other machines without that

enough-analyst-54434 · Answer

So if its installed in system site packages it must be generally installable in any site packages

enough-analyst-54434 · Answer

Ok it is but you have to do some extra work I followed the instructions here I m on Arch <https github com iovisor bcc blob master INSTALL md arch source> That left me with an sdist ```$ ls l bcc build src python bcc python3 dist total 44 rw r r 1 jsirois jsirois 42837 Jan 11 07 47 bcc 0 23 0 23a21423 tar gz``` That sdist can be then placed on an internal repo and you can configure Pants to use that repo or just use a direct reference url which sounds like its doing the trick for your investigation

enough-analyst-54434 · Answer

This will allow building a PEX uniformly via Pants It s true that the Pex will only successfully run on machines with the rest of bcc installed the native libs bits

enough-analyst-54434 · Answer

Yeah the sdist is pure python It uses ctypes to load the shared library that must be pre installed on the machine ```$ head 23 bcc src python bcc libbcc py | tail 9 import ctypes as ct lib = ct CDLL libbcc so 0 use errno=True needed for perf event attr ctype from perf import Perf keep in sync with bcc common h lib bpf module create c restype = ct c void p```

red-postman-3638 · Answer

Thanks for your quick response I believe I get the same sdist files when installing without the manual compilation ``` usr lib python3 10 site packages bcc$ tree containers py disassembler py init py libbcc py perf py pycache containers cpython 310 pyc disassembler cpython 310 pyc init cpython 310 pyc libbcc cpython 310 pyc perf cpython 310 pyc syscall cpython 310 pyc table cpython 310 pyc tcp cpython 310 pyc usdt cpython 310 pyc utils cpython 310 pyc version cpython 310 pyc syscall py table py tcp py usdt py utils py version py 1 directory 22 files```

enough-analyst-54434 · Answer

You do but the problem is you need a distribution not an installed distribution Your attempt was valiant but didn t work because a direct url reference needs to point at a file and not a directory

red-postman-3638 · Answer

Ah I see so the manual compilation will also give a distributable which contains the appropriate files and can than itself be linked

red-postman-3638 · Answer

Thanks again a lot for your help pray I will try the self compilation next

red-postman-3638 · Answer

Thank you so much and have a pleasant day