I would like to include a 3rd party library in my ...
# general
b
I would like to include a 3rd party library in my project. The project is like this
Copy code
/libs
/libs/my_code
/libs/3rd_party_code
In the 3rd party library, they have some C++ and CUDA code under
Copy code
/libs/3rd_party_code/src/c++_cuda_code
If I were to install it using the library's
setup.py
file, it would be dealt with through the following code snippet
Copy code
from torch.utils.cpp_extension import CUDAExtension

ext_modules=[
    CUDAExtension(
        name="pytorch_quantization.cuda_ext",
        sources=[os.path.join(abspath, "src/tensor_quant.cpp"),
                 os.path.join(abspath, "src/tensor_quant_gpu.cu")])
],
Is there anyway to get this to work with pants? Many thanks!
c
I'm assuming this is in a monorepo? If you have a
python_distribution
for the cuda-based library using their
setup.py
I think you should be able to depend on that dist from your code and pants will just do "the right thing". (and this dependency also ought to be inferred as soon as you import any modules from that library, so shouldn't even have to add explicit dep in any BUILD file..)
b
Hi Andreas, Correct. This is a monorepo and we have various libraries under
libs/
. This is the only one causing a bit of trouble. Ah I think I see! So I read https://www.pantsbuild.org/2.18/docs/python/overview/building-distributions So I would basically use the
setup.py
file from the cuda-based library and pants will automatically generate the
BUILD
file etc.?
As long the `project.toml`is properly configured etc.
or I need to specify the
python_distribution
manually in the
BUILD
for
libs/cuda_library
?
sorry for the confusion - I guess what you mean is that I would generate it myself
python setup.py sdist
and I can use that in the monorepo?
h
You create a BUILD file with a
python_distribution
that uses that preexisting setup.py, per the instructions in https://www.pantsbuild.org/2.18/docs/python/overview/building-distributions#setuppy for not generating a setup.py
Then Pants should run that setup.py to build the dist