https://pantsbuild.org/ logo
Join Slack
Powered by
Hello! I'm trying to write a custom target in orde...
# plugins
a
Hello! I'm trying to write a custom target in order to compile Open Dynamics Engine (written in C) & import the Python library that gets generated as an artifact of compilation into my project. Is there already a way that Pants 2 can do something like this? or am I going to have to write a plugin myself? For reference, here's the compilation process that I'm currently performing manually:
Copy code
(env)$ cd ode-0.16.2
(env)$ ./configure --enable-double-precision --with-trimesh=opcode --enable-new-trimesh --enable-shared
(env)$ make
(env)$ make install
(env)$ cd bindings/python/
(env)$ pip install cython
(env)$ pip install .
Thread in #general
@hundreds-father-404 replied:
I think your approach would be very similar, that you create a custom target and then add a codegen implementation to convert your C code into whatever the artifact type is. (What’s the file extension?)
This is what the artifact(s) look like after compilation:
Copy code
tyler@DESKTOP-5PA52E3:~/ode-test/ode-0.16.2/bindings/python$ tree
.
├── INSTALL.txt
├── TODO.txt
├── demos
│   ├── tutorial1.py
│   ├── tutorial2.py
│   └── tutorial3.py
├── ode.pxd
├── ode.pyx
└── setup.py

1 directory, 8 files
So yeah, it's also a Cython extension like mentioned previously
h
Hm, are all of these files created from the compilation, or any were there to begin with? That’s interesting if it does indeed generate both 
.py
files and resource files like like 
.txt
files. That should work just fine if true; you’re going to have two different codegen rules, one to generate 
PythonSources
and another to generate 
ResourcesSources
. Thanks to the engine’s design, you could have it so that you still only run the subprocess once to run compilation; both codegen rules can rely on the same compilation
a
My initial reaction is that this'll have to be a two step process: one target to compile the code, and one target to represent the compilation and installation of the Cython library once generated
Yeah! All those files are created by the compilation process.
👍 1
Wait, I take it back. They're not actually generated by the compilation process, they already exist in the source tree. However, they do rely on the ODE library being compiled and installed to some accessible PATH before you can compile the Cython. So it should be no different
👍 1
h
one target to compile the code
Rather than multiple targets, I suspect you might want one target, but multiple rules. Rules are where you define the logic of your plugin, such as running compilation. Targets are simply a declarative way to attach metadata to some code, such as saying “These 5 files are all used for my Cython plugin. They depend on these 3 other files.” You will likely want a target like 
cython_library
or 
c_project
which describes the initial C files. Your rules then will use the codegen plugin hook to convert the CSources to PythonSources, etc.
👍 1
They’re not actually generated by the compilation process
Okay, what files are being generated?
a
Gotcha re: the difference between targets and rules. I'm coming from the land of GNU Make, be easy on me! 😅
❤️ 1
Currently searching the source tree to see what 
make
is actually doing.... truth is, I can't quite figure it out... one moment...
👍 1
h
No worries there! That perspective is really helpful for us - any feedback is welcome on things like our docs, with where things are confusing
a
Alright -- 
make
here is actually just making 
.o
files for every single corresponding source file, then 
make install
is actually linking them up into `.so`'s. Very strange, I've never seen code do something like that before. The important element is that it drops 
/usr/local/lib/libode.so
and it drops 
/usr/local/include/ode/
h
Cool, that’s similar to Noah’s setup, from what I can tell. There, Noah is planning to convert 
.pyx
files into 
.so
resources, from what I understand. In Pants terms, the codegen will go from 
CythonSources -> ResourcesSources
Check out Noah’s thread for the 
setup.py
command. On the Cython docs, it looks like there are multiple ways to invoke Cython. Using 
setup.py
may end up being easier, but we should also be able to get things working with your current workflow
a
Sweet. Reading that thread now!
❤️ 1
Copy code
tyler@DESKTOP-5PA52E3:~/ode-test/ode-0.16.2/bindings/python$ cat setup.py
#! /usr/bin/env python
from distutils.core import setup
from distutils.extension import Extension
from subprocess import Popen, PIPE, CalledProcessError


try:
    from Cython.Distutils import build_ext
except ImportError:
    raise SystemExit("Requires Cython (<http://cython.org/>)")

try:
    ode_cflags = Popen(
        ["pkg-config", "--cflags", "ode"],
        stdout=PIPE).stdout.read().decode('ascii').split()
    ode_libs = Popen(
        ["pkg-config", "--libs", "ode"],
        stdout=PIPE).stdout.read().decode('ascii').split()
except (OSError, CalledProcessError):
    raise SystemExit("Failed to find ODE with 'pkg-config'. Please make sure "
                     "that it is installed and available on your system path.")

ode_ext = Extension("ode", ["ode.pyx"],
                    extra_compile_args=ode_cflags,
                    extra_link_args=ode_libs)

if __name__ == "__main__":
    setup(
        name="Open Dynamics Engine",
        version="0.12",
        author="Gideon Klompje",
#        author_email="",
#        maintainer="",
#        maintainer_email="",
        url="<http://www.ode.org>",
        description="Bindings for the Open Dynamics Engine",
        long_description=(
            "A free, industrial quality library for simulating articulated "
            "rigid body dynamics - for example ground vehicles, legged "
            "creatures, and moving objects in VR environments. It's fast, "
            "flexible & robust. Built-in collision detection."),
#        download_url="<https://opende.svn.sourceforge.net/svnroot/opende>",
#        classifiers=[],
#        platforms=[],
        license="BSD License, GNU Lesser General Public License (LGPL)",
        cmdclass={"build_ext": build_ext},
        ext_modules=[ode_ext]
    )
(setup.py, by the way)
h
Also, I think that you’re now the third user asking about Cython. It’s possible we’ll want to upstream this to Pants’s Python implementation, especially if y’all or the other users were interested in open sourcing it. (In the meantime, if you’re able to develop this in a public repo, it’ll be easier for us to help out, such as code reviews. Totally okay to not commit to open sourcing it, e.g. publishing it for external usage. Only having the repo be public)
a
Will definitely ask my boss if this is something we I can open source tomorrow
💯 1
h
Sounds good. Regardless of being able to open source it, we’ll be happy to help
3 Views
Open in Slack
PreviousNext
Powered by