https://pantsbuild.org/ logo
Join Slack
Powered by
Hey all, I'm about to work on a mid-sized embedded...
# plugins
w
Hey all, I'm about to work on a mid-sized embedded C/C++ project, with Python bindings - and I feel that this is a good time to also make headway into a more generic cpp plugin. If anyone has suggestions, recommendations, requests, I'm all ears: https://github.com/sureshjoshi/pants-plugins/issues/6
💯 3
❤️ 4
h
Other than general excitement, nope 🙂
w
Fair enough 🙂 I wasn't planning on this yet, but after working on a bunch of CMake hackery to do what Pants just "does" - I'm really inclined to not do that again. Several examples: • Dealing with compiling python bindings and then running python unit tests against specially compiled binaries... Nightmare • Having scripts or custom cmake commands for formatting/linting - not a huge problem, but messier than I feel it "should" be (whatever that sentiment is worth) • Compiling projects then linking to other projects which aren't intended to be statically linked - a bit of script soup • And about 10 other small annoyances, but nothing I haven't just dealt with
h
I guess I will have useful input when I understand better what you're trying to do
setuptools itself can build at least small C/C++ native bindings
Is this about building C/C++ as a standalone thing and then possibly also consuming object files by Python native bindings?
w
Yep, completely standalone C and C++ projects for embedded microcontrollers, and then a python thing on the side to make testing, hardware-in-loop, and a few other workflows easier
b
At my last company, we migrated to testing our C++ code with Python where possible because it was just easier 😂
💯 1
You can't test well underneath an API boundary, but right at it it becomes niiiiice
w
That's what I currently do with pybind11. Most importantly, it's for embedded systems (microcontrollers) which are pathetically untested in the wild, I'm depressed to say
Its a delicate balance with the python stuff and the C++ testing, because you have to have a binding layer, so that incurs code - so the python stuff has to be worth it. I found that state machines were a giant win when tested with python, And I think drivers were the other big win, since I could use python's mocking facilities - which are chef's kiss
f
I have thoughts on C and C++ backends. I wants Pants to be able to compile C shims to use in execution sandboxes.
And I bootstrapped the Go backend from scratch, so can provide what I learned from that experience.
I can write up a sketch of how development of a backend might proceed. Would that be helpful?
w
@fast-nail-55400 Yeah! That'd be great!
f
Will do this week.
❤️ 2
w
Whoaaaa - that's dope!
I also feel pretty good about the fact that I wasn't completely out to lunch with what I had in mind, and that some of the same questions I had in my head, you enumerated in the document. Specifically around system libs and whether headers should have their own target (my soul says no, but we'll see what happens as I port over some of my projects). Number 6, compiling and execution definitely feels like the crux of this to me.
👍 1
f
Decomposing targets into “compile C++ source to object file” and “link object files to library or executable” are probably the verbs for #6.
👍 1
The Go backend decomposes targets in a similar manner. The build graph of 
go_package
targets is turned into “compile the Go sources in this Digest to a package archive” and “link these Go package archives into a binary”
h
For Go, we made a split of 
build_pkg.py
vs 
build_pkg_target.py
. 
build_pkg.py
is highly generic rules to convert some Go files -> a 
__pkg__.a
file. 
build_pkg_target.py
converts targets into those generic compilation requests, e.g. dealing with 
dependencies
. I personally find that split really helpful for understanding what's going on
h
This is great!
I don't know if we need separate header targets, but we'd probably want to precompile headers, for performance
But that doesn't require separate targets, it's obvious what is a header file and what is not
h
Actually with "atom" targets corresponding to a single file, maybe we would want 
c_source
and 
c_header
"atom" targets, and then 
c_sources
generates both?
h
Possibly, not sure what that buys us?
f
only .c files get compiled, .h files don’t, so you’d want some way to distinguish them
➕ 1
but something that can be explored in developing native language backends
prior art: bazel’s 
cc_library
treats both as sources
(but would only compile the .c files)
h
.h files get precompiled
and we can distinguish them by the suffix
I assume 
./pants check path/to/header.h
should do something, i.e., precompile the header
TBH it's probably 6 of one, half a dozen of the other
w
Quick +1 to @hundreds-father-404 for mentioning in the JS/TS thread about getting some lint/fmt support in for the language, before going through the toil of the rest of the compilation. I just got a quick 
clang-format
running over my c/c++ code and working on adding 
cppcheck
(and some of my other pre-built linters) 🙂
💪 1
🙌 1
h
Sweet! That's in fact the strategy we often recommend for incremental adoption! Start w/ 
fmt
and 
lint
goals: https://www.pantsbuild.org/v2.10/docs/existing-repositories#2-set-up-formatterslinters-with-basic-build-files
w
Yep, it's not yet following the doc that tdyas put together, because I'll need some time to grok and plan, but it's just nice to kibosh some scripts or move stuff out of CMake every now and again
➕ 1
f
I will note that (5) in the doc (formatter / linter) does come before any the compilation stuff. 🙂
and could probably come before all of (4) (although tailor support is easy to do and super useful)
the doc is a long-term roadmap, feel free to break it down and adopt incrementally!
💯 1
w
Oh yeah for sure, I wasn't knocking it - just that I'm still in the grok phase, but I've added formatters and linters to a few other places, and that just feels like a win 🙂 Aside: small confession. I don't use the tailor command 😳 I like the single, simple BUILD file at the top of each project - anything more tends to feel overkillish, with some small exceptions
b
@wide-midnight-78598 there's a proposal for pants I'm pushing for which is basically that. One (-ish) build file
➕ 1
h
This has been my dream for so long
w
Wait, really? In the docs, it specifically mentions one per directory is better?
h
Well, my actual dream is no BUILD files at all except when needed
b
There's never enough time for all the features 😭
w
The experiment has begun. Linting and formatting with clang-format (no configuration) just uploaded https://github.com/sureshjoshi/pants-plugins/tree/main/pants-plugins/experimental/cpp https://github.com/sureshjoshi/pants-plugins/issues/6
🔥 1
h
Dammit Slack, not cooperating with my meme
Fine, we'll do this the hard way
➕ 1
w
+1 for AD references
h
... her?
😆 1
w
Hey all, reviving this thread for some relevant current work. Reviewing the 
go
build pipeline, and I've never used Go, so I guess one question I have here is... What is the minimal compilation unit? Is it a single .go file, or multiple .go files which constitute a "package"? Or a "module"? In the plugin, are we taking each 
.go
file, compiling that into an intermediate format, then linking all of those intermediate objects into a package/lib/executable? Or, is the minimal compilation unit a set of 
.go
files (N >= 1) which are all compiled into a package/lib/exe? And then these are linked with other libs?
To get a feel for the 
check
goal and compilable code, I'm playing around using Swift to make it really easy to compare and contrast my code. we have 
swift build
which is an all-in-one solution 
swiftc
which compiles individual files and automatically links them with whatever specified flags And we can even go one step further, and use the swift LLVM frontend basically, then generate 
.o
intermediate objects, which we can add a linker step to - and then create a single swift library or executable.
h
or multiple .go files which constitute a "package"?
This. You convert the dir into a 
__pkg__.a
file iirc. Then those can get linked to other packages
w
Thanks @hundreds-father-404, so does that mean that all .go files in a folder are necessarily compilable as a unit? Or is it just that we use Go's semantics to also pull in whatever dependencies it needs at 
check
time?
f
yes, go’s build model compiles whole packages at a time. (There is a probably a way to use 
go tool compile
to individually compile files into object files and then use 
go tool pack
to combine them into a package archive, but neither 
go
nor Bazel rules_go operates that way.)
👍 1
w
I'll re-evaluate Swift - might be a different structure to shift too. I can't build folders of swift files unless all first-party deps are there, so 🤷
Might just have to go module-centric
3 Views
Open in Slack
PreviousNext
Powered by