How do we feel about a #[native_rule] proc macro? ...
# development
w
How do we feel about a #[native_rule] proc macro? I want one because I want to port rule code to rust, and I think rust ports sharing shape with python rules is powerful. I also want one to make the instrinsic-to-intrinsic have 0 python frames and largely be about IO and that opimization space, instead of the weird place it is in now where python runtime is non-zero even when intrinsic rules call each other. I did this in a bit of a exploratory hacking to find the right shape trying to make
find_ancestor_files
(a very python hot rule in my work repo) portable without (what I hope) impact on readability. Design choices are: implicitly-discovery being included in the proc-macro to mirror python. Not as powerful as the python ast parser, but maybe good enough for native rules? https://github.com/pantsbuild/pants/blob/5ed90d488a23da2423ab55d7984e72e79b0f0ecb/src/rust/engine/src/native_rules_builtins.rs#L109 This work depends on my currently open PR:s.
🎉 2
I'll also add that coding in rust is more enjoyable to me than python (because python is my dayjob) which increases the likelihood of my contributions - maybe I have likeminded open source members?
The design would be to still expose all rule products and call_by_name functions to the plugin system. It'd be implemented in pyo3/engine though
h
I definitely enjoy Rust more than Python at this point
And I agree that being about to port rules to Rust without python intervening would be excellent
c
Naive questions: Would this allow a ported rule to re-use the existing python tests with no changes to said tests? (Or is that already the case?)
w
rule-runner based aye
h
Not sure how much you're using LLMs for this work, but from my past couple of months experimenting with Claude in the pants repo, which has been a mixed bag, I have a sense that it might be relatively helpful in this kind of work.
c
Are the series of ports a pre-req for a rule macro, because all rules happen to use those classes? Or a different reason?
w
Yes, sort of. Any type you want to "name" in rust has to be a pyo3 struct for it to be ergonomic to use. Writing native rules is way too verbose without it, and these base types (e.g Target, SourceField) are ubiquitous