Let's say someone added

RustPython

to our engine's deps. Now let's say someone wanted to rewrite our Python dep parser script in Rust. What do we think would yield the best perf in the average case? 1. An intrinsic which leverages the

local_cache

to cache Digest -> results ◦ Doesn't support remote cache 2. Build a full executable and use that 3. An intrinsic without caching I'm guessing 1 is enough because the code is likely faster than the latency of the network + overhead of the remote download code

If it is fast enough, which I suspect that it could be, then I don't think you have to worry about caching it at all.

Oh right, I remember there being 3 options. I forgot that one

I also suspect that's the easiest distribution model

Yeah building an exe seems time-cosnuming and space-wasteful

Well, it's 1 but without caching unless it looks necessary

How hypothetical is this …… hmmmmmm…..?

👀

I was inspired by Charlie Marsh’s new company post - all Rust tooling all the time

Hehe I'll be seeing him this weekend 🙂

Ask him if he’ll lend 5% of his brainpower to fixing the incredibly slow JS ecosystem. Prettier is great, but man - I could hand format faster sometimes

WIP, but https://github.com/thejcannon/pants/blob/intrinsic-parsing/src/rust/engine/src/externs/python_parsing.rs

Keep in mind that dep inference supports custom python code (e.g., for extracting django deps) so that would need to be considered.

Yeah that's why I CCd you

I don't think it's ideal to have two mechanisms for dep inference

No not ideal. But really really fast. 😈

There are already multiple implementations of inference: it's generic in that regard.

To be fair the existing customization points to the deo inference script could've just been unions to start with

I think rewriting the custom stuff in rust is not that prohibitive either

One thing to note though is that it would put a cap on supported python versions, unless you kept the old implementation as a fallback

Oh hmmm, true

Certainly!