Another way of framing #2...have a strong "separation of concerns", know what Python excels at and what Rust excels at My point is that the FFI boundary is awkward, and it's likely better to do full Python or full Rust if you don't have a nice separation. Pants works because we have this well scoped scheduler that's abstracted away

3 might be boring / kinda obvious? And I'm not really sure that's the right takeaway? I more think it could be insightful to talk about how we test things in pantsbuild/pants, but I'm not sure what the actual takeaway is

cc @average-vr-56795 @red-balloon-89377 @early-needle-54791, really curious what y'all think given your wisdom from the earlier days of the engine

(FYI this talk is targeted for intermediate.)

I wonder if there are intersting things to say about the rust build problem of creating the

libengine.so

file and getting python to understand it correctly

this is a corner of the codebase I don't understand as well as some other people; and I remember when we were running into issues with this lib being big enough to cause problems with test sandboxing

https://pantsbuild.slack.com/archives/C0D7TNJHL/p1612389789060100?thread_ts=1612389223.057800&cid=C0D7TNJHL this isn’t actually true… you can include types in the the FFI method signatures, and

cpython

handles the conversion.

via the

FromPythonObject

and

ToPythonObject

traits… which we could implement for more things

`PyObject ~= Value`… but you don’t need to use

PyObject

in your method signatures.

lots of examples of more complex types here: https://github.com/pantsbuild/pants/blob/95ed385f71feec341f02ef5476bf95ade1a0b4dc/src/rust/engine/src/externs/interface.rs#L86-L402

BUT, mypy doesn’t know about those. which is why we also have https://github.com/pantsbuild/pants/blob/95ed385f71feec341f02ef5476bf95ade1a0b4dc/src/python/pants/engine/internals/native_engine.pyi, unfortunately. maybe there is a better way there.

…could totally generate that file too.

one minor correction: the relevant “x vs y” here is

cpython

crate vs

py03

crate… we used

cpython

at the time because

py03

required nightly rust. you don’t need to go any deeper than that though… afaik,

py03

started as a fork of

cpython

, and they’re relatively similar in design still.

Re "unoriginal" points: Key is to decide whether a point is unoriginal in this unusual context + unoriginal to this particular audience. The audience will be Python intermediates, but presumably have little or no Rust background. So a point that seems unoriginal from a Pants 2 contributor's POV may be very valuable for this audience to hear —for possibly the first time, or as reminder/context within an uncommon situation for an intermediate Pythonist. At the other end of the spectrum, things that are axiomatic within Python community can be powerful for calling out how a point of yours reflects community values and community best practices. So, sometimes an "unoriginal" point is worth giving specialness too. Re tests: As a listener, I also appreciate hearing not just that I need good tests but what kind of surprises might be around that corner. Are there things that I wouldn't usually think to test for in Python that need more probing when writing tests for Rust? Are there any things I get to take for granted, e.g. from stdlib, that free me from things that I'd expect to test in Python? Re overall: You've gotten to dig into deep and fascinating corners of both languages. Are there corners of Python or Rust that it's uncommon to explore as much as happened on developing Pants 2? These are fascinating to hear about, and bonus great if hearing about them now can help someone in that audience avoid pain. If you see any interesting opportunities for Pythonistas to piggyback on the code that Pants community has written here, where a 3rd party would be able to a leg up for their own thing unrelated to Pants, these are great to hear about those. Maybe it unlocks someone else's imagination...

it’s an important point.

Rust is as fast as C or C++, but much safer. it might have a higher learning curve, but in terms of new contributors actually successfully landing new code, the hypothesis is that it is easier.

@hundreds-father-404: and that reminds me: here is the original motivating document. https://docs.google.com/document/d/1C64MreDeVoZAl3HrqtWUVE-qnj3MyWW0NQ52xejjaWw/edit

idk, just anything you can do to get people to stop using C++ haha

i don't think the hardcore C++ people are reachable, but i think teams that might be considering doing extension work for the first time might be an easier sell

C++ has been around longer, so there's a larger pool of people who are familiar with it

and in general I think the software industry should be trying to move away from C/C++