hey @witty-crayon-22786, I have a quiet weekend and might sprint on getting PyO3 over the threshold as a personal project. Any ideas on where to go incrementally, outside of getting Stdio ported? Almost everything remaining requires the Scheduler

I’m still a little hazy on if it’s valid for engine/ to depend on engine_pyo3 or vice versa, so wondering if maybe creating a 

core/

 create w/ things like 

<http://session.rs|session.rs>

 makes sense?

it can’t, yea.

How come? I've noticed that FFI is no longer purely in that folder "externs". I wonder if there's value in going back to having more delineation so our FFI layer's surface area is even smaller

that might be a clean separation… the only thing is that the non-public portions of the engine still depend on rust-cpython, so the broken out inner crate would still too

Would a prefactor make sense to make a core/ crate, without porting any of the code to PyO3 yet?

what you have in

engine_pyo3

(just the interface) would be nice.

Would a prefactor make sense to make a core/ crate, without porting any of the code to PyO3 yet?

That is, I'd like for session.rs et al to not know about FFI. Not sure if that's feasible

possibly, yes. although i expect that extracting

<http://interface.rs|interface.rs>

and anything else you can into an `engine_cpython`crate might make more sense than the other way around.

On that note, what's the motivation for a top level crate? I would naively want to organize this into crates ffi/ and core/, with no top level

having said all that: your original question was whether the

engine

crate could depend on the

engine_pyo3

crate: and it can’t, but in the factoring i’m suggesting, it wouldn’t need to

Okay given the issues in https://github.com/pantsbuild/pants/pull/12419... I'm going to take a new approach for PyO3 migration. Port some of the abstractions like

PyDigest

but have both a native_engine and native_engine_pyo3 implementation for now. And don't actually switch Python to use PyO3 yet Instead, write some unit tests to assert the equivalence of the two types Not great to violate DRY, but I think this allows some progress to reduce risk of a big migration