https://pantsbuild.org/ logo
#development
Title
# development
h

hundreds-father-404

07/22/2021, 6:37 PM
hey @witty-crayon-22786, I have a quiet weekend and might sprint on getting PyO3 over the threshold as a personal project. Any ideas on where to go incrementally, outside of getting Stdio ported? Almost everything remaining requires the Scheduler
I'm wondering if some things like setting up Tasks could be ported..where the FFI wrapper is worked around and the two crates are interacting solely w/ Rust types. I'm still a little hazy on if it's valid for engine/ to depend on engine_pyo3 or vice versa, so wondering if maybe creating a
core/
create w/ things like
<http://session.rs|session.rs>
makes sense?
w

witty-crayon-22786

07/22/2021, 7:16 PM
I’m still a little hazy on if it’s valid for engine/ to depend on engine_pyo3 or vice versa, so wondering if maybe creating a 
core/
 create w/ things like 
<http://session.rs|session.rs>
 makes sense?
it can’t, yea.
fwiw: we used to have an “engine ffi” crate (iirc), but i smashed it into the engine crate during the CFFI removal
h

hundreds-father-404

07/22/2021, 7:18 PM
How come? I've noticed that FFI is no longer purely in that folder "externs". I wonder if there's value in going back to having more delineation so our FFI layer's surface area is even smaller
w

witty-crayon-22786

07/22/2021, 7:19 PM
that might be a clean separation… the only thing is that the non-public portions of the engine still depend on rust-cpython, so the broken out inner crate would still too
well, the
<http://interface.rs|interface.rs>
file is huge, and almost encapsulates our API to the python code
h

hundreds-father-404

07/22/2021, 7:20 PM
Would a prefactor make sense to make a core/ crate, without porting any of the code to PyO3 yet?
w

witty-crayon-22786

07/22/2021, 7:20 PM
what you have in
engine_pyo3
(just the interface) would be nice.
h

hundreds-father-404

07/22/2021, 7:21 PM
Would a prefactor make sense to make a core/ crate, without porting any of the code to PyO3 yet?
That is, I'd like for session.rs et al to not know about FFI. Not sure if that's feasible
w

witty-crayon-22786

07/22/2021, 7:21 PM
possibly, yes. although i expect that extracting
<http://interface.rs|interface.rs>
and anything else you can into an `engine_cpython`crate might make more sense than the other way around.
👍 1
h

hundreds-father-404

07/22/2021, 7:22 PM
On that note, what's the motivation for a top level crate? I would naively want to organize this into crates ffi/ and core/, with no top level
w

witty-crayon-22786

07/22/2021, 7:22 PM
having said all that: your original question was whether the
engine
crate could depend on the
engine_pyo3
crate: and it can’t, but in the factoring i’m suggesting, it wouldn’t need to
👍 1
the
engine_pyo3
crate is the public interface… if the
engine
crate needs to depend on
pyo3
types, it can just do that
i think that it would mostly improve clarity to have an extracted
engine_cpython
crate.
On that note, what’s the motivation for a top level crate? I would naively want to organize this into crates ffi/ and core/, with no top level
not sure what you mean. but a Cargo workspace must have a top level crate
👍 1
h

hundreds-father-404

07/24/2021, 4:36 PM
Okay given the issues in https://github.com/pantsbuild/pants/pull/12419... I'm going to take a new approach for PyO3 migration. Port some of the abstractions like
PyDigest
but have both a native_engine and native_engine_pyo3 implementation for now. And don't actually switch Python to use PyO3 yet Instead, write some unit tests to assert the equivalence of the two types Not great to violate DRY, but I think this allows some progress to reduce risk of a big migration