@happy-kitchen-89482 https://github.com/pantsbuild/pants/pull/9469 https://github.com/pantsbuild/pants/issues/13749 Thoughts on this one now? You mentioned it wasn't worth the complexity in the past, but now that we're using 3.11 - its just updating the dataclass decorator

Have you thought about using optimized libraries like msgspec instead? This might also speed up communication with rust engine

Neat, I've never seen that lib. Just off the cuff, me personally, I'm generally against adding more libraries and replacing basically everything in our codebase with an external solution. We'd probably need to figure out how to actually benchmark the performance implication of just the Python side. I like the slots thing, because it's literally a free win with minimal effort, but compared to all the work done on the Rust side - I don't envision it being a huge, noticeable win for day-to-day performance (I would love to be wrong)

Well, personally I don't have any problems with 3rdparty dependencies if they actually come with performance improvements. The only thing that's annoying is that pants wheel pins all the dependencies with an exact version instead of >= which makes it very hard to use any 3rdparty library in in-repo plugins

Yeah, but like, replacing 2000 instances of classes with an object from a 3rd party library 🤷 Memory-wise, slots will re-coup a lot of the memory, and the bulk of our actual perf issues are likely in Rust-land anyways, which I'm chipping away at

Yeah, but it improves performance more than slots, object creation time might outweigh the import times and give speedup in total. And it's much easier to use than slots. I'm just saying it might be worth running some real pants benchmark before blindly going with slots

Interestingly, depending on whcih objects we put slots on - there is a 30% memory usage reduction, OR, an increase of about 10%

With 3.11, is this "just" using

slots=True

everywhere?

That's what I tried and then did some memory profiling. My first guess is that

EnvironmentAware

classes shouldn't have slots - but I haven't dug in too much. The cost of the python classes vs what happens in Rust doesn't seem to be comparable (though, it's always better to use less memory where possible, obvs)