I wrote down some ideas of projects for "pants core infra". Some (most?) of these don't originate with me. Just trying to collect them in one place. Would love feedback and any additional ideas: https://docs.google.com/document/d/1KDe25xpYQXVhgpBYL504T-_N_I243o-TGIlEqTO8lQg/edit?usp=sharing

This looks great! Another REAPI-related thing I would add is the ability to cache multiple facts that were derived from a single process run. For example, if we ran tests 1-10 in a single process batch, it would be great to cache the facts “test N passes” for N=1..10. This may be controversial since tests could theoretically behave differently when run separately vs in a batch, but in practice that should not usually be the case, and users definitely accept this as a possibility if they are using batching to begin with

I put this as a comment in the doc, but I've seen the idea of realms floated to solve the problem of slow target generation from some backends like Go infecting the entire repo. I had this problem, but don't love the idea of realms since, I have python tests of docker containers that have other language services in them (not Go any more since we just tore that all out). An alternative proposal to realms: If target generation is directory scoped (e.g. things in go/ won't create targets in python/), then you only need to run target generation for the directories you depend on with unresolved targets rather than running it globally. This allows having a dependency from python->go when you want it, but avoid it when you don't. If target generation/dependencies are not so neatly scoped under the hood, maybe it is possible to make the scope of target generation clearer in some way.

Well, I think if there was an explicit dep across realms the expected thing would happen? The idea is to not do things like bootstrap the Go toolchain unless there is some Go code in the transitive deps of the cmd line specs

Ah, I guess that makes sense if you have path-based definitions of realms (as the source roots idea implies)

Update: I appended some new ideas to the Core Infra Ideas doc.

Link?

Top of the thread?