as i debug a cache-related inconsistency in the ru...
# developmentw
witty-crayon-22786
10/21/2022, 6:11 PMas i debug a cache-related inconsistency in the rust bootstrap, i feel compelled to clarify that it wouldn’t be impossible for Pants to dogfood it’s own docker support + remote exec for all of wheel building and testing. it would just require what the bootstrap problem usually requires (afaik): using a stable or pinned binary version of Pants to build Pants, rather than Pants-from-sources.
👍 1
witty-crayon-22786
10/21/2022, 6:12 PMit’s unclear whether that would make the contributor experience better or worse… but it would be very nice to rely less on GitHub Actions.
👍 1
witty-crayon-22786
10/21/2022, 6:15 PMthe big question is just “which binary version of Pants?” and consequently also: “how often is the version bumped?”
c
curved-television-6568
10/21/2022, 6:51 PMI think a third question is good to consider all the way through as well, and that is how to bootstrap this process in the absence of a pre-existing Pants binary. (to not get locked into a chicken-and-egg problem)
e
enough-analyst-54434
10/21/2022, 7:48 PMWell, there is no solution for that IIUC. This is just like with self-hosting compilers. You simply must rewind far enough back in time to build the root version. Now that version uses some other compiler with the same problem and you go all the way back to the original compiler (assembler) written directly in machine code and then unwind the stack.
b
broad-processor-92400
10/21/2022, 8:12 PMI don't know if this is helpful, but I've worked on two(ish) bootstrapping compilers with different “which pinned version” policies:
• rust: most recent beta currently (https://rustc-dev-guide.rust-lang.org/building/bootstrapping.html#stage-0). I think it used to be most recent stable, but I don't know when or why it changed.
• Swift: all stable versions within the last year (https://forums.swift.org/t/implementing-parts-of-the-swift-compiler-in-swift/59524), plus latest beta and the
main👍 1
broad-processor-92400
10/21/2022, 8:25 PMSome ways I know of to solve the chicken-and-egg problem without going through the whole bootstrap are (I may be repeating content from that video, sorry):
1. cross compiling: if there's binaries for one platform, use that to create the first binaries for a new one
2. minimal reimplementation: a minimal implementation of the compiler/build system in a different language/build system, that supports just enough to build the real system. This can cut a bunch of corners (eg no error handling, slow, not supporting every feature), since it is designed to build a single code base that's known to be valid. https://github.com/thepowersgang/mrustc is an example for Rust.
3. Alternative implementation: generalising the previous point, one can also just use a full reimplementation (eg using gcc to compile clang)
1 is typically a lot less work than 2 and 3
👍 1
c
curved-television-6568
10/21/2022, 9:15 PMWell, there is no solution for that IIUC.I think in our case there is, if we have at least one supported way of building an initial bootstrap version from source, then that bootstrapped version can be used to build the real version as the bootstrap version may be built without relying on docker or remote execution..
w
witty-crayon-22786
10/25/2022, 5:04 PMI think in our case there is, if we have at least one supported way of building an initial bootstrap version from source, then that bootstrapped version can be used to build the real versionyea, that’s right. that’s related to @broad-processor-92400’s:
1. cross compiling: if there’s binaries for one platform, use that to create the first binaries for a new oneif we continued to bootstrap Pants as we do today, but then we used
environmentsdocker👍 1