I came here from poly-repo hell! 😃

If you have time, can you share your personal experience with poly repo hell? I have an uphill climb convincing management it's not the direction we should go

I can tell two stories. One of a system with a dozen or so services and a few front-ends, each code base in a different repo maintained by siloed teams. When features crossed silo boundaries they would take forever to deliver. A major problem was that each team had 3 stacks—dev, stage and prod—and all work had to flow through these stacks sequentially like a firehose through a straw. A couple of the teams formed a monorepo that included previously isolated code bases and, importantly, all the infrastructure as code. Suddenly they could deploy as many environments as they needed with all the services and apps in their repo. Even a single PR would ephemerally spin up all the infra (destroyed upon merge of the PR) and the service discovery could be done very easily and reliably because of conventions enforced by the monorepo tooling. So many problems previously solved with repetitive one-off fixes were now solved with conventions or automation. Multiple features could be moved through QA at once. Or prototyped cheaply, user-tested and abandoned if they weren't right. The single repo also improved cross-team communication at the cost of a bit more noise. But the big win was the monorepo coupled with IaC. Those teams were working in Typescript so used first Lerna then pnpm to accomplish roughly what Pants does. The other story is of a team of data scientists I now lead working on training ML models that will be embedded into production applications. Most data scientists use code quite differently from the way software engineers work. Modular code is not a big thing (they usually write scripts or Jupiter notebooks) and copy-pasting is common. Publishing versioned packages to share code? And checking that all consumers of their shared packages could be updated to the new version? Hard behavior to enforce. Having code copy-pasted with modifications across multiple code repos made the behavior difficult to discover and impossible to fix without adopting the publishing model. In the monorepo my IDE immediately notifies me of copied code and the monorepo makes it much easier to share modules between our many projects. Also makes it easier to automate workflows that touch different projects using tools like GitHub Actions. At least that's my pitch. Since we're working in Python Pants looked like the best choice to manage the repo. But I hope the project will persist in its polyglot ambitions!

Beautiful, thanks for sharing!

This is great. @plain-monkey-49051 Would you be interested in allowing us to publish this on blog.pantsbuild.org? With your byline, of course. Joshua is definitely not the only person who would appreciate being able to point to real-world experiences of polyrepos in making their case for a monorepo.

Sure, @busy-vase-39202 . Maybe I could write it up a bit better than that before you post. I'd be happy to do that.

Thanks! That'd be great. Much appreciated. Feel free to DM me if you have any questions or I can be helpful in any other way.