Hey, how are people sharding/distributing CI builds in larger monorepos using Pants?

One natural way is to split by goal. Eg have one shard for lint and one for test. Another pattern is to split by targets. Eg have the first shard do

test folder1 folder2...

and the second shard do

test folderk...foldern

. One limitation of this approach is that you might forget to add new folders to CI, unless they’re already a sub directory of one of the targets you have Finally, you could implement a mechanism to automatically shuffle your tests. Check in Pants’ github the file

ci.sh

for how we do this with our repository. (I’m afk, sorry I can’t grab the link for you)

Both the junit and pytest runners have

--test-shard

flags, see the output of

pants help test

for details.

Thanks, I’ll take a look at all of this