https://pantsbuild.org/ logo
#general
Title
# general
e

eager-librarian-15922

12/19/2018, 12:34 PM
Hey, how are people sharding/distributing CI builds in larger monorepos using Pants?
h

hundreds-father-404

12/19/2018, 4:17 PM
One natural way is to split by goal. Eg have one shard for lint and one for test. Another pattern is to split by targets. Eg have the first shard do
test folder1 folder2...
and the second shard do
test folderk...foldern
. One limitation of this approach is that you might forget to add new folders to CI, unless they’re already a sub directory of one of the targets you have Finally, you could implement a mechanism to automatically shuffle your tests. Check in Pants’ github the file
ci.sh
for how we do this with our repository. (I’m afk, sorry I can’t grab the link for you)
h

happy-kitchen-89482

12/19/2018, 5:28 PM
Both the junit and pytest runners have
--test-shard
flags, see the output of
pants help test
for details.
So for example you can run
./pants test --test-pytest-test-shard=0/10 src/python::
to run the first of 10 shards, and similarly for junit.
🎉 1
e

eager-librarian-15922

12/20/2018, 10:40 AM
Thanks, I’ll take a look at all of this
5 Views