We made our Pants CI pipeline mandatory at my client, and so developers start having concerns that our Pants pipeline is slower than the regular CI pipeline. The crux is that our regular CI spawns a test job per library of our monorepo being modified, whereas the Pants pipeline simply runs

./pants --changed-since=origin/main --changed-dependees=transitive test

. And because GitHub default runners have pretty low performance, the Pants CI is on the average 4 to 5 times slower, because it uses a single machine. I wonder where I should be headed to speed this up. For the moment we don't cache Pants' state and use

--changed-since=origin/main

, because downloading the cache was quite slow. Ways forward that I see: 1. Use persistent runners, keep Pants' state 2. Use remote executors 3. Use a remote cache Any experience to share on what would yield the biggest speedup?

You may want to look into sharding the tests. See https://www.pantsbuild.org/docs/reference-test#shard

Thanks @curved-television-6568, exactly the kind of things I was looking for 🙏

In 2.15 you can also run tests in larger batches than one-per-file: https://www.pantsbuild.org/v2.15/docs/python-test-goal#batching-and-parallelism

@happy-kitchen-89482> yes remote caching is an option 🙂 I wanted to have opinions on the balance between providing a quick enhancement and a more long-term solution.