Does pants support distributed testing?

Yes, this is a main motivation for why we built the V2 test runner. With it, Pants automatically distributes testing and has sane caching of those results, e.g. only re-running targets that had changes (either to the test code or any dependency) What language are you hoping to use distributed testing? Currently, V2 is implemented for Python

Hi Eric, thank you for your response. Yes, I know that pants can use a centralized cache. I have a centralized cache server for our CI server. It's not really distributed.

Yes, I know that pants can use a centralized cache. I have a centralized cache server for our CI server. It’s not really distributed.

The V2 system is different and an entire re-architecture of how Pants does caching and parallelizes work (originally inspired by how Bazel does things). For example, when running tests with V2, you can use remote execution to have each test target run with its own worker. So if you have 100 tests and 100 workers, you can run all 100 tests concurrently. With V2's caching, if you rerun those 100 tests and only 2 actually changed, then you’d immediately get the results for 98 of the 100 tests. Locally, you can configure how many threads you want to allow to get the same effect.

Does pants support running multiple workers in a pool, then use that pool to run the unittests?

Yes. To use this, you’d switch to using the V2 test runner (which Pants has been using for 5 months now and Toolchain has been using for 3 months), then possibly tweak

--process-execution-local-parallelism

if you don’t like the default If you’re interested in trying it out, we would be eager to help get you set up with the V2 test runner! cc @happy-kitchen-89482

process-execution-local-parallelism

Does that flag run only on a single machine?

Yes, and there’s

--process-execution-remote-parallelism

to complement it

Ok, let me give you a background. I'm trying to write a pants plugin to collect tests, and distribute each test in Kubernetes, and report the results back to pants cli. If I want to run a flakiness test, the plugin should be able to run the same test in parallel within the Kubernetes cluster, and Kubernetes takes cares of scaling up and down the resources, and isolation for infrastructure such as databases.

Hi @fresh-architect-74417. To follow up on Eric's replies - Pants v2 is very different from v1 (which it sounds like is what you're using). It's basically an entirely new system. It already supports distributing tests (and build work in general). So it sounds like what you might want is to A) switch to using pants v2, and then B) set up a remote execution cluster (aka a "build farm"), or use an existing one. This is probably a lot more straightforward than rolling your own test distribution system in Pants v1. So first question is - what languages are you looking to support? Pants v2 currently supports Python, with other language support to follow soon.

Hi Ben, thank you for your response. Will that work with a custom pants plugins? If I have a new build target type. Would that work? Also, does the build farm supports flakiness tests? (e.g: Run the same test over 100 time.

Yes, v2 supports custom rules via plugins (in v2 work is done in "rules", as opposed to v1 "tasks"). We're still figuring out the exact API for custom target types. It should be better defined in a couple of weeks. Can you give an example of your use case?

Two things I want to gain: 1-Add custom build targets that pants does not support and run them in a distribute way. The custom targets will be a custom pants plugin to support javascript tests and custom pytest test 2-Flakiness test. Run the same test N number of times.

Re flakiness tests, there's no built in support right now, but it could be added (e.g., in a custom rule in a plugin).