Any thoughts on a Pants equivalent to Bazel s `exclusive` ta Pants #general

Any thoughts on a Pants-equivalent to Bazel's `exc...

bitter-ability-32190

05/05/2022, 5:09 PM

Any thoughts on a Pants-equivalent to Bazel's

exclusive

tag for tests? We have one test which if it doesn't get sufficient CPU resources will become flaky.

➕ 1

👀 1

bitter-ability-32190

05/05/2022, 5:10 PM

Tangentially related, can we specify the # of process slots to consume for a test? It would solve this problem, and maybe help other problems we have?

rhythmic-morning-87313

05/05/2022, 5:12 PM

I had a same question actually! Q13. Good to know that Bazel calls it "exclusive".

witty-crayon-22786

05/05/2022, 5:44 PM

Tangentially related, can we specify the # of process slots to consume for a test?

yea, we should probably do this. but note that with our existing mechanism, it wouldn’t give you guaranteed slots. so it might be independent from an exclusive flag.

bitter-ability-32190

05/05/2022, 5:44 PM

Can you elaborate on "it wouldn’t give you guaranteed slots"?

witty-crayon-22786

05/05/2022, 5:45 PM

there was another thread about this recently over here: https://pantsbuild.slack.com/archives/C046T6T9U/p1648616964165199

witty-crayon-22786

05/05/2022, 5:46 PM

@bitter-ability-32190: the concurrency flag added in https://github.com/pantsbuild/pants/issues/9964 is the “maximum” amount of concurrency that a process is capable of, but we will still schedule the process with less than that.

witty-crayon-22786

05/05/2022, 5:47 PM

i.e., if you say that you are capable of 256 concurrency because you’re operating on 256 files, we might only give you 8 because there are already processes running, and the machine only has 16 cores.

bitter-ability-32190

05/05/2022, 5:47 PM

Ah right

witty-crayon-22786

05/05/2022, 5:50 PM

not sure whether those are max/min values… but it’s still a bit challenging because if the minimum is configured as an absolute value (rather than a relative value), if you say that you need 16 cores to run, we’d need to decide what to do on an 8 core machine.

witty-crayon-22786

05/05/2022, 5:50 PM

(“exclusive” would be a relative value… “all the cores please”)

witty-crayon-22786

05/05/2022, 5:51 PM

but yea, this is definitely an area where we should make more changes.

witty-crayon-22786

05/05/2022, 5:59 PM

so… yea. i could imagine this as converting the concurrency_available parameter to a (rust-style) enum of

Range(max: int, min: int | None = None) | Exclusive

… and then exposing it to pytest the same way we expose the execution-slot-var

✅ 1

🙌 1

witty-crayon-22786

05/05/2022, 6:16 PM

i do wonder whether folks need more subtlety than “exclusive: bool”, and whether they need groups (these tests must run exclusive of one another). it sounds like in your cases (@bitter-ability-32190, @rhythmic-morning-87313), a bool would be sufficient?

bitter-ability-32190

05/05/2022, 6:18 PM

One test in particular, yes. I've also seen tests that we have that use the bazel

cpu:N

tag (documented here) but generally I'm just ignoring those 🙈

rhythmic-morning-87313

05/06/2022, 2:05 AM

i think just the exclusive option should suffice me. In my case, it is not related to the performance but concurrency of the test case.

rhythmic-morning-87313

05/15/2022, 4:25 AM

How is it going with the

exclusive

option? One motivation behind me is to run test cases that requires spawning of Redis clusters using docker-compose exclusively to each other, to prevent port overlaps, i.e., tests with fixtures that have side effects to other tests and their fixtures.

rhythmic-morning-87313

05/15/2022, 5:26 AM

I just created an explicit issue for the

exclusive

option: https://github.com/pantsbuild/pants/issues/15482

rhythmic-morning-87313

05/15/2022, 6:07 AM

ultimately i think it would be nice if there is a guide to have external dependencies like docker containers for tests in a pants way

👀 1

bitter-ability-32190

05/15/2022, 2:02 PM

I thinks that's lightly possible because runtime dependencies will give you the docker image in the sandbox. But you'd have to run it yourself

witty-crayon-22786

05/16/2022, 10:13 PM

thanks for filing… that usecase seems like it could potentially use the execution_slot_var facility: commented there https://github.com/pantsbuild/pants/issues/15482#issuecomment-1128188679

👍 1

rhythmic-morning-87313

05/17/2022, 3:31 AM

Great! This will help me to configure unique port numbers, etc. for parallel execution

rhythmic-morning-87313

05/17/2022, 6:42 AM

image.png

rhythmic-morning-87313

05/17/2022, 6:42 AM

this is another way to do it by letting OS assign a free port number (previously it was hardcoded)

rhythmic-morning-87313

05/17/2022, 6:43 AM

for a cluster, i'm going to use the execution slot variable

bitter-ability-32190

05/19/2022, 4:50 PM

@witty-crayon-22786 any thoughts on a way to hack this Pants 2.11 in a plugin? I'm coming up empty

bitter-ability-32190

05/19/2022, 5:10 PM

I'm hoping I don't need to implement something injected into pytest which uses a RW file lock or something equivalently clever. Maybe a copied-and-pasted

test-exclusive

goal which I run after

test

and "does the right thing"?

witty-crayon-22786

05/19/2022, 5:15 PM

that, or filtering to a tag like in the other thread?

witty-crayon-22786

05/19/2022, 5:16 PM

i.e

./pants --tags=-exclusive test :: && ./pants --tags=exclusive --process-parallelism=1 test ::

bitter-ability-32190

05/19/2022, 5:16 PM

Well I have several exclusive tests, so I guess it'd be a script which runs

./pants test

for each file with the tag? 🤔

witty-crayon-22786

05/19/2022, 5:16 PM

or that, yea.

bitter-ability-32190

05/19/2022, 5:16 PM

Ohhh

--process-parallelism

Interesting 🤔

bitter-ability-32190

05/19/2022, 5:17 PM

Then I'm just slightly bothered by https://github.com/pantsbuild/pants/issues/15012

5 Views

Open in Slack

Previous Next