The fact that we partition and attach metadata is also key to the complexity as well. It helps us front-load expensive work, and also allows for more complicated formatting schemas (like if

black

HAD to be run using an IC exactly equal to the file's expected IC)

(LMK if you wanna call to discuss as well)

I think I got all the code mapped out LMK if you want me to push branch

@enough-analyst-54434 I’m going to start working on the 2.1.105 release if thats sounds good to you

um, let me stare at this paragraph for a few minutes.

lol it took me a while to even get here, so take your time. Liek I said call/current code is on the table for reference

so, although partitioning is now delegated to individual implementations, i’m not seeing why it ends up being different from the previous implementation in terms of how

fmt

partitions are built (these words are very overloaded, but still)

basically, we used to batch/partition by “the sequence of Request classes that applies”… and now we’re going to batch/partition by “the requests + assigned partitions”…?

The types are no longer relevant as the partitions themselves are type-encoded. Consider the following partitions:

A -> [f1, f2, f3], metadata=X
B -> [f2]
C -> [f4]

We should batch such that we invoke

[A,B]

in a serial run and

[C]

in another

ah… ok. and when you do that, you’re going to end up giving part of the declared partition to the Request.

The request itself receieves it's partition

(gotta bounce, will respond asyncly on mobile)

Another piece here is we don't want too reconstruct the partitions, since we want our input to match lint, for memoization

is CI timing out for people? I'm on my 6th rebuild of https://github.com/pantsbuild/pants/pull/16914 which just changes some strings

basically a different test times out each time and that kills the build

it's been timing out way more the past 3 weeks than normal imo. idk why, like if machines got slower. or pants got slower

For me it’s been the same test each time, but yes

I'm not very familiar with

py-spy

, but is there a way to profile just steps relevant to a backend? I don't really have a technical definition of "relevant" unfortunately, more of a semantic one. Like, would monitoring time spent in the

cc

module be practically good enough? Feels so-so https://www.pantsbuild.org/docs/contributions-debugging#profiling-with-py-spy Idea here is that I want to test having the

cc

backend compile/link natively vs creating Ninja build files, and then having ninja perform the compilation/linking step to see whether that step is a bottleneck (when compared with the other Pants overhead - digests, daemon, dep inference, cache checking, etc).

I noticed that

buf

and

terraform

have different strategies for handling not being dedicated to a single goal:

buf

has one subsystem:

buf

which has

format/lint_skip

and

format/lint_args

(sic on

format

)

terraform

has

terraform-fmt

terraform-validate

, etc... It seems to me what we don't have is a guide on how to shape subsystems, what conventions we have, etc... One piece of this would be to introduce "pluggable" options for goals (like injected field for a target). So

black-skip

would be

fmt-skip-black

(which would match the field name 🎉 ) Thoughts y'all?

filed https://github.com/pantsbuild/pants/issues/16927 for the

pex_test.py

case, which is unusual i think.

anecdotally, i’ve seen it hang/timeout at

test_lockfiles

at least twice.

yep, me too on that one! every time it's been

test_lockfiles

(since Stu added the output)

So per flaky test policy, we should skip that test, right?

we could try just skipping that method and seeing what happens, although that’s a bit of a shot in the dark. but yea.

i’ll do that.

Members of Pants Team, if you have given me your preferred email address (via https://docs.google.com/forms/d/1BTBvdaaPkJ_2Fp7zkSFRvD0dCBXrZp58-_V7uTyGECE) you should now have a Google Calendar invite to October's meetup and a welcome message from the brand new Google Group. If that's not the case, ping me please.

🧵 re using the multi-

Parameter

syntax of

await Get

(and the joys of rule graph errors)

@witty-crayon-22786 @happy-kitchen-89482 (and anyone else interested) here is the proposal for batching

pytest

calls you asked for on Friday: https://docs.google.com/document/d/1U0Q43bRod_EeVP4eQpcN36NMlxZ4CpRzTl2gMQ5HHvg/edit?usp=sharing

Does anyone recollect if pantsbuild-ci-bot actually does anything today? It was introduced in 2018 to provide auth for TravisCI

Just a love letter to @hundreds-father-404 and @witty-crayon-22786 for looking at my PRs today ❤️