https://pantsbuild.org/ logo
#development
Title
# development
h

hundreds-father-404

03/12/2021, 5:29 PM
Hey, do we need to preserve that you can call
StreamingWorkunitHandler.session()
multiple times, rather than only once per
StreamingWorkunitHandler
instance? In practice, we only ever call
.session()
once in
local_pants_runner.py
, but a test calls it twice on the same instance and it's making some perf work I'm doing harder to land
Okay this works great as a class-based context manager 🙂
👍 1
w

witty-crayon-22786

03/12/2021, 6:08 PM
are you working on the multiple-threads bit or the trailing tasks bit?
h

hundreds-father-404

03/12/2021, 6:11 PM
multiple threads
w

witty-crayon-22786

03/12/2021, 6:29 PM
that one can probably wait? i think it’s fairly challenging, and the impact is pretty minimal right now
maybe you’ve already fixed it, but i think that to poll independently from multiple threads you’d need to change the rust-side polling method to keep multiple streams of workunits, or have one thread that polled and then sent copies to the two handlers or something
h

hundreds-father-404

03/12/2021, 6:44 PM
It was super trivial to get working on the Python side, but I haven't stress tested if it works In The Wild
w

witty-crayon-22786

03/12/2021, 6:53 PM
each call to “poll_workunits” has a sideeffect of consuming everything
👀 1
h

hundreds-father-404

03/12/2021, 6:54 PM
K, will post the PR purely as a reference for the future but not pursue further today
What was the motivation for tail tasks again? To ensure that when
--no-pantsd
, we're blocking on finishing the cache write?
w

witty-crayon-22786

03/12/2021, 8:08 PM
yea
that’s the big one
…so i guess that we could decide how to prioritize it based on how many writes we’re losing currently.
maybe less critical.
h

hundreds-father-404

03/12/2021, 8:10 PM
<1% according to the metrics last week. So I think we probably leave it off for now
👍 1
w

witty-crayon-22786

03/12/2021, 8:42 PM
yea, fine with that.