Hey, do we need to preserve that you can call `Str...
# developmenth
hundreds-father-404
03/12/2021, 5:29 PMHey, do we need to preserve that you can call
StreamingWorkunitHandler.session()StreamingWorkunitHandler.session()local_pants_runner.pyhundreds-father-404
03/12/2021, 5:54 PMOkay this works great as a class-based context manager 🙂
đź‘Ť 1
w
witty-crayon-22786
03/12/2021, 6:08 PMare you working on the multiple-threads bit or the trailing tasks bit?
h
hundreds-father-404
03/12/2021, 6:11 PMmultiple threads
w
witty-crayon-22786
03/12/2021, 6:29 PMthat one can probably wait? i think it’s fairly challenging, and the impact is pretty minimal right now
witty-crayon-22786
03/12/2021, 6:30 PMmaybe you’ve already fixed it, but i think that to poll independently from multiple threads you’d need to change the rust-side polling method to keep multiple streams of workunits, or have one thread that polled and then sent copies to the two handlers or something
h
hundreds-father-404
03/12/2021, 6:44 PMIt was super trivial to get working on the Python side, but I haven't stress tested if it works In The Wild
w
witty-crayon-22786
03/12/2021, 6:53 PMeach call to “poll_workunits” has a sideeffect of consuming everything
đź‘€ 1
h
hundreds-father-404
03/12/2021, 6:54 PMK, will post the PR purely as a reference for the future but not pursue further today
hundreds-father-404
03/12/2021, 7:55 PMWhat was the motivation for tail tasks again? To ensure that when
--no-pantsdw
witty-crayon-22786
03/12/2021, 8:08 PMyea
witty-crayon-22786
03/12/2021, 8:08 PMthat’s the big one
witty-crayon-22786
03/12/2021, 8:09 PM…so i guess that we could decide how to prioritize it based on how many writes we’re losing currently.
witty-crayon-22786
03/12/2021, 8:09 PMmaybe less critical.
h
hundreds-father-404
03/12/2021, 8:10 PM<1% according to the metrics last week. So I think we probably leave it off for now
đź‘Ť 1
w
witty-crayon-22786
03/12/2021, 8:42 PMyea, fine with that.