https://pantsbuild.org/ logo
Join Slack
Powered by
Hm, `./v2 fmt` is not terminating (or maybe just t...
# development
h
Hm, 
./v2 fmt
is not terminating (or maybe just taking over a minute) in the 
fmt
stage, meaning after all the formatters have run and write to the build root. It was also taking 100% of my CPU according to 
top
. I didn’t notice this on Friday. Maybe the tokio upgrade has something to do with it?
cc @witty-crayon-22786
Also, when I ran 
./v2 fmt src/python/pants/util/strutil.py
(no changes), it exited cleanly and quickly. So maybe the issue is when we modify the build root
w
can bisect if you have a repro
h
hm I'm seeing this myself I think
h
I’ll bisect in a couple minutes and try to get the reproduction more precise
w
py-spy shows it's spending time in materialize_directories... so yea, seems like it.
and since it's not more specific than that, can assume it is spending time in the rust code. so next step is to attach a native-code profiler. on linux that might be 
perf
... on OSX, it's 
Instruments
ok. so yea, definitely something inside 
materialize_directories
, but the native profile doesn't indicate that any particular function in there is taking any longer... just possibly that it's looping more than it should be.
will bisect.
❤️ 2
h
Cool, thank you Stu!
w
mm. yea, it looks like it was definitely github.com/pantsbuild/pants/pull/9467
will revert and report.
h
Huh. I wonder what caused it. Thank you for investigating and reverting
👍 1
w
they're pretty responsive. i'll report it upstream.
❤️ 1
h
I wonder why CI didn't catch this
h
I don’t know how CI would catch it. Things still work, they just take substantially longer. We already have major flakiness from timeouts, so we bias towards a really generous timeout, which means we wouldn’t catch it
w
have you seen the run actually finish?
...oh. i guess you saw it finish for a smaller run?
h
Yes, when running on a single file. I got too impatient when running against 6 targets (—changed-since)
h
not locally. and I noticed that the linter job on https://github.com/pantsbuild/pants/pull/9310/checks?check_run_id=565041648 was failing when I'm pretty sure it had passed before, so I chalked it up to this issue
h
I don’t think this affects the lint goal because it never materializes directories. Locally, V2 lint is speedy for me
w
yea, wouldn't affect lint.
h
That linter job never calls fmt, only lint
h
ah
w
...oh. so that's why we wouldn't see it in CI.
there are tests of materialize directory though. hm.
h
Well, sort of. We do still run fmt in all the integration tests
w
but yea, just slower.
👍 1
h
Huh. 
./v2 test --debug
is really really slow to start the test in between the time of 
--v2-ui
and when the output is supposed to render. I think this is the 
InteractiveRunner
piece that took a perf hit
w
possibly. use py-spy
h
./v2 test
behaved normally
w
(InteractiveRunner uses 
materialize_directory
, so it's possible. but easy to confirm with py-spy.)
👍 1
h
So it looks like 
./v2 fmt
is actually overwriting the build root just as quickly as normal. The files are being updated properly. The issue is not terminating.
w
ok. i wasn't able to get a simpler repro on https://github.com/pantsbuild/pants/issues/9476 , so I guess i'll need to resort to logging.
👍 1
e
revert is ready to merge
💯 1
w
thanks
Open in Slack
PreviousNext
Powered by