https://pantsbuild.org/ logo
#development
Title
# development
s

sparse-lifeguard-95737

11/08/2022, 10:48 PM
I’m trying to figure out why
lint
is consistently getting OOM killed in our CI when testing out 2.15.x - is
py-spy
the appropriate tool for that sort of thing? Or is there a better tool to use?
w

witty-crayon-22786

11/08/2022, 10:58 PM
generally memory usage comes along with CPU usage, so yea:
py-spy
might be good to try
you can also try setting https://www.pantsbuild.org/docs/reference-stats#memory_summary … it will only be rendered on a successful run, but it might highlight the issue
s

sparse-lifeguard-95737

11/09/2022, 2:57 PM
core/util_rules/source_files.py:51
is ~1% of the time on 2.14, ~10% of the time on 2.15.x 🤯
hmmmm
awaiting the
MultiGet
of
HydratedSources
thought I already fixed this 🤔 but apparently more to be done…
ah I think I see it - 2.14.x was using
FrozenOrderedSet
everywhere, 2.15.x is using
CoarsenedTargets
, probably still a ton of duplicates getting passed around
CoarsenedTargets.closure()
is yielding duplicate targets - sometimes 10s of thousands of duplicates for Color’s dependency graph - is that intended? in 2.14.x the pylint impl wrapped it in a
FrozenOrderedSet
so it doesn’t look like new behavior, but it feels… wrong
w

witty-crayon-22786

11/09/2022, 4:53 PM
Yikes. Yea, bug.
closure
should only yield unique targets
If you want to assign an issue to me I can get a fix out today
👍 2
s

sparse-lifeguard-95737

11/09/2022, 5:04 PM
I put it in the 2.15.x milestone as a guess
w

witty-crayon-22786

11/09/2022, 5:10 PM
Thanks: have a doctor's appointment, and then will be on it.
wow. this one is embarassing. will have the fix out shortly.
s

sparse-lifeguard-95737

11/09/2022, 7:24 PM
TIL you can use emojis as commit messages 😂
😓 1
w

witty-crayon-22786

11/09/2022, 7:24 PM
i nearly put it in the PR description, but this will squash away my shame