https://pantsbuild.org/ logo
Join Slack
Powered by
I’m trying to figure out why `lint` is consistentl...
# development
s
I’m trying to figure out why 
lint
is consistently getting OOM killed in our CI when testing out 2.15.x - is 
py-spy
the appropriate tool for that sort of thing? Or is there a better tool to use?
w
generally memory usage comes along with CPU usage, so yea: 
py-spy
might be good to try
you can also try setting https://www.pantsbuild.org/docs/reference-stats#memory_summary … it will only be rendered on a successful run, but it might highlight the issue
s
core/util_rules/source_files.py:51
is ~1% of the time on 2.14, ~10% of the time on 2.15.x 🤯
hmmmm
awaiting the 
MultiGet
of 
HydratedSources
thought I already fixed this 🤔 but apparently more to be done…
ah I think I see it - 2.14.x was using 
FrozenOrderedSet
everywhere, 2.15.x is using 
CoarsenedTargets
, probably still a ton of duplicates getting passed around
CoarsenedTargets.closure()
is yielding duplicate targets - sometimes 10s of thousands of duplicates for Color’s dependency graph - is that intended? in 2.14.x the pylint impl wrapped it in a 
FrozenOrderedSet
so it doesn’t look like new behavior, but it feels… wrong
w
Yikes. Yea, bug. 
closure
should only yield unique targets
If you want to assign an issue to me I can get a fix out today
👍 2
s
I put it in the 2.15.x milestone as a guess
w
Thanks: have a doctor's appointment, and then will be on it.
wow. this one is embarassing. will have the fix out shortly.
s
TIL you can use emojis as commit messages 😂
😓 1
w
i nearly put it in the PR description, but this will squash away my shame
2 Views
Open in Slack
PreviousNext
Powered by