cool-easter-32542
04/14/2025, 12:21 AMSummary of `local_store_read_blob_size` observation histogram:
min: 1
max: 4317183
mean: 25249.341
std dev: 158077.770
total observations: 863
sum: 21790181
p25: 368
p50: 1870
p75: 8959
p90: 44191
p95: 110527
p99: 337919
These are shown by the [stats] subsystem, when [stats].log = true, but require the hdrhistogram package to be installed. If [stats].log = true and that package isn't installed, Pants emits a warning:
`` Please run withThe warning can be silenced either withif you would like histogram summaries to be shown at the end of the run, or permanently add--plugins=hdrhistogram. This will cause Pants to install the[GLOBAL].plugins = ['hdrhistogram']dependency from PyPI. ``hdrhistogram
[GLOBAL].ignore_warnings (if the user doesn't want to install the extra package), or by ... following the instructions:
[GLOBAL]
...
plugins = ['hdrhistogram']
...
[stats]
log = true
Describe the solution you'd like
Include the hdrhistogram package unconditionally, and thus avoid needing to encourage users to install it.
This introduces (pinned) versions of two new dependencies: hdrhistogram itself and its transitive dependency pbr, which total about ~1 MB uncompressed. Given we now package Pants as a PEX with all deps included, this doesn't seem like a major problem.
There's always legimatinate concerns about introducing new dependencies / having "too many dependencies", but in this case hdrhistogram is already an optional dependency (although I think this relationship is not currently explicitly specified in metadata anywhere), we're not introducing new load.
NB. as a side benefit, doing this reduces scope for supply chain attacks on Pants installs, because we won't be encouraging users to do unpinned installs of packages from PyPI (which is what [GLOBAL].plugins = ['hdrhistogram'] means). See also #20166.
Describe alternatives you've considered
• Reimplementing the histogram tracking support to avoid hdrhistogram (e.g. as new code in the Pants repo, or shifting all logic to Rust code): this may be a sensible longer-term improvement, but doesn't seem like it should block the short-term improvement.
Additional context
Discussed in https://pantsbuild.slack.com/archives/C046T6T9U/p1744381091598199?thread_ts=1744347143.246149&cid=C046T6T9U
pantsbuild/pantscool-easter-32542
04/14/2025, 8:49 PMcool-easter-32542
04/14/2025, 8:49 PM