https://pantsbuild.org/ logo
#development
Title
# development
c

careful-address-89803

10/26/2022, 3:32 PM
hiyas, do we have a standard way of benchmarking Pants? I'd like to make sure that my changes haven't caused a slowdown.
b

bitter-ability-32190

10/26/2022, 3:35 PM
c

careful-address-89803

10/26/2022, 3:36 PM
yeah, I've messed around with hyperfine a bit. Do we have a standard set of commands I should run? Or a repository other than Pants itself?
b

bitter-ability-32190

10/26/2022, 3:37 PM
Repo other than pants is probably good if you have one, because of the dogfood can cause funniness sometimes. No standard set of commands, use your imagination for what makes sense based ont he changes.
If you wanna be more specific on your changes and your thinking, we can also auggest commands ๐Ÿ™‚
c

careful-address-89803

10/26/2022, 3:39 PM
All my repos are smaller and less diverse than pants. The MR in question is https://github.com/pantsbuild/pants/pull/17057
dependencies --transitive ::
coke 1
w

wide-midnight-78598

10/26/2022, 3:53 PM
A standard benchmark (if one doesn't currently exist) would be dope. Even if not a
pants
goal/command, but a script/repo/whatever The only thing I can think of as a reasonable equivalent would be running on the pants repo - BUT that changes daily, so ... ๐Ÿคท
However, pre-PR, post-PR would be a fair test on that repo
c

careful-address-89803

10/26/2022, 4:11 PM
running pre/post MR on the Pants repo itself is good enough. It does pose the problem that the MR modifies the tested repository. So adding more tests would increase the time taken to run the "test" goal regardless of any changes to the code for that goal. I think it would be neat to identify repositories that use Pants and test the pre/post MR Pants on that.
I think we should also identify a set of commands to benchmark so we can have confidence that we've tested most of the important parts
w

witty-crayon-22786

10/26/2022, 5:30 PM
However, pre-PR, post-PR would be a fair test on that repo
yea, this is the best we have right now, unfortunately. itโ€™s decent though.
the standard commands depend on what you are benchmarking. for your python debug goals change, just benchmarking
./pants dependencies ::
is likely sufficient.