hiyas, do we have a standard way of benchmarking P...
# developmentc
careful-address-89803
10/26/2022, 3:32 PMhiyas, do we have a standard way of benchmarking Pants? I'd like to make sure that my changes haven't caused a slowdown.
b
bitter-ability-32190
10/26/2022, 3:35 PMc
careful-address-89803
10/26/2022, 3:36 PMyeah, I've messed around with hyperfine a bit. Do we have a standard set of commands I should run? Or a repository other than Pants itself?
b
bitter-ability-32190
10/26/2022, 3:37 PMRepo other than pants is probably good if you have one, because of the dogfood can cause funniness sometimes.
No standard set of commands, use your imagination for what makes sense based ont he changes.
bitter-ability-32190
10/26/2022, 3:37 PMIf you wanna be more specific on your changes and your thinking, we can also auggest commands 🙂
bitter-ability-32190
10/26/2022, 3:38 PM😃 1
c
careful-address-89803
10/26/2022, 3:39 PMAll my repos are smaller and less diverse than pants. The MR in question is https://github.com/pantsbuild/pants/pull/17057
dependencies --transitive ::coke 1
w
wide-midnight-78598
10/26/2022, 3:53 PMA standard benchmark (if one doesn't currently exist) would be dope. Even if not a
pantswide-midnight-78598
10/26/2022, 3:54 PMHowever, pre-PR, post-PR would be a fair test on that repo
c
careful-address-89803
10/26/2022, 4:11 PMrunning pre/post MR on the Pants repo itself is good enough. It does pose the problem that the MR modifies the tested repository. So adding more tests would increase the time taken to run the "test" goal regardless of any changes to the code for that goal. I think it would be neat to identify repositories that use Pants and test the pre/post MR Pants on that.
careful-address-89803
10/26/2022, 4:13 PMI think we should also identify a set of commands to benchmark so we can have confidence that we've tested most of the important parts
w
witty-crayon-22786
10/26/2022, 5:30 PMHowever, pre-PR, post-PR would be a fair test on that repoyea, this is the best we have right now, unfortunately. it’s decent though.
witty-crayon-22786
10/26/2022, 5:31 PMthe standard commands depend on what you are benchmarking. for your python debug goals change, just benchmarking
./pants dependencies ::