Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add command to compare multiple benchmarks #9

Merged
merged 46 commits into from
Jan 9, 2025
Merged

Conversation

nazarhussain
Copy link

Motivation

Add a new command to support comparing multiple benchmarks.

Description

This will helps us to compare benchmark results across different runtimes.

Steps to test or reproduce

Run all tests and workflows.

Copy link

github-actions bot commented Jan 6, 2025

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: 7cac592 Previous: null Ratio
sum array with raw for loop 933.79 us/op
sum array with reduce 10.186 ms/op
sum array with reduce beforeEach 10.219 ms/op
sum array with reduce before beforeEach 10.224 ms/op
sum array with reduce high threshold 10.276 ms/op
sum array with reduce no threshold 10.175 ms/op

by benchmarkbot/action

Copy link

github-actions bot commented Jan 7, 2025

Benchmark Comparison Report

Benchmark suite nodejs-benchmark deno-benchmark bun-benchmark
Average Ns Average Ns Ratio Average Ns Ratio
sum array with raw for loop 937.14 us/op 935.92 us/op 1.00 592.07 us/op 0.63
sum array with reduce 9.2522 ms/op 9.2838 ms/op 1.00 3.6311 ms/op 0.39
sum array with reduce beforeEach 9.2872 ms/op 9.3488 ms/op 1.01 3.6738 ms/op 0.40
sum array with reduce before beforeEach 9.3053 ms/op 9.3442 ms/op 1.00 3.6886 ms/op 0.40
sum array with reduce high threshold 9.2776 ms/op 9.2781 ms/op 1.00 7.1059 ms/op 0.77
sum array with reduce no threshold 9.2947 ms/op 9.3507 ms/op 1.01 7.0874 ms/op 0.76

by benchmarkbot/compare

@ChainSafe ChainSafe deleted a comment from github-actions bot Jan 7, 2025
@nazarhussain nazarhussain self-assigned this Jan 7, 2025
@@ -0,0 +1,153 @@
name: Run and compare benchmarks on different JS runtimes
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Created this workflow as reusable workflow so we can utilize it in other repos.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pretty sweet. Do we need some docs on how to use it? Maybe a comment block at the head for consumers that goes through what it does and how to run it so they dont have to study the code to grok whats cooking

Copy link
Member

@matthewkeil matthewkeil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 🎸 Only the couple spelling errors but approving and can fix them at your leisure

src/cli/compare.ts Outdated Show resolved Hide resolved
src/cli/compare.ts Outdated Show resolved Hide resolved
@matthewkeil matthewkeil merged commit 80d3b92 into main Jan 9, 2025
13 checks passed
@nazarhussain nazarhussain deleted the nh/compare branch January 10, 2025 08:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants