Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Benchmark NewReno and Cubic #1732

Closed
wants to merge 25 commits into from
Closed
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
12386a3
ci: Benchmark NewReno and Cubic
larseggert Mar 11, 2024
de64b3e
Wait for output before continuing
larseggert Mar 11, 2024
e7bf509
Merge branch 'main' into ci-bench-cc
larseggert Mar 11, 2024
bc7b99f
Fix
larseggert Mar 11, 2024
bd742af
mkdir -p
larseggert Mar 11, 2024
681bbb7
Merge branch 'main' into ci-bench-cc
larseggert Mar 11, 2024
db1dbb2
Merge branch 'main' into ci-bench-cc
larseggert Mar 11, 2024
9ea3a99
Address comments from @martinthomson
larseggert Mar 12, 2024
b32ce9e
Merge branch 'ci-bench-cc' of github.com:larseggert/neqo into ci-benc…
larseggert Mar 12, 2024
02620a7
Try to kill via `$!`
larseggert Mar 12, 2024
3241f93
Merge branch 'main' into ci-bench-cc
larseggert Mar 12, 2024
925cc12
Merge branch 'main' into ci-bench-cc
larseggert Mar 12, 2024
84e2206
Merge branch 'main' into ci-bench-cc
larseggert Mar 12, 2024
9d3a8b7
Use temp dir, and fix path error
larseggert Mar 12, 2024
407bd4f
kill -> killall
larseggert Mar 12, 2024
11320d0
No -INT
larseggert Mar 12, 2024
d37e706
Go back to `killall`
larseggert Mar 12, 2024
c82ff3a
`killall` -> `pkill`
larseggert Mar 12, 2024
4ea2c56
Merge branch 'main' into ci-bench-cc
larseggert Mar 12, 2024
d072504
Reorder things so `results.ms` is included in the exported artifact
larseggert Mar 12, 2024
b2bb855
Merge branch 'ci-bench-cc' of github.com:larseggert/neqo into ci-benc…
larseggert Mar 12, 2024
f0cd19e
Merge branch 'main' into ci-bench-cc
larseggert Mar 12, 2024
8e5290b
Merge branch 'main' into ci-bench-cc
larseggert Mar 12, 2024
f50f414
Merge branch 'main' into ci-bench-cc
larseggert Mar 13, 2024
9ca5ecc
Merge branch 'main' into ci-bench-cc
larseggert Mar 14, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 26 additions & 23 deletions .github/workflows/bench.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ env:
RUST_BACKTRACE: 1
TOOLCHAIN: nightly
RUSTFLAGS: -C link-arg=-fuse-ld=lld -C link-arg=-Wl,--no-rosegment, -C force-frame-pointers=yes
PERF_CMD: record -o perf.data -F997 --call-graph fp -g
PERF_OPT: record -F997 --call-graph fp -g

jobs:
bench:
Expand Down Expand Up @@ -67,30 +67,30 @@ jobs:
mv target/criterion target/criterion-bench
mv target/criterion-transfer-profile target/criterion || true
taskset -c 0 nice -n -20 \
cargo "+$TOOLCHAIN" flamegraph -v -c "$PERF_CMD" --features bench --bench transfer -- \
cargo "+$TOOLCHAIN" flamegraph -o transfer.svg -v -c "$PERF_OPT -o transfer.perf" \
--features bench --bench transfer -- \
--bench --exact "Run multiple transfers with varying seeds" --noplot
# And now restore the directories.
mv target/criterion target/criterion-transfer-profile
mv target/criterion-bench target/criterion

- name: Profile client/server transfer
run: |
{ mkdir server; \
cd server; \
for CC in newreno cubic; do
TMP=$(mktemp -d)
taskset -c 0 nice -n -20 \
cargo "+$TOOLCHAIN" flamegraph -v -c "$PERF_CMD" \
--bin neqo-server -- --db ../test-fixture/db "$HOST:4433" || true; } &
mkdir client; \
cd client; \
cargo "+$TOOLCHAIN" flamegraph -o "server-$CC.svg" -v -c "$PERF_OPT -o server-$CC.perf" \
--bin neqo-server -- --cc "$CC" "$HOST:4433" &
time taskset -c 1 nice -n -20 \
cargo "+$TOOLCHAIN" flamegraph -v -c "$PERF_CMD" \
--bin neqo-client -- --output-dir . "https://$HOST:4433/$SIZE"
killall -INT neqo-server
cd ${{ github.workspace }}
[ "$(wc -c < client/"$SIZE")" -eq "$SIZE" ] || exit 1
cargo "+$TOOLCHAIN" flamegraph -o "client-$CC.svg" -v -c "$PERF_OPT -o client-$CC.perf" \
--bin neqo-client -- --output-dir "$TMP" --cc "$CC" "https://$HOST:4433/$SIZE"
pkill -U bench -o -x -INT neqo-server
[ "$(wc -c < "$TMP/$SIZE")" -eq "$SIZE" ] || exit 1
rm -rf "$TMP"
done
env:
HOST: localhost
SIZE: 1073741824 # 1 GB
SIZE: 536870912 # 512 MB

# Re-enable turboboost, hyperthreading and use powersave governor.
- name: Restore machine
Expand All @@ -99,20 +99,23 @@ jobs:

- name: Convert for profiler.firefox.com
run: |
perf script -i perf.data -F +pid > transfer.perf &
perf script -i client/perf.data -F +pid > client.perf &
perf script -i server/perf.data -F +pid > server.perf &
perf script -i transfer.perf -F +pid > transfer.fx.perf &
for CC in newreno cubic; do
for PEER in client server; do
perf script -i "$PEER-$CC.perf" -F +pid > "$PEER-$CC.fx.perf" &
done
done
wait
mv flamegraph.svg transfer.svg
mv client/flamegraph.svg client.svg
mv server/flamegraph.svg server.svg
rm neqo.svg

- name: Generate perf reports
run: |
perf report -i perf.data --no-children --stdio > transfer.perf.txt &
perf report -i client/perf.data --no-children --stdio > client.perf.txt &
perf report -i server/perf.data --no-children --stdio > server.perf.txt &
perf report -i transfer.perf --no-children --stdio > transfer.perf.txt &
for CC in newreno cubic; do
for PEER in client server; do
perf report -i "$PEER-$CC.perf" --no-children --stdio > "$PEER-$CC.txt" &
done
done
wait

- name: Format results as Markdown
Expand Down
Loading