Skip to content

Commit

Permalink
build: Parallelize 'run-tests'
Browse files Browse the repository at this point in the history
Because nextest does not support running doctests, we are parallelizing
the run-tests.sh run in buildkite to run nextest in one job and doctests
in another.  This means spending a little more on CPU, but shortening
the total wall clock duration of a build (plus gaining the features
provided by nextest).

Change-Id: I8211a10ba41179d2b0ed1d888588863011df5afc
  • Loading branch information
ronh-rs committed Jan 3, 2025
1 parent 8b688cf commit e22980a
Show file tree
Hide file tree
Showing 2 changed files with 22 additions and 4 deletions.
1 change: 1 addition & 0 deletions .buildkite/pipeline.public-common.yml
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,7 @@ steps:

- label: ':rust: Run tests'
key: rust-tests
parallelism: 2
commands:
- '[ -d public ] && cd public'
- '.buildkite/run-tests.sh'
Expand Down
25 changes: 21 additions & 4 deletions .buildkite/run-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,24 @@ upload_artifacts() {
echo "+++ :rust: Run tests"
export DISABLE_TELEMETRY=true
export PROPTEST_MAX_SHRINK_TIME=1800000
cargo --locked nextest run --workspace --features failure_injection --exclude readyset-clustertest \
--exclude benchmarks --ignore-default-filter \
&& cargo --locked test --workspace --features failure_injection --exclude readyset-clustertest \
--exclude benchmarks --doc || upload_artifacts

# If we aren't actually running run-tests.sh as a parallel job, just follow the
# "job 0" path.
: "${BUILDKITE_PARALLEL_JOB:=0}"

if [[ "$BUILDKITE_PARALLEL_JOB" == "0" ]]; then
# Run nextest
cargo --locked nextest run --workspace --features failure_injection \
--exclude readyset-clustertest \
--exclude benchmarks --ignore-default-filter \
|| upload_artifacts
elif [[ "$BUILDKITE_PARALLEL_JOB" == "1" ]]; then
# Run doctests, because at this time nextest does not support doctests
cargo --locked test --workspace --features failure_injection \
--exclude readyset-clustertest \
--exclude benchmarks --doc \
|| upload_artifacts
else
echo "No command defined for this parallel job."
exit 1
fi

0 comments on commit e22980a

Please sign in to comment.