-
Notifications
You must be signed in to change notification settings - Fork 106
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
ci: create reusable workflow for nix kernel builds
Adds a reusable workflow `build-kernel.yml` to cover the build-kernel job using the new Nix based build process on the dedicated runner. All of the `build-kernel` jobs are identical except for their git repo and git branch name. Factor these out into a reusable workflow to reduce code duplication. This also removes any suffixes from the cache, which might (unlikely) increase hit rates. Each build was suffixing their kernels separately even though the build was identical, which was slightly wasteful. It is unlikely any of these repos/branches have identical states though. Test plan: - Ran the CI and waited for build-kernel-nix to succeed in each workflow (added a temporary `push:` condition to make sure they all build). Cancelled the rest as this built kernel isn't used yet so there's no point causing extra queuing.
- Loading branch information
1 parent
9e7418b
commit 5b148f3
Showing
5 changed files
with
91 additions
and
57 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,69 @@ | ||
on: | ||
workflow_call: | ||
inputs: | ||
git-repo: | ||
required: true | ||
type: string | ||
branch: | ||
required: true | ||
type: string | ||
|
||
jobs: | ||
build-kernel: | ||
runs-on: [ "self-hosted", "linux", "x64" ] | ||
steps: | ||
# Uncomment these if reverting to GitHub hosted runners | ||
# - name: Install Nix | ||
# uses: DeterminateSystems/nix-installer-action@main | ||
|
||
# Make very basic dependencies available in PATH | ||
- name: Install basic dependencies | ||
run: | | ||
nix profile install nixpkgs#{git,openssl,gawk,gnutar,zstd} | ||
echo "$HOME/.nix-profile/bin" >> $GITHUB_PATH | ||
# get latest head commit of sched_ext for-next | ||
- run: echo "SCHED_EXT_KERNEL_COMMIT=$(git ls-remote ${{ inputs.git-repo }} heads/${{ inputs.branch }} | awk '{print $1}')" >> $GITHUB_ENV | ||
|
||
# check for cached kernel without downloading | ||
- name: Cache Kernel | ||
id: cache-kernel | ||
uses: actions/cache@v4 | ||
with: | ||
path: | | ||
linux/arch/x86/boot/bzImage | ||
linux/usr/include | ||
linux/**/*.h | ||
key: kernel-build-${{ env.SCHED_EXT_KERNEL_COMMIT }}-7 | ||
lookup-only: true | ||
|
||
- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }} | ||
uses: actions/checkout@v4 | ||
|
||
- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }} | ||
uses: nicknovitski/nix-develop@v1 | ||
with: | ||
arguments: ./.github/workflows#build-kernel | ||
|
||
- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }} | ||
name: Clone Kernel | ||
uses: cytopia/[email protected] | ||
with: | ||
retries: 10 | ||
pause: 18 | ||
command: git clone --single-branch -b ${{ inputs.branch }} --depth 1 ${{ inputs.git-repo }} linux | ||
|
||
- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }} | ||
name: Select correct commit for cache hash | ||
run: | | ||
cd linux | ||
git switch --detach ${{ env.SCHED_EXT_KERNEL_COMMIT }} | ||
git log -1 --pretty=format:"%h %ad %s" --date=short | ||
- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }} | ||
# Build a minimal kernel (with sched-ext enabled) using virtme-ng | ||
run: cd linux && virtme-ng -v --build --config ../.github/workflows/sched-ext.config | ||
|
||
- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }} | ||
# Generate kernel headers | ||
run: cd linux && make headers |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -27,63 +27,10 @@ jobs: | |
- run: git diff --exit-code | ||
|
||
build-kernel-nix: | ||
runs-on: [ "self-hosted", "linux", "x64" ] | ||
steps: | ||
# Uncomment these if reverting to GitHub hosted runners | ||
# - name: Install Nix | ||
# uses: DeterminateSystems/nix-installer-action@main | ||
|
||
# Make very basic dependencies available in PATH | ||
- name: Install basic dependencies | ||
run: | | ||
nix profile install nixpkgs#{git,openssl,gawk,gnutar,zstd} | ||
echo "$HOME/.nix-profile/bin" >> $GITHUB_PATH | ||
# get latest head commit of sched_ext for-next | ||
- run: echo "SCHED_EXT_KERNEL_COMMIT=$(git ls-remote https://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext.git heads/for-6.13 | awk '{print $1}')" >> $GITHUB_ENV | ||
|
||
# check for cached kernel without downloading | ||
- name: Cache Kernel | ||
id: cache-kernel | ||
uses: actions/cache@v4 | ||
with: | ||
path: | | ||
linux/arch/x86/boot/bzImage | ||
linux/usr/include | ||
linux/**/*.h | ||
key: kernel-build-${{ env.SCHED_EXT_KERNEL_COMMIT }}-7 | ||
lookup-only: true | ||
|
||
- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }} | ||
uses: actions/checkout@v4 | ||
|
||
- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }} | ||
uses: nicknovitski/nix-develop@v1 | ||
with: | ||
arguments: ./.github/workflows#build-kernel | ||
|
||
- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }} | ||
name: Clone Kernel | ||
uses: cytopia/[email protected] | ||
with: | ||
retries: 10 | ||
pause: 18 | ||
command: git clone --single-branch -b for-6.13 --depth 1 https://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext.git linux | ||
|
||
- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }} | ||
name: Select correct commit for cache hash | ||
run: | | ||
cd linux | ||
git switch --detach ${{ env.SCHED_EXT_KERNEL_COMMIT }} | ||
git log -1 --pretty=format:"%h %ad %s" --date=short | ||
- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }} | ||
# Build a minimal kernel (with sched-ext enabled) using virtme-ng | ||
run: cd linux && virtme-ng -v --build --config ../.github/workflows/sched-ext.config | ||
|
||
- if: ${{ steps.cache-kernel.outputs.cache-hit != 'true' }} | ||
# Generate kernel headers | ||
run: cd linux && make headers | ||
uses: ./.github/workflows/build-kernel.yml | ||
with: | ||
git-repo: "https://git.kernel.org/pub/scm/linux/kernel/git/tj/sched_ext.git" | ||
branch: for-6.13 | ||
|
||
build-kernel: | ||
runs-on: ubuntu-24.04 | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters