Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): Improve docker releases #952

Merged
merged 7 commits into from
Jan 26, 2025
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 0 additions & 40 deletions .github/workflows/cross_docker.yaml

This file was deleted.

157 changes: 157 additions & 0 deletions .github/workflows/docker.yaml
clabby marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
name: Build and Publish Docker Image

on:
workflow_dispatch:
inputs:
target:
type: choice
description: Which image to release
required: true
options:
- kona-host
- asterisc-builder
- cannon-builder
- kona-asterisc-prestate
push:
tags:
# matches tags like `service/v1.0.0`
- '*/v*'

env:
REGISTRY: ghcr.io
REGISTRY_IMAGE: ghcr.io/op-rs/kona
GIT_REF_NAME: ${{ github.ref_name }}

jobs:
prepare:
name: Prepare Bake
runs-on: ubuntu-latest
outputs:
matrix: ${{ steps.platforms.outputs.matrix }}
target: ${{ steps.target-spec.outputs.target }}
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Specify Target
id: target-spec
run: |
export TARGET="${{ inputs.target }}"
if [[ -z $TARGET ]]; then
export TARGET="${GIT_REF_NAME%/*}"
fi
echo "Target: $TARGET"
echo "target=$TARGET" >> $GITHUB_OUTPUT
- name: Create matrix
id: platforms
run: |
echo "matrix=$(docker buildx bake -f docker/docker-bake.hcl ${{ steps.target-spec.outputs.target }} --print | jq -cr '.target."${{ steps.target-spec.outputs.target }}".platforms')" >> ${GITHUB_OUTPUT}
- name: Show matrix
run: |
echo ${{ steps.platforms.outputs.matrix }}
- name: Docker meta
id: meta
uses: docker/metadata-action@v5
with:
images: ${{ env.REGISTRY_IMAGE }}/${{ steps.target-spec.outputs.target }}
tags: |
type=ref,event=branch
type=match,pattern=v(.*),group=1,event=tag
type=ref,event=pr
- name: Rename meta bake definition file
run: |
mv "${{ steps.meta.outputs.bake-file }}" "${{ runner.temp }}/bake-meta.json"
- name: Upload meta bake definition
uses: actions/upload-artifact@v4
with:
name: bake-meta
path: ${{ runner.temp }}/bake-meta.json
if-no-files-found: error
retention-days: 1

build:
name: Build Image (${{ needs.prepare.outputs.target }} - ${{ matrix.platform }})
runs-on: ${{ matrix.platform == 'linux/amd64' && 'ubuntu-latest' || 'ubuntu-22.04-arm' }}
needs:
- prepare
strategy:
fail-fast: false
matrix:
platform: ${{ fromJson(needs.prepare.outputs.matrix) }}
steps:
- name: Prepare
run: |
platform=${{ matrix.platform }}
echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
- name: Download meta bake definition
uses: actions/download-artifact@v4
with:
name: bake-meta
path: ${{ runner.temp }}
- name: Authenticate with container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Build
id: bake
uses: docker/bake-action@v6
with:
files: |
./docker/docker-bake.hcl
cwd://${{ runner.temp }}/bake-meta.json
targets: ${{ needs.prepare.outputs.target }}
set: |
*.tags=
*.platform=${{ matrix.platform }}
*.output=type=image,"name=${{ env.REGISTRY_IMAGE }}/${{ needs.prepare.outputs.target }}",push-by-digest=true,name-canonical=true,push=true
- name: Export digest
run: |
mkdir -p ${{ runner.temp }}/digests
digest="${{ fromJSON(steps.bake.outputs.metadata)[needs.prepare.outputs.target]['containerimage.digest'] }}"
touch "${{ runner.temp }}/digests/${digest#sha256:}"
- name: Upload digest
uses: actions/upload-artifact@v4
with:
name: digests-${{ env.PLATFORM_PAIR }}
path: ${{ runner.temp }}/digests/*
if-no-files-found: error
retention-days: 1

merge:
name: Publish Manifest (${{ needs.prepare.outputs.target }})
runs-on: ubuntu-latest
needs:
- build
- prepare
steps:
- name: Download meta bake definition
uses: actions/download-artifact@v4
with:
name: bake-meta
path: ${{ runner.temp }}
- name: Download digests
uses: actions/download-artifact@v4
with:
path: ${{ runner.temp }}/digests
pattern: digests-*
merge-multiple: true
- name: Authenticate with container registry
uses: docker/login-action@v3
with:
registry: ${{ env.REGISTRY }}
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Create manifest list and push
working-directory: ${{ runner.temp }}/digests
run: |
docker buildx imagetools create $(jq -cr '.target."docker-metadata-action".tags | map(select(startswith("${{ env.REGISTRY_IMAGE }}/${{ needs.prepare.outputs.target }}")) | "-t " + .) | join(" ")' ${{ runner.temp }}/bake-meta.json) \
$(printf '${{ env.REGISTRY_IMAGE }}/${{ needs.prepare.outputs.target }}@sha256:%s ' *)
- name: Inspect image
run: |
docker buildx imagetools inspect ${{ env.REGISTRY_IMAGE }}/${{ needs.prepare.outputs.target }}:$(jq -r '.target."docker-metadata-action".args.DOCKER_META_VERSION' ${{ runner.temp }}/bake-meta.json)

20 changes: 0 additions & 20 deletions .github/workflows/issue_cycler.yaml

This file was deleted.

54 changes: 0 additions & 54 deletions .github/workflows/kona_fpp_docker.yaml

This file was deleted.

5 changes: 0 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,6 @@ see the [SDK section of the book](https://op-rs.github.io/kona/sdk/intro.html).
- [`client`](./bin/client): The bare-metal program that runs on top of a [fault proof VM][g-fault-proof-vm].
- [`host`](./bin/host): The host program that runs natively alongside the FPVM, serving as the [Preimage Oracle][g-preimage-oracle] server.

**Build Pipelines**

- [`cannon`](./build/cannon): Docker image for compiling to the bare-metal `mips64-unknown-none` target.
- [`asterisc`](./build/asterisc): Docker image for compiling to the bare-metal `riscv64imac-unknown-none-elf` target.

**Protocol**
- [`mpt`](./crates/mpt): Utilities for interacting with the Merkle Patricia Trie in the client program.
- [`executor`](./crates/executor): `no_std` stateless block executor for the [OP Stack][op-stack].
Expand Down
33 changes: 0 additions & 33 deletions build/README.md

This file was deleted.

15 changes: 0 additions & 15 deletions build/justfile

This file was deleted.

58 changes: 58 additions & 0 deletions docker/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# `docker`

This directory contains all of the repositories' dockerfiles as well as the [bake file](https://docs.docker.com/build/bake/)
used to define this repository's docker build configuration.

## Install Dependencies

* `docker`: https://www.docker.com/get-started/
* `docker-buildx`: https://github.com/docker/buildx?tab=readme-ov-file#installing

## Building Locally

To build any image in the bake file locally, use `docker bake`:
clabby marked this conversation as resolved.
Show resolved Hide resolved

```sh
export TARGET="<target_name>"
clabby marked this conversation as resolved.
Show resolved Hide resolved

# Optional: adjust the tag for the image
# Defaults to `kona:local`
export DEFAULT_TAG="my-image:local"

# Optional: Override the platforms to build the image for.
# Defaults to `linux/amd64,linux/arm64`
export PLATFORMS="<platforms>"

# Optional: Override the git ref to use for the current repo. Must exist
# on the `op-rs/kona` remote.
#
# Used by:
# - `kona-host`
# - `kona-asterisc-prestate`
export GIT_REF_NAME="my/feature/branch"

docker buildx bake \
--progress plain \
-f docker/docker-bake.hcl \
$TARGET
```

## Cutting a Release (for maintainers / forks)

To cut a release of the docker image for any of the targets, cut a new annotated tag for the target like so:

```sh
# Example formats:
# - `kona-host/v0.1.0-beta.8`
# - `cannon-builder/v1.2.0`
TAG="<target_name>/<version>"
git tag -a $TAG -m "<tag description>" && git push origin tag $TAG
```

To run the workflow manually, navigate over to the ["Build and Publish Docker Image"](https://github.com/op-rs/kona/actions/workflows/docker.yaml)
action. From there, run a `workflow_dispatch` trigger, select the tag you just pushed, and then finally select the image to release.

Or, if you prefer to use the `gh` CLI, you can run:
```sh
gh workflow run "Build and Publish Docker Image" --ref <tag> -f image_to_release=<target>
```
Loading
Loading