Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for patch release tags. #1246

Merged
merged 22 commits into from
Jul 22, 2024
Merged

Support for patch release tags. #1246

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
698469e
Support for patch release tags.
joshuabaird Jul 17, 2024
9d666ba
Tail.
joshuabaird Jul 17, 2024
5fc1bd9
Remove VERSION_WITHOUT_V.
joshuabaird Jul 17, 2024
c4095d3
Add env.
joshuabaird Jul 17, 2024
579126f
Add env.
joshuabaird Jul 17, 2024
b399cbb
Add env.
joshuabaird Jul 17, 2024
f289481
Fix the tail.
joshuabaird Jul 17, 2024
91c7e0a
Add logs.
joshuabaird Jul 17, 2024
1d251aa
Fix last patch.
joshuabaird Jul 18, 2024
99b7ff3
Fix last patch.
joshuabaird Jul 18, 2024
4bc9b54
Fix last patch.
joshuabaird Jul 18, 2024
0e2edd3
Add fluentbit debug image.
joshuabaird Jul 18, 2024
49b6707
Add fluentbit debug image.
joshuabaird Jul 18, 2024
b039899
Typos.
joshuabaird Jul 18, 2024
300218f
Re-factor fluentd build.
joshuabaird Jul 18, 2024
df4de2e
Re-factor fluentd build.
joshuabaird Jul 18, 2024
c9eb9eb
Re-factor fluentd build.
joshuabaird Jul 18, 2024
3a41d2f
Re-factor fluentd build.
joshuabaird Jul 18, 2024
fd741c4
Re-factor fluentd build.
joshuabaird Jul 18, 2024
054553c
Re-factor fluentd build.
joshuabaird Jul 18, 2024
ab48d8e
Add README.
joshuabaird Jul 18, 2024
bfe3aea
Build latest.
joshuabaird Jul 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 50 additions & 8 deletions .github/workflows/build-fb-image.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,27 @@ jobs:
VERSION_WITHOUT_V=${VERSION#v}
MAJOR_MINOR=$(echo $VERSION_WITHOUT_V | cut -d. -f1-2)

echo "VERSION=$VERSION" >> $GITHUB_ENV
echo "VERSION_WITHOUT_V=$VERSION_WITHOUT_V" >> $GITHUB_ENV
echo "MAJOR_MINOR=$MAJOR_MINOR" >> $GITHUB_ENV

- name: Determine image version tag
id: determine-tags
run: |
if skopeo inspect docker://ghcr.io/${{ env.GITHUB_IMAGE }}:${{ env.VERSION }}; then
echo "${{ env.VERSION }} tag already exists, assuming we're building a patch release!"
LATEST_PATCH_VERSION=$(skopeo list-tags docker://ghcr.io/${{ env.GITHUB_IMAGE }} | grep -E "${{ env.VERSION }}-[0-9]+" | sort | uniq | tail -1 | tr -d \" | cut -d'-' -f2)
NEW_PATCH_VERSION=$((LATEST_PATCH_VERSION + 1))
IMAGE_TAG="${{ env.VERSION }}-${NEW_PATCH_VERSION}"
echo "Building patch release ${IMAGE_TAG}!"
else
echo "${{ env.VERSION }} tag does not exist, assuming we're building a new release!"
IMAGE_TAG="${{ env.VERSION }}"
fi

echo "IMAGE_TAG=$IMAGE_TAG" >> $GITHUB_ENV
env:
VERSION: ${{env.VERSION }}

- name: docker metadata for building
id: image-metadata
Expand All @@ -49,11 +68,12 @@ jobs:
images: "ghcr.io/${{ env.GITHUB_IMAGE }}"
tags: |
raw,latest
type=raw,value=${{ github.event.inputs.docker_tag_version }}
type=raw,value=v${{ github.event.inputs.docker_tag_version }}
type=raw,value=${{ env.IMAGE_TAG }}
type=raw,value=v${{ env.IMAGE_TAG }}
type=raw,value=${{ env.MAJOR_MINOR }}
type=raw,value=v${{ env.MAJOR_MINOR }}
env:
IMAGE_TAG: ${{ env.IMAGE_TAG }}
MAJOR_MINOR: ${{ env.MAJOR_MINOR }}

- name: docker tags for cloning
Expand All @@ -62,11 +82,12 @@ jobs:
with:
tags: |
raw,latest
type=raw,value=${{ github.event.inputs.docker_tag_version }}
type=raw,value=v${{ github.event.inputs.docker_tag_version }}
type=raw,value=${{ env.IMAGE_TAG }}
type=raw,value=v${{ env.IMAGE_TAG }}
type=raw,value=${{ env.MAJOR_MINOR }}
type=raw,value=v${{ env.MAJOR_MINOR }}
env:
IMAGE_TAG: ${{ env.IMAGE_TAG }}
MAJOR_MINOR: ${{ env.MAJOR_MINOR }}

- name: Set outputs
Expand Down Expand Up @@ -96,9 +117,28 @@ jobs:
VERSION_WITHOUT_V=${VERSION#v}
MAJOR_MINOR=$(echo $VERSION_WITHOUT_V | cut -d. -f1-2)

echo "VERSION=$VERSION" >> $GITHUB_ENV
echo "VERSION_WITHOUT_V=$VERSION_WITHOUT_V" >> $GITHUB_ENV
echo "MAJOR_MINOR=$MAJOR_MINOR" >> $GITHUB_ENV

- name: Determine image version tag
id: determine-tags
run: |
if skopeo inspect docker://ghcr.io/${{ env.GITHUB_IMAGE }}:${{ env.VERSION }}-debug; then
echo "${{ env.VERSION }}-debug tag already exists, assuming we're building a patch release!"
LATEST_PATCH_VERSION=$(skopeo list-tags docker://ghcr.io/${{ env.GITHUB_IMAGE }} | grep -E "${{ env.VERSION }}-[0-9]+" | sort | uniq | tail -1 | tr -d \" | cut -d'-' -f2)
NEW_PATCH_VERSION=$((LATEST_PATCH_VERSION + 1))
IMAGE_TAG="${{ env.VERSION }}-${NEW_PATCH_VERSION}"
echo "Building patch release ${IMAGE_TAG}-debug!"
else
echo "${{ env.VERSION }}-debug tag does not exist, assuming we're building a new release!"
IMAGE_TAG="${{ env.VERSION }}"
fi

echo "IMAGE_TAG=$IMAGE_TAG" >> $GITHUB_ENV
env:
VERSION: ${{env.VERSION }}

- name: docker metadata
id: image-metadata
uses: docker/metadata-action@v5
Expand All @@ -108,11 +148,12 @@ jobs:
latest=false
suffix=-debug
tags: |
type=raw,value=${{ github.event.inputs.docker_tag_version }}
type=raw,value=v${{ github.event.inputs.docker_tag_version }}
type=raw,value=${{ env.IMAGE_TAG }}
type=raw,value=v${{ env.IMAGE_TAG }}
type=raw,value=${{ env.MAJOR_MINOR }}
type=raw,value=v${{ env.MAJOR_MINOR }}
env:
IMAGE_TAG: ${{ env.IMAGE_TAG }}
MAJOR_MINOR: ${{ env.MAJOR_MINOR }}

- name: docker tags for cloning
Expand All @@ -123,11 +164,12 @@ jobs:
latest=false
suffix=-debug
tags: |
type=raw,value=${{ github.event.inputs.docker_tag_version }}
type=raw,value=v${{ github.event.inputs.docker_tag_version }}
type=raw,value=${{ env.IMAGE_TAG }}
type=raw,value=v${{ env.IMAGE_TAG }}
type=raw,value=${{ env.MAJOR_MINOR }}
type=raw,value=v${{ env.MAJOR_MINOR }}
env:
IMAGE_TAG: ${{ env.IMAGE_TAG }}
MAJOR_MINOR: ${{ env.MAJOR_MINOR }}

- name: Set outputs
Expand Down
139 changes: 109 additions & 30 deletions .github/workflows/build-fd-image.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,28 +35,52 @@ jobs:
id: extract_version
run: |
VERSION=${{ github.event.inputs.docker_tag_version }}
VERSION_SUFFIX="amd64"
VERSION_WITHOUT_V=${VERSION#v}
MAJOR_MINOR=$(echo $VERSION_WITHOUT_V | cut -d. -f1-2)

echo "VERSION=$VERSION" >> $GITHUB_ENV
echo "VERSION_SUFFIX=$VERSION_SUFFIX" >> $GITHUB_ENV
echo "VERSION_WITHOUT_V=$VERSION_WITHOUT_V" >> $GITHUB_ENV
echo "MAJOR_MINOR=$MAJOR_MINOR" >> $GITHUB_ENV


- name: Determine image version tag
id: determine-tags
run: |
if skopeo inspect docker://ghcr.io/${{ env.GITHUB_IMAGE }}:${{ env.VERSION }}-${{ env.VERSION_SUFFIX }}; then
echo "${{ env.VERSION }}-${{ env.VERSION_SUFFIX }} tag already exists, assuming we're building a patch release!"
LATEST_PATCH_VERSION=$(skopeo list-tags docker://ghcr.io/${{ env.GITHUB_IMAGE }} | grep -E "${{ env.VERSION }}-[0-9]+" | sort | uniq | tail -1 | tr -d \" | cut -d'-' -f2)
NEW_PATCH_VERSION=$((LATEST_PATCH_VERSION + 1))
IMAGE_TAG="${{ env.VERSION }}-${NEW_PATCH_VERSION}"
echo "Building patch release ${IMAGE_TAG}-${{ env.VERSION_SUFFIX }}!"
else
echo "${{ env.VERSION }}-${{ env.VERSION_SUFFIX }} tag does not exist, assuming we're building a new release!"
IMAGE_TAG="${{ env.VERSION }}"
fi

echo "IMAGE_TAG=$IMAGE_TAG" >> $GITHUB_ENV
env:
VERSION: ${{env.VERSION }}
VERSION_SUFFIX: ${{ env.VERSION_SUFFIX }}

- name: docker metadata for amd64
id: image-metadata
uses: docker/metadata-action@v5
with:
images: "ghcr.io/${{ env.GITHUB_IMAGE }}"
flavor: |
latest=false
suffix=-amd64
suffix=-${{ env.VERSION_SUFFIX }}
tags: |
raw,latest
type=raw,value=${{ github.event.inputs.docker_tag_version }}
type=raw,value=v${{ github.event.inputs.docker_tag_version }}
type=raw,value=${{ env.IMAGE_TAG }}
type=raw,value=v${{ env.IMAGE_TAG }}
type=raw,value=${{ env.MAJOR_MINOR }}
type=raw,value=v${{ env.MAJOR_MINOR }}
env:
IMAGE_TAG: ${{ env.IMAGE_TAG }}
MAJOR_MINOR: ${{ env.MAJOR_MINOR }}
VERSION_SUFFIX: ${{ env.VERSION_SUFFIX }}

- name: Set outputs
id: set-outputs
Expand All @@ -79,28 +103,52 @@ jobs:
id: extract_version
run: |
VERSION=${{ github.event.inputs.docker_tag_version }}
VERSION_SUFFIX="arm64"
VERSION_WITHOUT_V=${VERSION#v}
MAJOR_MINOR=$(echo $VERSION_WITHOUT_V | cut -d. -f1-2)

echo "VERSION=$VERSION" >> $GITHUB_ENV
echo "VERSION_SUFFIX=$VERSION_SUFFIX" >> $GITHUB_ENV
echo "VERSION_WITHOUT_V=$VERSION_WITHOUT_V" >> $GITHUB_ENV
echo "MAJOR_MINOR=$MAJOR_MINOR" >> $GITHUB_ENV


- name: Determine image version tag
id: determine-tags
run: |
if skopeo inspect --override-arch arm64 docker://ghcr.io/${{ env.GITHUB_IMAGE }}:${{ env.VERSION }}-${{ env.VERSION_SUFFIX }}; then
echo "${{ env.VERSION }}-${{ env.VERSION_SUFFIX }} tag already exists, assuming we're building a patch release!"
LATEST_PATCH_VERSION=$(skopeo list-tags docker://ghcr.io/${{ env.GITHUB_IMAGE }} | grep -E "${{ env.VERSION }}-[0-9]+" | sort | uniq | tail -1 | tr -d \" | cut -d'-' -f2)
NEW_PATCH_VERSION=$((LATEST_PATCH_VERSION + 1))
IMAGE_TAG="${{ env.VERSION }}-${NEW_PATCH_VERSION}"
echo "Building patch release ${IMAGE_TAG}-${{ env.VERSION_SUFFIX }}!"
else
echo "${{ env.VERSION }}-${{ env.VERSION_SUFFIX }} tag does not exist, assuming we're building a new release!"
IMAGE_TAG="${{ env.VERSION }}"
fi

echo "IMAGE_TAG=$IMAGE_TAG" >> $GITHUB_ENV
env:
VERSION: ${{ env.VERSION }}
VERSION_SUFFIX: ${{ env.VERSION_SUFFIX }}

- name: docker metadata for arm64
id: image-metadata
uses: docker/metadata-action@v5
with:
images: "ghcr.io/${{ env.GITHUB_IMAGE }}"
flavor: |
latest=false
suffix=-arm64
suffix=-${{ env.VERSION_SUFFIX }}
tags: |
raw,latest
type=raw,value=${{ github.event.inputs.docker_tag_version }}
type=raw,value=v${{ github.event.inputs.docker_tag_version }}
type=raw,value=${{ env.IMAGE_TAG }}
type=raw,value=v${{ env.IMAGE_TAG }}
type=raw,value=${{ env.MAJOR_MINOR }}
type=raw,value=v${{ env.MAJOR_MINOR }}
env:
IMAGE_TAG: ${{ env.IMAGE_TAG }}
MAJOR_MINOR: ${{ env.MAJOR_MINOR }}
VERSION_SUFFIX: ${{ env.VERSION_SUFFIX }}

- name: Set outputs
id: set-outputs
Expand All @@ -125,41 +173,51 @@ jobs:
id: extract_version
run: |
VERSION=${{ github.event.inputs.docker_tag_version }}
VERSION_SUFFIX="arm64-base"
VERSION_WITHOUT_V=${VERSION#v}
MAJOR_MINOR=$(echo $VERSION_WITHOUT_V | cut -d. -f1-2)

echo "VERSION=$VERSION" >> $GITHUB_ENV
echo "VERSION_SUFFIX=$VERSION_SUFFIX" >> $GITHUB_ENV
echo "VERSION_WITHOUT_V=$VERSION_WITHOUT_V" >> $GITHUB_ENV
echo "MAJOR_MINOR=$MAJOR_MINOR" >> $GITHUB_ENV


- name: Determine image version tag
id: determine-tags
run: |
if skopeo inspect --override-arch arm64 docker://ghcr.io/${{ env.GITHUB_IMAGE }}:${{ env.VERSION }}-${{ env.VERSION_SUFFIX }}; then
echo "${{ env.VERSION }}-${{ env.VERSION_SUFFIX }} tag already exists, assuming we're building a patch release!"
LATEST_PATCH_VERSION=$(skopeo list-tags docker://ghcr.io/${{ env.GITHUB_IMAGE }} | grep -E "${{ env.VERSION }}-[0-9]+" | sort | uniq | tail -1 | tr -d \" | cut -d'-' -f2)
NEW_PATCH_VERSION=$((LATEST_PATCH_VERSION + 1))
IMAGE_TAG="${{ env.VERSION }}-${NEW_PATCH_VERSION}"
echo "Building patch release ${IMAGE_TAG}-${{ env.VERSION_SUFFIX }}!"
else
echo "${{ env.VERSION }}-${{ env.VERSION_SUFFIX }} tag does not exist, assuming we're building a new release!"
IMAGE_TAG="${{ env.VERSION }}"
fi

echo "IMAGE_TAG=$IMAGE_TAG" >> $GITHUB_ENV
env:
VERSION: ${{ env.VERSION }}
VERSION_SUFFIX: ${{ env.VERSION_SUFFIX }}

- name: docker metadata for arm64 base image
id: image-metadata
uses: docker/metadata-action@v5
with:
images: "ghcr.io/${{ env.GITHUB_IMAGE }}"
flavor: |
latest=false
suffix=-arm64-base
suffix=-${{ env.VERSION_SUFFIX }}
tags: |
type=raw,value=${{ github.event.inputs.docker_tag_version }}
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was a duplicate step that is not necessary.

type=raw,value=v${{ github.event.inputs.docker_tag_version }}
type=raw,value=${{ env.MAJOR_MINOR }}
type=raw,value=v${{ env.MAJOR_MINOR }}
env:
MAJOR_MINOR: ${{ env.MAJOR_MINOR }}
- name: docker metadata for arm64 base image
id: image-tags
uses: docker/metadata-action@v5
with:
flavor: |
latest=false
suffix=-arm64-base
tags: |
type=raw,value=${{ github.event.inputs.docker_tag_version }}
type=raw,value=v${{ github.event.inputs.docker_tag_version }}
type=raw,value=${{ env.IMAGE_TAG }}
type=raw,value=v${{ env.IMAGE_TAG }}
type=raw,value=${{ env.MAJOR_MINOR }}
type=raw,value=v${{ env.MAJOR_MINOR }}
env:
IMAGE_TAG: ${{ env.IMAGE_TAG }}
MAJOR_MINOR: ${{ env.MAJOR_MINOR }}
VERSION_SUFFIX: ${{ env.VERSION_SUFFIX }}

- name: Set outputs
id: set-outputs
Expand Down Expand Up @@ -349,32 +407,53 @@ jobs:
VERSION_WITHOUT_V=${VERSION#v}
MAJOR_MINOR=$(echo $VERSION_WITHOUT_V | cut -d. -f1-2)

echo "VERSION=$VERSION" >> $GITHUB_ENV
echo "VERSION_WITHOUT_V=$VERSION_WITHOUT_V" >> $GITHUB_ENV
echo "MAJOR_MINOR=$MAJOR_MINOR" >> $GITHUB_ENV

- name: Determine image version tag
id: determine-tags
run: |
if skopeo inspect docker://ghcr.io/${{ env.GITHUB_IMAGE }}:${{ env.VERSION }}; then
echo "${{ env.VERSION }} tag already exists, assuming we're building a patch release!"
LATEST_PATCH_VERSION=$(skopeo list-tags docker://ghcr.io/${{ env.GITHUB_IMAGE }} | grep -E "${{ env.VERSION }}-[0-9]+" | sort | uniq | tail -1 | tr -d \" | cut -d'-' -f2)
NEW_PATCH_VERSION=$((LATEST_PATCH_VERSION + 1))
IMAGE_TAG="${{ env.VERSION }}-${NEW_PATCH_VERSION}"
echo "Building patch release ${IMAGE_TAG}!"
else
echo "${{ env.VERSION }} tag does not exist, assuming we're building a new release!"
IMAGE_TAG="${{ env.VERSION }}"
fi

echo "IMAGE_TAG=$IMAGE_TAG" >> $GITHUB_ENV
env:
VERSION: ${{ env.VERSION }}

- name: docker metadata for manifest
id: image-metadata
uses: docker/metadata-action@v5
with:
images: "ghcr.io/${{ env.GITHUB_IMAGE }}"
tags: |
type=raw,value=${{ github.event.inputs.docker_tag_version }}
type=raw,value=v${{ github.event.inputs.docker_tag_version }}
type=raw,value=${{ env.IMAGE_TAG }}
type=raw,value=v${{ env.IMAGE_TAG }}
type=raw,value=${{ env.MAJOR_MINOR }}
type=raw,value=v${{ env.MAJOR_MINOR }}
env:
IMAGE_TAG: ${{ env.IMAGE_TAG }}
MAJOR_MINOR: ${{ env.MAJOR_MINOR }}

- name: docker tags for cloning
id: image-tags
uses: docker/metadata-action@v5
with:
tags: |
type=raw,value=${{ github.event.inputs.docker_tag_version }}
type=raw,value=v${{ github.event.inputs.docker_tag_version }}
type=raw,value=${{ env.IMAGE_TAG }}
type=raw,value=v${{ env.IMAGE_TAG }}
type=raw,value=${{ env.MAJOR_MINOR }}
type=raw,value=v${{ env.MAJOR_MINOR }}
env:
IMAGE_TAG: ${{ env.IMAGE_TAG }}
MAJOR_MINOR: ${{ env.MAJOR_MINOR }}

- name: Set outputs
Expand Down
19 changes: 19 additions & 0 deletions cmd/fluent-watcher/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# fluent-watcher

This directory contains `Dockerfiles` for the custom build of both `fluentd` and `fluent-bit` that `fluent-operator` uses.

These custom builds include a wrapper that watches for configuration changes and reloads/restarts the `fluentd` and `fluent-bit` processes. These images are required for running the fluent-operator.

## Versioning

These images track upstream versions/tags. For example, Fluent Operator's `ghcr.io/fluent/fluent-operator/fluent-bit:3.1.12` image is based off of the upstream `fluentd/fluent-bit:3.1.12` image.

Occasionally, changes to Fluent Operator's customizations need to be introduced into the images out of band of upstream version changes. When this happens, we add a "patch version" to the tag. For example, the `ghcr.io/fluent/fluent-operator/fluent-bit:3.1.12-1` image remains based off of the upstream `fluent/fluent-bit:3.1.2` image but contains a Fluent Operator-specific patch (`-1`).

We strive to never overwrite existing image tags (eg, `ghcr.io/fluent/fluent-operator/fluent-bit:3.1.12`) with customizations and/or patches.

## Building

As a maintainer, to build the `ghcr.io/fluent/fluent-operator/fluent-bit` and `ghcr.io/fluent/fluent-operator/fluentd` images, you can run the "Build Fluent Bit image" or "Build Fluentd image" Github Action workflows in the "Actions" tab of this repository.

Always specify the upstream Fluent Bit version (eg, `3.1.2`, `3.1.3`, etc) when running this workflow. If the CI workflow detects that an image tag already exists for the version specified, it will assume that a patch release needs to be built and will automatically add a patch version to the image tag (eg, `3.1.2-1`, `3.1.2-2`, etc).
Loading