Push Images to GHCR.io #1071

sarathchandra24 · 2024-02-26T12:25:17Z

What this PR does / why we need it:

As specified in #1068, we would like to push current images to multiple registries, this PR starts pushing images to GitHub Container Registry ghcr.io

Which issue(s) this PR fixes:

Fixes #1068

Does this PR introduced a user-facing change?

Additional documentation, usage docs, etc.:

Signed-off-by: Sarath Chandra Oruganti <[email protected]>

sarathchandra24 · 2024-02-26T12:32:22Z

.github/workflows/build-fb-image.yaml

      - "cmd/fluent-watcher/fluentbit/**"
      - "cmd/fluent-watcher/hooks/**"
      - "pkg/filenotify/**"

 env:
-  FB_IMG: 'kubesphere/fluent-bit:v2.2.2'
-  FB_IMG_DEBUG: 'kubesphere/fluent-bit:v2.2.2-debug'
+  FB_IMG: "kubesphere/fluent-bit:${{ github.ref_name }}"                # kubesphere/fluent-bit:v2.2.2


I would like to recommend using GitHub action variables here, something like vars.FB_IMG. While testing actions on contributors side, it makes hard as they don't have permissions to push to this repo and it takes 1 or 2 commits to rollback to desired changes.

This follows to other env's as well.

Be careful with ref_name though as it won't be what's expected for PRs.

None of this should be done by variables like this really, we should use the docker/metadata-action for this as it properly supports multiple tags and configuring them (e.g. for default branch, tags, etc.).

sarathchandra24 · 2024-02-26T12:34:42Z

.github/workflows/build-fb-image.yaml

+  FB_IMG: "kubesphere/fluent-bit:${{ github.ref_name }}"                # kubesphere/fluent-bit:v2.2.2
+  FB_IMG_DEBUG: "kubesphere/fluent-bit:${{ github.ref_name }}-debug"    # kubesphere/fluent-bit:v2.2.2-debug
+
+  FB_IMG_GHCR: "${{ github.repository }}/fluent-bit:${{ github.ref_name }}"              # fluent/fluent-operator/fluent-bit:v2.2.2


We have fluent-bit image built by other https://github.com/fluent/fluent-bit/ to differentiate between the images built by both, I am pushing the image to current repository only.

sarathchandra24 · 2024-02-26T12:35:48Z

.github/workflows/build-fd-image.yaml

-  ARCH: '-arm64'
-  FD_IMG_BASE: 'kubesphere/fluentd:v1.15.3-arm64-base'
+
+  FD_IMG: "kubesphere/fluentd:${{ github.ref_name }}"                                 # kubesphere/fluentd:v2.2.2


Taking branch name / git tag as the image tag as I couldn't figure out what to keep.

Let's switch to the metadata action for docker

patrick-stephens

This looks ok but a few things I think need changing and you're also missing auth for skopeo.

We should not be using env vars to name the images, we should be using the dedicated action for this as it is much better and extensible plus will not need changing every time.
https://github.com/docker/metadata-action
e.g. https://github.com/fluent/fluent-bit/blob/9d9ac68a2b45a4cedeafbf7c5aba513f494eced6/.github/workflows/call-build-images.yaml#L107

We should be pushing to ghcr.io initially as that simplifies authentication and is consistent across the Fluent repositories.
We can just login with GITHUB_TOKEN then rather than needing a secret and it will work for forks, etc.

permissions:
  contents: read
  packages: write

I would then split the release to other registries out to a separate workflow, only run for tagged releases.

patrick-stephens · 2024-02-29T10:46:04Z

.github/workflows/build-fb-image.yaml

      - "cmd/fluent-watcher/fluentbit/**"
      - "cmd/fluent-watcher/hooks/**"
      - "pkg/filenotify/**"

 env:
-  FB_IMG: 'kubesphere/fluent-bit:v2.2.2'
-  FB_IMG_DEBUG: 'kubesphere/fluent-bit:v2.2.2-debug'
+  FB_IMG: "kubesphere/fluent-bit:${{ github.ref_name }}"                # kubesphere/fluent-bit:v2.2.2


Be careful with ref_name though as it won't be what's expected for PRs.

None of this should be done by variables like this really, we should use the docker/metadata-action for this as it properly supports multiple tags and configuring them (e.g. for default branch, tags, etc.).

patrick-stephens · 2024-02-29T10:47:02Z

.github/workflows/build-fb-image.yaml


 jobs:
  build:
    runs-on: ubuntu-latest
    timeout-minutes: 30
    name: Build Image for Fluent Bit
+    outputs:


Outputs should come from steps really, e.g. the metadata action would give you a way to get this. That way if we modify something (or for PRs, non-main branch, etc.) it will carry on working.

patrick-stephens · 2024-02-29T10:47:41Z

.github/workflows/build-fb-image.yaml

+          file: ./cmd/fluent-watcher/fluentbit/Dockerfile
+          push: true
+          platforms: linux/amd64,linux/arm64


This will be pretty slow btw, it may not be worthwhile to do ARM builds for PRs.

We are currently building in our existing approach, so I have added.

https://github.com/fluent/fluent-operator/blob/master/Makefile#L99-L103

patrick-stephens · 2024-02-29T10:48:36Z

.github/workflows/build-fb-image.yaml


      - name: Login to Docker Hub
        uses: docker/login-action@v3
        with:
+          registry: docker.io


Are we pushing to docker.io now instead? I would keep things in ghcr.io if you can particularly for PRs otherwise we have to start dealing with rate limiting.

patrick-stephens · 2024-02-29T10:49:23Z

.github/workflows/build-fb-image.yaml

+      registry_password: ${{ secrets.REGISTRY_PASSWORD }}
+
+  release-image-to-gchr:


I think this is the wrong way round, all images should start in ghcr.io as it'll be public and not rate limited. We then push to the others only release images (not all the PR images or whatever too).

patrick-stephens · 2024-02-29T10:52:23Z

.github/workflows/clone-docker-image-action.yaml

+          - name: Promote container images from ${{ inputs.source_registry }} to ${{ inputs.target_registry }}
+            run: |
+              echo "Promoting $SOURCE_IMAGE to $RELEASE_IMAGE"
+              docker run --rm  \


Note we may have to drop signatures if a registry does not support it so it might be useful to make that an optional parameter?

patrick-stephens · 2024-02-29T10:52:46Z

.github/workflows/clone-docker-image-action.yaml

+        needs: check-image-exists
+        runs-on: ubuntu-latest
+        steps:
+          - name: Promote container images from ${{ inputs.source_registry }} to ${{ inputs.target_registry }}


You need to log into the registry first and then pass the auth.json to skopeo in the next step as a bind mount.

patrick-stephens · 2024-02-29T10:53:03Z

.github/workflows/scan-docker-image-action.yaml

+          - name: Checkout code
+            uses: actions/checkout@v4
+            with:
+              fetch-depth: 0


What's the reason for this?

patrick-stephens · 2024-02-29T10:53:13Z

.github/workflows/scan-docker-image-action.yaml

+            with:
+              fetch-depth: 0
+
+          - name: Login to Docker Hub


Use the input variable

patrick-stephens · 2024-02-29T10:53:41Z

.github/workflows/scan-docker-image-action.yaml

+            run: |
+              echo "Platform: ${{ matrix.platform }}"
+
+          - name: Dockle - multi-arch


We need to support an option to ignore any false positives here as well.

benjaminhuo · 2024-03-04T02:00:30Z

PR closed? @sarathchandra24

sarathchandra24 · 2024-03-04T05:43:16Z

I was making some changes and I got errors which to push images my personal repo; So I deleted the repo and forked it again. It closed this PR.

benjaminhuo · 2024-03-04T15:40:22Z

I was making some changes and I got errors which to push images my personal repo; So I deleted the repo and forked it again. It closed this PR.

Looks like this cannot be reopened, you'll submit another PR for this with all the comments resolved? @sarathchandra24

sarathchandra24 · 2024-03-04T23:48:56Z

created a new PR: #1079

sarathchandra24 added 2 commits February 26, 2024 07:28

Push Images to GHCR.io

c4734cc

Signed-off-by: Sarath Chandra Oruganti <[email protected]>

sign off

82d0aa0

Signed-off-by: Sarath Chandra Oruganti <[email protected]>

sarathchandra24 commented Feb 26, 2024

View reviewed changes

benjaminhuo requested review from wenchajun and wanjunlei February 27, 2024 01:21

sarathchandra24 marked this pull request as draft February 29, 2024 00:03

sarathchandra24 marked this pull request as ready for review February 29, 2024 00:04

sarathchandra24 mentioned this pull request Feb 29, 2024

kubesphere fluent* images should be published to multiple registries #1068

Closed

patrick-stephens requested changes Feb 29, 2024

View reviewed changes

Merge branch 'fluent:master' into images-publish-multi-branch

dfebb9e

sarathchandra24 closed this by deleting the head repository Mar 3, 2024

sarathchandra24 mentioned this pull request Mar 4, 2024

push image to multiple registry #1079

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Push Images to GHCR.io #1071

Push Images to GHCR.io #1071

sarathchandra24 commented Feb 26, 2024

sarathchandra24 Feb 26, 2024 •

edited

Loading

patrick-stephens Feb 29, 2024

sarathchandra24 Feb 26, 2024

sarathchandra24 Feb 26, 2024

patrick-stephens Feb 29, 2024

patrick-stephens left a comment

patrick-stephens Feb 29, 2024

patrick-stephens Feb 29, 2024

patrick-stephens Feb 29, 2024

sarathchandra24 Mar 3, 2024

patrick-stephens Feb 29, 2024

patrick-stephens Feb 29, 2024

patrick-stephens Feb 29, 2024

patrick-stephens Feb 29, 2024

patrick-stephens Feb 29, 2024

patrick-stephens Feb 29, 2024

patrick-stephens Feb 29, 2024

benjaminhuo commented Mar 4, 2024

sarathchandra24 commented Mar 4, 2024

benjaminhuo commented Mar 4, 2024 •

edited

Loading

sarathchandra24 commented Mar 4, 2024

		registry_password: ${{ secrets.REGISTRY_PASSWORD }}

		release-image-to-gchr:

Push Images to GHCR.io #1071

Push Images to GHCR.io #1071

Conversation

sarathchandra24 commented Feb 26, 2024

What this PR does / why we need it:

Which issue(s) this PR fixes:

Does this PR introduced a user-facing change?

Additional documentation, usage docs, etc.:

sarathchandra24 Feb 26, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

patrick-stephens left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

benjaminhuo commented Mar 4, 2024

sarathchandra24 commented Mar 4, 2024

benjaminhuo commented Mar 4, 2024 • edited Loading

sarathchandra24 commented Mar 4, 2024

sarathchandra24 Feb 26, 2024 •

edited

Loading

benjaminhuo commented Mar 4, 2024 •

edited

Loading