Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OSTree Plugin #28

Merged
merged 34 commits into from
Jan 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
4c078da
adds plugins.md as starting point for plugin how-to documentation
Nov 16, 2023
1a683b2
adds IsTLSMiddleware for convenience
Dec 7, 2023
f94b00e
ignore intellij project state files
Dec 7, 2023
60456e8
refactors service to a non-generic form
Dec 7, 2023
e3c4d74
initial implementation of ostree plugin (without mirroring)
Dec 7, 2023
a0e014e
gofumpt
Dec 8, 2023
0124423
gofumpt
Dec 15, 2023
f34ec0c
adds plugins.md as starting point for plugin how-to documentation
Nov 16, 2023
052611d
adds IsTLSMiddleware for convenience
Dec 7, 2023
46978c5
ignore intellij project state files
Dec 7, 2023
4375255
refactors service to a non-generic form
Dec 7, 2023
95adf48
initial implementation of ostree plugin (without mirroring)
Dec 7, 2023
29f5dbb
gofumpt
Dec 8, 2023
80d309d
gofumpt
Dec 15, 2023
d4db914
Merge branch 'main' into docs
Dec 15, 2023
d75a5b1
various changes from PR review
Dec 15, 2023
efa5db0
Merge remote-tracking branch 'origin/docs' into docs
Dec 15, 2023
7b00e6d
more fixes for linter
Dec 15, 2023
6fb8f97
Merge branch 'main' into docs
Jan 2, 2024
70da0f2
adds libostree testdata to gitignore
Jan 11, 2024
aac24f2
libostree finished
Jan 11, 2024
9478fb4
fixes various bugs discovered after manual testing
Jan 12, 2024
7876f55
gofumpt
Jan 12, 2024
4320e9c
removes unused dockerfile
Jan 12, 2024
3158651
fixes unit test build failure by introducing buildEnv and buildExecSt…
Jan 12, 2024
feae074
removes mistaken C import causing mage to no longer see test targets.
Jan 12, 2024
793ac41
fixes logic error in Depth() libostree option
Jan 12, 2024
0109dc8
Merge remote-tracking branch 'upstream/main' into docs
Jan 12, 2024
9d39093
rebase upstream main
Jan 12, 2024
fb8337f
removed comments again...
Jan 13, 2024
85c6ea5
fixes linter issues
Jan 15, 2024
7049ce3
adds libostree testdata
Jan 15, 2024
bca7378
adds libostree testdata
Jan 15, 2024
a95e957
adds --priviledged option to test:unit container
Jan 15, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 15 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -69,4 +69,18 @@ jobs:
- name: Release beskar-static image
run: ./scripts/mage ci:image ghcr.io/ctrliq/beskar-static:${{ github.ref_name }} "${{ github.actor }}" "${{ secrets.GITHUB_TOKEN }}"
- name: Release beskar-static helm chart
run: ./scripts/mage ci:chart ghcr.io/ctrliq/helm-charts/beskar-static:${{ github.ref_name }} "${{ github.actor }}" "${{ secrets.GITHUB_TOKEN }}"
run: ./scripts/mage ci:chart ghcr.io/ctrliq/helm-charts/beskar-static:${{ github.ref_name }} "${{ github.actor }}" "${{ secrets.GITHUB_TOKEN }}"

release-beskar-ostree:
name: release beskar-ostree
needs: [lint, tests]
runs-on: ubuntu-22.04
steps:
- uses: actions/setup-go@v3
with:
go-version: '1.21'
- uses: actions/checkout@v3
- name: Release beskar-ostree image
run: ./scripts/mage ci:image ghcr.io/ctrliq/beskar-ostree:${{ github.ref_name }} "${{ github.actor }}" "${{ secrets.GITHUB_TOKEN }}"
- name: Release beskar-ostree helm chart
run: ./scripts/mage ci:chart ghcr.io/ctrliq/helm-charts/beskar-ostree:${{ github.ref_name }} "${{ github.actor }}" "${{ secrets.GITHUB_TOKEN }}"
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,3 +3,7 @@ build/output
.envrc.local
vendor
go.work.sum


*/.idea
/.idea/
8 changes: 8 additions & 0 deletions .idea/.gitignore

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 9 additions & 0 deletions .idea/beskar.iml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions .idea/modules.xml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

9 changes: 8 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,10 @@ It's designed to support various artifacts and expose them through dedicated plu

### Features

* Modular/Extensible via plugins
* Modular/Extensible via [plugins](docs/plugins.md)
* Support for YUM repositories (beskar-yum)
* Support for static file repositories (beskar-static)
* Support for OSTree repositories ([beskar-ostree](internal/plugins/ostree/README.md))

### Docker images

Expand Down Expand Up @@ -41,6 +42,12 @@ For beskar-static helm chart:
helm pull oci://ghcr.io/ctrliq/helm-charts/beskar-static --version 0.0.1 --untar
```

For beskar-static helm chart:

```
helm pull oci://ghcr.io/ctrliq/helm-charts/beskar-ostree --version 0.0.1 --untar
```

### Compilation

Binaries are not provided as part of releases, you can compile it yourself by running:
Expand Down
54 changes: 53 additions & 1 deletion build/mage/build.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,13 +55,16 @@ type binaryConfig struct {
buildTags []string
baseImage string
integrationTest *integrationTest
buildEnv map[string]string
buildExecStmts [][]string
}

const (
BeskarBinary = "beskar"
BeskarctlBinary = "beskarctl"
BeskarYUMBinary = "beskar-yum"
BeskarStaticBinary = "beskar-static"
BeskarOSTreeBinary = "beskar-ostree"
)

var binaries = map[string]binaryConfig{
Expand Down Expand Up @@ -101,7 +104,7 @@ var binaries = map[string]binaryConfig{
},
useProto: true,
// NOTE: restore in case alpine createrepo_c package is broken again
//baseImage: "debian:bullseye-slim",
// baseImage: "debian:bullseye-slim",
integrationTest: &integrationTest{
isPlugin: true,
envs: map[string]string{
Expand Down Expand Up @@ -130,6 +133,46 @@ var binaries = map[string]binaryConfig{
},
},
},
BeskarOSTreeBinary: {
configFiles: map[string]string{
"internal/plugins/ostree/pkg/config/default/beskar-ostree.yaml": "/etc/beskar/beskar-ostree.yaml",
},
genAPI: &genAPI{
path: "pkg/plugins/ostree/api/v1",
filename: "api.go",
interfaceName: "OSTree",
},
useProto: true,
execStmts: [][]string{
{
"apk", "add", "ostree", "ostree-dev",
},
},
buildExecStmts: [][]string{
{
// pkg-config is needed to compute CFLAGS
"apk", "add", "pkgconfig",
},
{
// Install gcc. Could have installed gc directly but this seems to be the recommended way for alpine.
"apk", "add", "build-base",
},
{
// Install ostree development libraries
"apk", "add", "ostree", "ostree-dev",
},
},
buildEnv: map[string]string{
"CGO_ENABLED": "1",
},
excludedPlatforms: map[dagger.Platform]struct{}{
"linux/arm64": {},
"linux/s390x": {},
"linux/ppc64le": {},
"linux/arm/v6": {},
"linux/arm/v7": {},
},
},
}

type Build mg.Namespace
Expand Down Expand Up @@ -170,6 +213,7 @@ func (b Build) Plugins(ctx context.Context) {
ctx,
mg.F(b.Plugin, BeskarYUMBinary),
mg.F(b.Plugin, BeskarStaticBinary),
mg.F(b.Plugin, BeskarOSTreeBinary),
)
}

Expand Down Expand Up @@ -274,6 +318,14 @@ func (b Build) build(ctx context.Context, name string) error {
golang = golang.WithEnvVariable(key, value)
}

for key, value := range binaryConfig.buildEnv {
golang = golang.WithEnvVariable(key, value)
}

for _, execStmt := range binaryConfig.buildExecStmts {
golang = golang.WithExec(execStmt)
}

path := filepath.Join("/output", binary)
inputCmd := filepath.Join("cmd", name)

Expand Down
12 changes: 12 additions & 0 deletions build/mage/lint.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,18 @@ func (Lint) Go(ctx context.Context) error {
WithWorkdir("/src").
With(goCache(client))

// Set up the environment for the linter per the settings of each binary.
// This could lead to conflicts if the binaries have different settings.
for _, config := range binaries {
for key, value := range config.buildEnv {
golangciLint = golangciLint.WithEnvVariable(key, value)
}

for _, execStmt := range config.buildExecStmts {
golangciLint = golangciLint.WithExec(execStmt)
}
}
Comment on lines +39 to +49
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The loop introduced to set up the environment for the linter based on each binary's settings could lead to conflicts if binaries have different settings. Consider implementing a check to ensure that there are no conflicting settings or redesigning this to isolate environments per binary.


golangciLint = golangciLint.WithExec([]string{
"golangci-lint", "-v", "run", "--modules-download-mode", "readonly", "--timeout", "5m",
})
Expand Down
17 changes: 14 additions & 3 deletions build/mage/test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,10 @@ package mage

import (
"context"
"fmt"
"strings"

"dagger.io/dagger"
"fmt"
"github.com/magefile/mage/mg"
"strings"
)

type Test mg.Namespace
Expand All @@ -29,8 +28,20 @@ func (Test) Unit(ctx context.Context) error {
WithWorkdir("/src").
With(goCache(client))

for _, config := range binaries {
for key, value := range config.buildEnv {
unitTest = unitTest.WithEnvVariable(key, value)
}

for _, execStmt := range config.buildExecStmts {
unitTest = unitTest.WithExec(execStmt)
}
}

unitTest = unitTest.WithExec([]string{
"go", "test", "-v", "-count=1", "./...",
}, dagger.ContainerWithExecOpts{
InsecureRootCapabilities: true,
})

return printOutput(ctx, unitTest)
Expand Down
23 changes: 23 additions & 0 deletions charts/beskar-ostree/.helmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*.orig
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
12 changes: 12 additions & 0 deletions charts/beskar-ostree/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: v2
description: A Helm chart for Beskar OSTree Repository Plugin
name: beskar-ostree
version: 0.0.1
appVersion: 0.0.1
home: https://github.com/ctrliq/beskar
maintainers:
- email: [email protected]
name: CtrlIQ Inc.
url: https://github.com/ctrliq/beskar
sources:
- https://github.com/ctrliq/beskar
Loading