Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add promotion step to remove a file/directory #3086

Merged
merged 3 commits into from
Dec 20, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions docs/docs/35-references/10-promotion-steps.md
Original file line number Diff line number Diff line change
Expand Up @@ -300,6 +300,53 @@ steps:
# Render manifests to ./out, commit, push, etc...
```

### `delete`

`delete` deletes the files or the directory,
nitishfy marked this conversation as resolved.
Show resolved Hide resolved

#### `delete` Configuration

| Name | Type | Required | Description |
|-----------|------|----------|------------------------------------------|
| `path` | `string` | Y | Path to the file or directory to delete. |

#### `delete` Example

The most common usage of this step is to clean up unwanted files
or directories before proceeding to subsequent steps in the promotion
process. For instance, you might use the delete step to remove
intermediate configurations or artifacts generated in earlier steps.

Consider a scenario where a Stage combines content from two working trees
to render Stage-specific manifests. If temporary files are generated during
this process, you can use the delete step to clean them up:

```yaml
vars:
- name: gitRepo
value: https://github.com/example/repo.git
steps:
- uses: git-clone
config:
repoURL: ${{ vars.gitRepo }}
checkout:
- commit: ${{ commitFrom(vars.gitRepo, warehouse("base")).ID }}
path: ./src
- commit: ${{ commitFrom(vars.gitRepo, warehouse(ctx.stage + "-overlay")).ID }}
path: ./overlay
- branch: stage/${{ ctx.stage }}
create: true
path: ./out
- uses: copy
config:
inPath: ./overlay/stages/${{ ctx.stage }}/kustomization.yaml
outPath: ./src/stages/${{ ctx.stage }}/kustomization.yaml
- uses: delete
config:
path: ./overlay/stages/${{ ctx.stage }}/temp-file.yaml
# Render manifests to ./out, commit, push, etc...
```
krancour marked this conversation as resolved.
Show resolved Hide resolved

### `kustomize-set-image`
krancour marked this conversation as resolved.
Show resolved Hide resolved

`kustomize-set-image` updates the `kustomization.yaml` file in a specified
Expand Down
151 changes: 151 additions & 0 deletions internal/directives/file_deleter.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
package directives

import (
"context"
"errors"
"fmt"
"os"
"path/filepath"
"strings"

securejoin "github.com/cyphar/filepath-securejoin"
"github.com/xeipuuv/gojsonschema"

kargoapi "github.com/akuity/kargo/api/v1alpha1"
)

func init() {
builtins.RegisterPromotionStepRunner(newFileDeleter(), nil)
}

// fileDeleter is an implementation of the PromotionStepRunner interface that
// deletes a file or directory.
type fileDeleter struct {
schemaLoader gojsonschema.JSONLoader
}

// newFileDeleter returns an implementation of the PromotionStepRunner interface
// that deletes a file or directory.
func newFileDeleter() PromotionStepRunner {
r := &fileDeleter{}
r.schemaLoader = getConfigSchemaLoader(r.Name())
return r
}

// Name implements the PromotionStepRunner interface
func (f *fileDeleter) Name() string {
return "delete"
}

// RunPromotionStep implements the PromotionStepRunner interface.
func (f *fileDeleter) RunPromotionStep(
ctx context.Context,
stepCtx *PromotionStepContext,
) (PromotionStepResult, error) {
// Validate the configuration against the JSON Schema.
if err := validate(f.schemaLoader, gojsonschema.NewGoLoader(stepCtx.Config), f.Name()); err != nil {
return PromotionStepResult{Status: kargoapi.PromotionPhaseErrored}, err
}

Check warning on line 48 in internal/directives/file_deleter.go

View check run for this annotation

Codecov / codecov/patch

internal/directives/file_deleter.go#L44-L48

Added lines #L44 - L48 were not covered by tests

// Convert the configuration into a typed object.
cfg, err := ConfigToStruct[DeleteConfig](stepCtx.Config)
if err != nil {
return PromotionStepResult{Status: kargoapi.PromotionPhaseErrored},
fmt.Errorf("could not convert config into %s config: %w", f.Name(), err)
}

Check warning on line 55 in internal/directives/file_deleter.go

View check run for this annotation

Codecov / codecov/patch

internal/directives/file_deleter.go#L51-L55

Added lines #L51 - L55 were not covered by tests

return f.runPromotionStep(ctx, stepCtx, cfg)

Check warning on line 57 in internal/directives/file_deleter.go

View check run for this annotation

Codecov / codecov/patch

internal/directives/file_deleter.go#L57

Added line #L57 was not covered by tests
}

func (f *fileDeleter) runPromotionStep(
_ context.Context,
stepCtx *PromotionStepContext,
cfg DeleteConfig,
) (PromotionStepResult, error) {
absPath, err := f.resolveAbsPath(stepCtx.WorkDir, cfg.Path)
if err != nil {
return PromotionStepResult{Status: kargoapi.PromotionPhaseErrored},
fmt.Errorf("could not secure join path %q: %w", cfg.Path, err)
}

Check warning on line 69 in internal/directives/file_deleter.go

View check run for this annotation

Codecov / codecov/patch

internal/directives/file_deleter.go#L67-L69

Added lines #L67 - L69 were not covered by tests

symlink, err := f.isSymlink(absPath)
if err != nil {
return PromotionStepResult{Status: kargoapi.PromotionPhaseErrored}, err
}

Check warning on line 74 in internal/directives/file_deleter.go

View check run for this annotation

Codecov / codecov/patch

internal/directives/file_deleter.go#L73-L74

Added lines #L73 - L74 were not covered by tests

if symlink {
err = os.Remove(absPath)
if err != nil {
return PromotionStepResult{Status: kargoapi.PromotionPhaseErrored}, err
}

Check warning on line 80 in internal/directives/file_deleter.go

View check run for this annotation

Codecov / codecov/patch

internal/directives/file_deleter.go#L79-L80

Added lines #L79 - L80 were not covered by tests
} else {
// Secure join the paths to prevent path traversal.
pathToDelete, err := securejoin.SecureJoin(stepCtx.WorkDir, cfg.Path)
if err != nil {
return PromotionStepResult{Status: kargoapi.PromotionPhaseErrored},
fmt.Errorf("could not secure join path %q: %w", cfg.Path, err)
}

Check warning on line 87 in internal/directives/file_deleter.go

View check run for this annotation

Codecov / codecov/patch

internal/directives/file_deleter.go#L85-L87

Added lines #L85 - L87 were not covered by tests

if err = removePath(pathToDelete); err != nil {
return PromotionStepResult{Status: kargoapi.PromotionPhaseErrored},
fmt.Errorf("failed to delete %q: %w", cfg.Path, sanitizePathError(err, stepCtx.WorkDir))
}
}

return PromotionStepResult{Status: kargoapi.PromotionPhaseSucceeded}, nil
}

func (f *fileDeleter) isSymlink(path string) (bool, error) {
fi, err := os.Lstat(path)
if err != nil {
// if file doesn't exist, it's not a symlink
if os.IsNotExist(err) {
return false, nil
}
return false, err

Check warning on line 105 in internal/directives/file_deleter.go

View check run for this annotation

Codecov / codecov/patch

internal/directives/file_deleter.go#L105

Added line #L105 was not covered by tests
}

return fi.Mode()&os.ModeSymlink != 0, nil
}

// resolveAbsPath resolves the absolute path from the workDir base path
func (f *fileDeleter) resolveAbsPath(workDir string, path string) (string, error) {
absBase, err := filepath.Abs(workDir)
if err != nil {
return "", err
}

Check warning on line 116 in internal/directives/file_deleter.go

View check run for this annotation

Codecov / codecov/patch

internal/directives/file_deleter.go#L115-L116

Added lines #L115 - L116 were not covered by tests

fullPath := filepath.Join(workDir, path)
absPath, err := filepath.Abs(fullPath)
if err != nil {
return "", err
}

Check warning on line 122 in internal/directives/file_deleter.go

View check run for this annotation

Codecov / codecov/patch

internal/directives/file_deleter.go#L121-L122

Added lines #L121 - L122 were not covered by tests

// Get the relative path from base to the requested path
// If the requested path tries to escape, this will return
// an error or a path starting with "../"
relPath, err := filepath.Rel(absBase, absPath)
if err != nil {
return "", err
}

Check warning on line 130 in internal/directives/file_deleter.go

View check run for this annotation

Codecov / codecov/patch

internal/directives/file_deleter.go#L129-L130

Added lines #L129 - L130 were not covered by tests

// Check if path attempts to escape
if strings.HasPrefix(relPath, "..") {
krancour marked this conversation as resolved.
Show resolved Hide resolved
return "", errors.New("path attempts to traverse outside the working directory")
}

Check warning on line 135 in internal/directives/file_deleter.go

View check run for this annotation

Codecov / codecov/patch

internal/directives/file_deleter.go#L134-L135

Added lines #L134 - L135 were not covered by tests

return absPath, nil
}

func removePath(path string) error {
fi, err := os.Lstat(path)
if err != nil {
return err
}

if fi.IsDir() {
return os.RemoveAll(path)
}

return os.Remove(path)
}
151 changes: 151 additions & 0 deletions internal/directives/file_deleter_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
package directives

import (
"context"
"os"
"path/filepath"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"

kargoapi "github.com/akuity/kargo/api/v1alpha1"
)

func Test_fileDeleter_runPromotionStep(t *testing.T) {
tests := []struct {
name string
setupFiles func(*testing.T) string
cfg DeleteConfig
assertions func(*testing.T, string, PromotionStepResult, error)
}{
{
name: "succeeds deleting file",
setupFiles: func(t *testing.T) string {
tmpDir := t.TempDir()

path := filepath.Join(tmpDir, "input.txt")
require.NoError(t, os.WriteFile(path, []byte("test content"), 0o600))

return tmpDir
},
cfg: DeleteConfig{
Path: "input.txt",
},
assertions: func(t *testing.T, _ string, result PromotionStepResult, err error) {
assert.NoError(t, err)
assert.Equal(t, PromotionStepResult{Status: kargoapi.PromotionPhaseSucceeded}, result)

_, statError := os.Stat("input.txt")
assert.True(t, os.IsNotExist(statError))
},
},
{
name: "succeeds deleting directory",
setupFiles: func(t *testing.T) string {
tmpDir := t.TempDir()
dirPath := filepath.Join(tmpDir, "dirToDelete")
require.NoError(t, os.Mkdir(dirPath, 0o700))
return tmpDir
},
cfg: DeleteConfig{
Path: "dirToDelete",
},
assertions: func(t *testing.T, workDir string, result PromotionStepResult, err error) {
assert.NoError(t, err)
assert.Equal(t, PromotionStepResult{Status: kargoapi.PromotionPhaseSucceeded}, result)

_, statErr := os.Stat(filepath.Join(workDir, "dirToDelete"))
assert.True(t, os.IsNotExist(statErr))
},
},
{
name: "fails for non-existent path",
hiddeco marked this conversation as resolved.
Show resolved Hide resolved
setupFiles: func(t *testing.T) string {
return t.TempDir()
},
cfg: DeleteConfig{
Path: "nonExistentFile.txt",
},
assertions: func(t *testing.T, _ string, result PromotionStepResult, err error) {
assert.Error(t, err)
assert.Equal(t, PromotionStepResult{Status: kargoapi.PromotionPhaseErrored}, result)
},
},
{
name: "removes symlink only",
nitishfy marked this conversation as resolved.
Show resolved Hide resolved
setupFiles: func(t *testing.T) string {
tmpDir := t.TempDir()

inDir := filepath.Join(tmpDir, "input")
require.NoError(t, os.Mkdir(inDir, 0o755))

filePath := filepath.Join(inDir, "input.txt")
require.NoError(t, os.WriteFile(filePath, []byte("test content"), 0o600))

symlinkPath := filepath.Join(inDir, "symlink.txt")
require.NoError(t, os.Symlink("input.txt", symlinkPath))

return tmpDir
},
cfg: DeleteConfig{
Path: "input/symlink.txt",
},
assertions: func(t *testing.T, workDir string, result PromotionStepResult, err error) {
assert.NoError(t, err)
require.Equal(t, PromotionStepResult{Status: kargoapi.PromotionPhaseSucceeded}, result)

_, statErr := os.Stat(filepath.Join(workDir, "input", "input.txt"))
assert.NoError(t, statErr)

_, statErr = os.Lstat(filepath.Join(workDir, "input", "symlink.txt"))
assert.Error(t, statErr)
assert.True(t, os.IsNotExist(statErr))
},
},
{
name: "removes a file within a symlink",
setupFiles: func(t *testing.T) string {
tmpDir := t.TempDir()

inDir := filepath.Join(tmpDir, "bar")
require.NoError(t, os.Mkdir(inDir, 0o755))

filePath := filepath.Join(inDir, "file.txt")
require.NoError(t, os.WriteFile(filePath, []byte("test content"), 0o600))

symlinkPath := filepath.Join(tmpDir, "foo")
require.NoError(t, os.Symlink(inDir, symlinkPath))

return tmpDir
},
cfg: DeleteConfig{
Path: "foo/",
},
assertions: func(t *testing.T, workDir string, result PromotionStepResult, err error) {
assert.NoError(t, err)
require.Equal(t, PromotionStepResult{Status: kargoapi.PromotionPhaseSucceeded}, result)

_, statErr := os.Stat(filepath.Join(workDir, "foo", "file.txt"))
assert.Error(t, statErr)
assert.True(t, os.IsNotExist(statErr))

_, statErr = os.Stat(filepath.Join(workDir, "bar", "file.txt"))
assert.NoError(t, statErr)
},
},
}
runner := &fileDeleter{}

for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
workDir := tt.setupFiles(t)
result, err := runner.runPromotionStep(
context.Background(),
&PromotionStepContext{WorkDir: workDir},
tt.cfg,
)
tt.assertions(t, workDir, result, err)
})
}
}
14 changes: 14 additions & 0 deletions internal/directives/schemas/delete-config.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"title": "DeleteConfig",
"type": "object",
"additionalProperties": false,
"required": ["path"],
"properties": {
"path": {
"type": "string",
"description": "Path is the path to the file or directory to delete.",
"minLength": 1
}
}
}
5 changes: 5 additions & 0 deletions internal/directives/zz_config_types.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading