diff --git a/docs/addons/kubedump/README.md b/docs/addons/kubedump/README.md index 167d214..e194f54 100644 --- a/docs/addons/kubedump/README.md +++ b/docs/addons/kubedump/README.md @@ -25,8 +25,8 @@ This guide will give you an overview of how the docs are organized. Stash KubeDump backup documentations are organized as below: -- [How does it work?](/docs/addons/kubedump/overview/index.md) gives an overview of how manifest backup works in Stash. +- [How does it work?](/docs/guides/kubedump/overview/index.md) gives an overview of how manifest backup works in Stash. - [Cluster Backup](/docs/addons/kubedump/cluster/index.md) shows how to backup all the cluster manifests. - [Namespace Backup](/docs/addons/kubedump/namespace/index.md) shows how to backup the manifests only for a particular namespace. -- [Application Backup](/docs/addons/kubedump/application/index.md) shows how to manifests of a particular application. +- [Application Backup](/docs/guides/kubedump/application/index.md) shows how to manifests of a particular application. - [Customizing Backup](/docs/addons/kubedump/customization/index.md) shows how to customize the backup process. diff --git a/docs/addons/kubedump/application/examples/backupconfiguration.yaml b/docs/addons/kubedump/application/examples/backupconfiguration.yaml deleted file mode 100644 index 8cbc8aa..0000000 --- a/docs/addons/kubedump/application/examples/backupconfiguration.yaml +++ /dev/null @@ -1,27 +0,0 @@ -apiVersion: stash.appscode.com/v1beta1 -kind: BackupConfiguration -metadata: - name: application-manifest-backup - namespace: demo -spec: - schedule: "*/5 * * * *" - task: - name: kubedump-backup-0.1.0 - params: - - name: includeDependants - value: "true" - repository: - name: application-resource-storage - target: - ref: - apiVersion: apps/v1 - kind: Deployment - name: stash-stash-enterprise - namespace: kube-system - runtimeSettings: - pod: - serviceAccountName: cluster-resource-reader - retentionPolicy: - name: keep-last-5 - keepLast: 5 - prune: true diff --git a/docs/addons/kubedump/application/examples/repository.yaml b/docs/addons/kubedump/application/examples/repository.yaml deleted file mode 100644 index c2afa3b..0000000 --- a/docs/addons/kubedump/application/examples/repository.yaml +++ /dev/null @@ -1,11 +0,0 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: application-resource-storage - namespace: demo -spec: - backend: - gcs: - bucket: stash-testing - prefix: /manifests/applications/kube-system/stash-enterprise - storageSecretName: gcs-secret diff --git a/docs/addons/kubedump/application/images/application_manifest_backup.png b/docs/addons/kubedump/application/images/application_manifest_backup.png deleted file mode 100644 index 303719e..0000000 Binary files a/docs/addons/kubedump/application/images/application_manifest_backup.png and /dev/null differ diff --git a/docs/addons/kubedump/application/index.md b/docs/addons/kubedump/application/index.md deleted file mode 100644 index 9439c87..0000000 --- a/docs/addons/kubedump/application/index.md +++ /dev/null @@ -1,454 +0,0 @@ ---- -title: Backup YAMLs of an Application | Stash -description: Backup application manifests along with it's dependant using Stash -menu: - docs_{{ .version }}: - identifier: stash-kubedump-application - name: Application Manifest Backup - parent: stash-kubedump - weight: 40 -product_name: kubestash -menu_name: docs_{{ .version }} -section_menu_id: stash-addons ---- - -# Backup YAMLs of an Application using Stash - -Stash `{{< param "info.version" >}}` supports taking backup of the resource YAMLs using `kubedump` plugin. This guide will show you how you can take a backup of the YAMLs of an application along with it's dependant using Stash. - -## Before You Begin - -- At first, you need to have a Kubernetes cluster, and the `kubectl` command-line tool must be configured to communicate with your cluster. -- Install Stash in your cluster following the steps [here](/docs/setup/install/stash/index.md). -- Install Stash `kubectl` plugin in your local machine following the steps [here](/docs/setup/install/kubectl-plugin/index.md). -- If you are not familiar with how Stash backup the resource YAMLs, please check the following guide [here](/docs/addons/kubedump/overview/index.md). - -You have to be familiar with the following custom resources: - -- [AppBinding](/docs/concepts/crds/appbinding/index.md) -- [Function](/docs/concepts/crds/function/index.md) -- [Task](/docs/concepts/crds/task/index.md) -- [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md) -- [BackupSession](/docs/concepts/crds/backupsession/index.md) - -To keep things isolated, we are going to use a separate namespace called `demo` throughout this tutorial. Create the `demo` namespace if you haven't created it already. - -```bash -$ kubectl create ns demo -namespace/demo created -``` - -> Note: YAML files used in this tutorial are stored [here](https://github.com/kubestash/docs/tree/{{< param "info.version" >}}/docs/addons/kubedump/application/examples). - -### Prepare for Backup - -In this section, we are going to configure a backup for YAML definition of a Deployment along with its ReplicaSet and its Pods. - -#### Ensure `kubedump` Addon - -When you install the Stash, it will automatically install all the official addons. Make sure that `kubedump` addon was installed properly using the following command. - -```bash -❯ kubectl get tasks.stash.appscode.com | grep kubedump -kubedump-backup-0.1.0 23s -``` - -#### Prepare Backend - -We are going to store our backed-up data into a GCS bucket. So, we need to create a Secret with GCS credentials and a `Repository` object with the bucket information. If you want to use a different backend, please read the respective backend configuration doc from [here](/docs/guides/backends/overview/index.md). - -**Create Storage Secret:** - -At first, let's create a secret called `gcs-secret` with access credentials to our desired GCS bucket, - -```bash -$ echo -n 'changeit' > RESTIC_PASSWORD -$ echo -n '' > GOOGLE_PROJECT_ID -$ cat downloaded-sa-json.key > GOOGLE_SERVICE_ACCOUNT_JSON_KEY -$ kubectl create secret generic -n demo gcs-secret \ - --from-file=./RESTIC_PASSWORD \ - --from-file=./GOOGLE_PROJECT_ID \ - --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY -secret/gcs-secret created -``` - -**Create Repository:** - -Now, crete a `Repository` object with the information of your desired bucket. Below is the YAML of `Repository` object we are going to create, - -```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: application-resource-storage - namespace: demo -spec: - backend: - gcs: - bucket: stash-testing - prefix: /manifests/applications/kube-system/stash-enterprise - storageSecretName: gcs-secret -``` - -Let's create the `Repository` we have shown above, - -```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/addons/kubedump/application/examples/repository.yaml -repository.stash.appscode.com/application-resource-storage created -``` - -#### Create RBAC - -The `kubedump` plugin requires read permission for the application resources. By default, Stash does not grant such permissions. We have to provide the necessary permissions manually. - -Here, is the YAML of the `ServiceAccount`, `ClusterRole`, and `ClusterRoleBinding` that we are going to use for granting the necessary permissions. - -```yaml -apiVersion: v1 -kind: ServiceAccount -metadata: - name: cluster-resource-reader - namespace: demo ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRole -metadata: - name: cluster-resource-reader -rules: -- apiGroups: ["*"] - resources: ["*"] - verbs: ["get","list"] ---- -apiVersion: rbac.authorization.k8s.io/v1 -kind: ClusterRoleBinding -metadata: - name: cluster-resource-reader -subjects: -- kind: ServiceAccount - name: cluster-resource-reader - namespace: demo -roleRef: - kind: ClusterRole - name: cluster-resource-reader - apiGroup: rbac.authorization.k8s.io -``` - -Here, we have give permission to read all the cluster resources. You can restrict this permission to a particular application resources only. - -Let's create the RBAC resources we have shown above, - -```bash -❯ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/addons/kubedump/application/examples/rbac.yaml -serviceaccount/cluster-resource-reader created -clusterrole.rbac.authorization.k8s.io/cluster-resource-reader created -clusterrolebinding.rbac.authorization.k8s.io/cluster-resource-reader created -``` - -Now, we are ready for backup. In the next section, we are going to schedule a backup for our cluster resources. - -### Backup - -To schedule a backup, we have to create a `BackupConfiguration` object. Then Stash will create a CronJob to periodically backup the database. - -At first, lets list available Deployment in `kube-system` namespace. - -```bash -❯ kubectl get deployments -n kube-system -NAME READY UP-TO-DATE AVAILABLE AGE -coredns 2/2 2 2 13d -stash-stash-enterprise 1/1 1 1 30h -``` - -Here, we are going to setup backup YAMLs for `stash-stash-enterprise` Deployment. - -#### Create BackupConfiguration - -Below is the YAML for `BackupConfiguration` object we care going to use to backup the YAMLs of the cluster resources, - -```yaml -apiVersion: stash.appscode.com/v1beta1 -kind: BackupConfiguration -metadata: - name: application-manifest-backup - namespace: demo -spec: - schedule: "*/5 * * * *" - task: - name: kubedump-backup-0.1.0 - params: - - name: includeDependants - value: "true" - repository: - name: application-resource-storage - target: - ref: - apiVersion: apps/v1 - kind: Deployment - name: stash-stash-enterprise - namespace: kube-system - runtimeSettings: - pod: - serviceAccountName: cluster-resource-reader - retentionPolicy: - name: keep-last-5 - keepLast: 5 - prune: true -``` - -Here, - -- `.spec.schedule` specifies that we want to backup the cluster resources at 5 minutes intervals. -- `.spec.task.name` specifies the name of the Task object that specifies the necessary Functions and their execution order to backup the resource YAMLs. -- `.spec.repository.name` specifies the Repository CR name we have created earlier with backend information. -- `.spec.target` specifies the targeted application that we are going to backup. -- `.spec.runtimeSettings.pod.serviceAccountName` specifies the ServiceAccount name that we have created earlier with cluster-wide resource reading permission. -- `.spec.retentionPolicy` specifies a policy indicating how we want to cleanup the old backups. - -Let's create the `BackupConfiguration` object we have shown above, - -```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/addons/kubedump/application/examples/backupconfiguration.yaml -backupconfiguration.stash.appscode.com/application-manifest-backup created -``` - -#### Verify Backup Setup Successful - -If everything goes well, the phase of the `BackupConfiguration` should be `Ready`. The `Ready` phase indicates that the backup setup is successful. Let's verify the `Phase` of the BackupConfiguration, - -```bash -❯ kubectl get backupconfiguration -n demo -NAME TASK SCHEDULE PAUSED PHASE AGE -application-manifest-backup kubedump-backup-0.1.0 */5 * * * * Ready 2m46s -``` - -#### Verify CronJob - -Stash will create a CronJob with the schedule specified in `spec.schedule` field of `BackupConfiguration` object. - -Verify that the CronJob has been created using the following command, - -```bash -❯ kubectl get cronjob -n demo -NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE -stash-trigger-demo-application-manifest-backup */5 * * * * False 0 36s -``` - -#### Wait for BackupSession - -The `stash-trigger-application-manifest-backup` CronJob will trigger a backup on each scheduled slot by creating a `BackupSession` object. - -Now, wait for a schedule to appear. Run the following command to watch for a `BackupSession` object, - -```bash -❯ kubectl get backupsession -n demo -w -NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE -application-manifest-backup-1652269801 BackupConfiguration application-manifest-backup 0s -application-manifest-backup-1652269801 BackupConfiguration application-manifest-backup Pending 0s -application-manifest-backup-1652269801 BackupConfiguration application-manifest-backup Running 0s -application-manifest-backup-1652269801 BackupConfiguration application-manifest-backup Succeeded 59s 59s -``` - -Here, the phase `Succeeded` means that the backup process has been completed successfully. - -#### Verify Backup - -Now, we are going to verify whether the backed-up data is present in the backend or not. Once a backup is completed, Stash will update the respective `Repository` object to reflect the backup completion. Check that the repository `application-resource-storage` has been updated by the following command, - -```bash -❯ kubectl get repository -n demo -NAME INTEGRITY SIZE SNAPSHOT-COUNT LAST-SUCCESSFUL-BACKUP AGE -application-resource-storage true 14.902 KiB 1 61s 11m -``` - -Now, if we navigate to the GCS bucket, we will see the backed up data has been stored in `/manifest/applications/kube-system/stash-enterprise` directory as specified by `.spec.backend.gcs.prefix` field of the `Repository` object. - -
- Backup data in GCS Bucket -
Fig: Backup data in GCS Bucket
-
- -> Note: Stash keeps all the backed-up data encrypted. So, data in the backend will not make any sense until they are decrypted. - -## Restore - -Stash does not provide any automatic mechanism to restore the cluster resources from the backed-up YAMLs. Your application might be managed by Helm or by an operator. In such cases, just applying the YAMLs is not enough to restore the application. Furthermore, there might be an order issue. Some resources must be applied before others. It is difficult to generalize and codify various application-specific logic. - -Therefore, it is the user's responsibility to download the backed-up YAMLs and take the necessary steps based on his application to restore it properly. - -### Download the YAMLs - -Stash provides a [kubectl plugin](https://stash.run/docs/v2022.05.12/guides/cli/cli/#download-snapshots) for making it easy to download a snapshot locally. - -Now, let's download the latest Snapshot from our backed-up data into the `$HOME/Downloads/stash/applications/kube-system/stash-enterprise` folder of our local machine. - -```bash -❯ kubectl stash download -n demo application-resource-storage --destination=$HOME/Downloads/stash/applications/kube-system/stash-enterprise --snapshots="latest" -``` - -Now, lets use [tree](https://linux.die.net/man/1/tree) command to inspect downloaded YAMLs files. - -```bash -❯ tree $HOME/Downloads/stash/applications/kube-system/stash-enterprise -/home/emruz/Downloads/stash/applications/kube-system/stash-enterprise -└── latest - └── tmp - └── resources - ├── ReplicaSet - │   └── stash-stash-enterprise-567dd95f5b - │   ├── Pod - │   │   └── stash-stash-enterprise-567dd95f5b-6xtxg - │   │   └── stash-stash-enterprise-567dd95f5b-6xtxg.yaml - │   └── stash-stash-enterprise-567dd95f5b.yaml - └── stash-stash-enterprise.yaml - -7 directories, 3 files -``` - -As you can see that the Deployment has been backed up along with it's ReplicaSet and Pods. - -Let's inspect the YAML of `stash-stash-enterprise.yaml` file, - -```yaml -❯ cat $HOME/Downloads/stash/applications/kube-system/stash-enterprise/latest/tmp/resources/stash-stash-enterprise.yaml -apiVersion: apps/v1 -kind: Deployment -metadata: - annotations: - deployment.kubernetes.io/revision: "1" - meta.helm.sh/release-name: stash - meta.helm.sh/release-namespace: kube-system - labels: - app.kubernetes.io/instance: stash - app.kubernetes.io/managed-by: Helm - app.kubernetes.io/name: stash-enterprise - app.kubernetes.io/version: v0.20.0 - helm.sh/chart: stash-enterprise-v0.20.0 - name: stash-stash-enterprise - namespace: kube-system -spec: - progressDeadlineSeconds: 600 - replicas: 1 - revisionHistoryLimit: 10 - selector: - matchLabels: - app.kubernetes.io/instance: stash - app.kubernetes.io/name: stash-enterprise - strategy: - rollingUpdate: - maxSurge: 25% - maxUnavailable: 25% - type: RollingUpdate - template: - metadata: - annotations: - checksum/apiregistration.yaml: ea1443f1d9a807c14104b3e24ca051acb32c215743fde21c682ccb1876deee8d - labels: - app.kubernetes.io/instance: stash - app.kubernetes.io/name: stash-enterprise - spec: - containers: - - args: - - run - - --v=3 - - --docker-registry=stashed - - --image=stash-enterprise - - --image-tag=v0.20.0 - - --secure-port=8443 - - --audit-log-path=- - - --tls-cert-file=/var/serving-cert/tls.crt - - --tls-private-key-file=/var/serving-cert/tls.key - - --pushgateway-url=http://stash-stash-enterprise.kube-system.svc:56789 - - --enable-mutating-webhook=true - - --enable-validating-webhook=true - - --bypass-validating-webhook-xray=false - - --use-kubeapiserver-fqdn-for-aks=true - - --cron-job-psp=baseline - - --backup-job-psp=baseline - - --restore-job-psp=baseline - - --nva-cpu=100m - - --nva-memory=128Mi - - --nva-user=2000 - - --nva-privileged-mode=false - - --nva-psp=baseline - - --license-file=/var/run/secrets/appscode/license/key.txt - - --license-apiservice=v1beta1.admission.stash.appscode.com - env: - - name: POD_NAME - valueFrom: - fieldRef: - apiVersion: v1 - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - apiVersion: v1 - fieldPath: metadata.namespace - image: stashed/stash-enterprise:v0.20.0 - imagePullPolicy: IfNotPresent - name: operator - ports: - - containerPort: 8443 - protocol: TCP - resources: - requests: - cpu: 100m - securityContext: {} - terminationMessagePolicy: File - volumeMounts: - - mountPath: /var/serving-cert - name: serving-cert - - mountPath: /var/run/secrets/appscode/license - name: license - - args: - - --web.listen-address=:56789 - - --persistence.file=/var/pv/pushgateway.dat - image: prom/pushgateway:v1.4.2 - imagePullPolicy: IfNotPresent - name: pushgateway - ports: - - containerPort: 56789 - protocol: TCP - resources: {} - securityContext: {} - terminationMessagePolicy: File - volumeMounts: - - mountPath: /var/pv - name: data-volume - - mountPath: /tmp - name: stash-scratchdir - nodeSelector: - kubernetes.io/os: linux - restartPolicy: Always - schedulerName: default-scheduler - securityContext: - fsGroup: 65535 - serviceAccount: stash-stash-enterprise - serviceAccountName: stash-stash-enterprise - volumes: - - emptyDir: {} - name: data-volume - - emptyDir: {} - name: stash-scratchdir - - name: serving-cert - secret: - defaultMode: 420 - secretName: stash-stash-enterprise-apiserver-cert - - name: license - secret: - defaultMode: 420 - secretName: stash-stash-enterprise-license -``` - -Now, you can use these YAML files to re-create your desired application. - -## Cleanup - -To cleanup the Kubernetes resources created by this tutorial, run: - -```bash -kubectl delete -n demo backupconfiguration application-manifest-backup -kubectl delete -n demo repository application-resource-storage -kubectl delete -n demo serviceaccount cluster-resource-reader -kubectl delete -n demo clusterrole cluster-resource-reader -kubectl delete -n demo clusterrolebinding cluster-resource-reader -``` diff --git a/docs/addons/kubedump/cluster/examples/backupconfiguration.yaml b/docs/addons/kubedump/cluster/examples/backupconfiguration.yaml index e9ee92b..9ed1ea8 100644 --- a/docs/addons/kubedump/cluster/examples/backupconfiguration.yaml +++ b/docs/addons/kubedump/cluster/examples/backupconfiguration.yaml @@ -1,18 +1,36 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: name: cluster-resources-backup namespace: demo spec: - schedule: "*/5 * * * *" - task: - name: kubedump-backup-0.1.0 - repository: - name: cluster-resource-storage - runtimeSettings: - pod: - serviceAccountName: cluster-resource-reader - retentionPolicy: - name: keep-last-5 - keepLast: 5 - prune: true + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /cluster-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader \ No newline at end of file diff --git a/docs/addons/kubedump/cluster/examples/repository.yaml b/docs/addons/kubedump/cluster/examples/repository.yaml index fdc38ef..0461b26 100644 --- a/docs/addons/kubedump/cluster/examples/repository.yaml +++ b/docs/addons/kubedump/cluster/examples/repository.yaml @@ -1,11 +1,17 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: cluster-resource-storage + name: gcs-storage namespace: demo spec: - backend: + storage: + provider: gcs gcs: - bucket: stash-testing - prefix: /manifests/cluster - storageSecretName: gcs-secret + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/addons/kubedump/cluster/images/cluster_manifests_backup.png b/docs/addons/kubedump/cluster/images/cluster_manifests_backup.png index 2ea0e44..830e412 100644 Binary files a/docs/addons/kubedump/cluster/images/cluster_manifests_backup.png and b/docs/addons/kubedump/cluster/images/cluster_manifests_backup.png differ diff --git a/docs/addons/kubedump/cluster/index.md b/docs/addons/kubedump/cluster/index.md index 339b080..01f096e 100644 --- a/docs/addons/kubedump/cluster/index.md +++ b/docs/addons/kubedump/cluster/index.md @@ -12,24 +12,25 @@ menu_name: docs_{{ .version }} section_menu_id: stash-addons --- -# Backup resource YAMLs of entire cluster using Stash +# Backup resource YAMLs of entire cluster using KubeStash -Stash `{{< param "info.version" >}}` supports taking backup of the resource YAMLs using `kubedump` plugin. This guide will show you how you can take a backup of the resource YAMLs of your entire cluster using Stash. +KubeStash `{{< param "info.version" >}}` supports taking backup of the resource YAMLs using `kubedump` plugin. This guide will show you how you can take a backup of the resource YAMLs of your entire cluster using KubeStash. ## Before You Begin - At first, you need to have a Kubernetes cluster, and the `kubectl` command-line tool must be configured to communicate with your cluster. -- Install Stash in your cluster following the steps [here](/docs/setup/install/stash/index.md). -- Install Stash `kubectl` plugin in your local machine following the steps [here](/docs/setup/install/kubectl-plugin/index.md). -- If you are not familiar with how Stash backup the resource YAMLs, please check the following guide [here](/docs/addons/kubedump/overview/index.md). +- Install KubeStash in your cluster following the steps [here](/docs/setup/install/kubestash/index.md). +- Install KubeStash `kubectl` plugin in your local machine following the steps [here](/docs/setup/install/kubectl-plugin/index.md). +- If you are not familiar with how KubeStash backup the resource YAMLs, please check the following guide [here](/docs/guides/kubedump/overview/index.md). You have to be familiar with the following custom resources: -- [AppBinding](/docs/concepts/crds/appbinding/index.md) -- [Function](/docs/concepts/crds/function/index.md) -- [Task](/docs/concepts/crds/task/index.md) +- [BackupStorage](/docs/concepts/crds/backupstorage/index.md) - [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md) - [BackupSession](/docs/concepts/crds/backupsession/index.md) +- [SnapShot](/docs/concepts/crds/snapshot/index.md) +- [RestoreSession](/docs/concepts/crds/restoresession/index.md) +- [RetentionPolicy](/docs/concepts/crds/retentionpolicy/index.md) To keep things isolated, we are going to use a separate namespace called `demo` throughout this tutorial. Create the `demo` namespace if you haven't created it already. @@ -38,68 +39,139 @@ $ kubectl create ns demo namespace/demo created ``` -> Note: YAML files used in this tutorial are stored [here](https://github.com/kubestash/docs/tree/{{< param "info.version" >}}/docs/addons/kubedump/cluster/examples). +> Note: YAML files used in this tutorial are stored [here](https://github.com/kubestash/docs/tree/{{< param "info.version" >}}/docs/guides/kubedump/namespace/examples). ### Prepare for Backup -In this section, we are going to configure a backup for all the resource YAMLs of our cluster. +In this section, we are going to configure a backup for all the resource YAMLs of `kube-system` namespace. #### Ensure `kubedump` Addon -When you install the Stash, it will automatically install all the official addons. Make sure that `kubedump` addon was installed properly using the following command. +**Verify necessary Addons and Functions:** + +When you install KubeStash, it automatically creates the necessary `Addon` and `Function` to backup Kubernetes Resource YAML. + +Let's verify that KubeStash has created the necessary `Function` to backup Kubernetes Resource YAML by the following command, + +```bash +$ kubectl get functions | grep 'kubedump' +kubedump-backup 142m +``` + +Also, verify that the necessary `Addon` has been created, ```bash -❯ kubectl get tasks.stash.appscode.com | grep kubedump -kubedump-backup-0.1.0 23s +❯ kubectl get addons | grep 'kubedump' +kubedump-addon 143m +``` + +Now, you can view all `BackupTasks` and `RestoreTasks` of the `pvc-addon` addon using the following command + +```bash +❯ kubectl get addon kubedump-addon -o=jsonpath='{.spec.backupTasks[*].name}'| tr ' ' '\n' +manifest-backup ``` #### Prepare Backend -We are going to store our backed-up data into a GCS bucket. So, we need to create a Secret with GCS credentials and a `Repository` object with the bucket information. If you want to use a different backend, please read the respective backend configuration doc from [here](/docs/guides/backends/overview/index.md). +Now, we are going to store our backed-up data into a GCS bucket using KubeStash. We have to create a Secret and a `BackupStorage` object with access credentials and backend information respectively. If you want to use a different backend, please read the respective backend configuration doc from [here](/docs/guides/backends/overview/index.md). + +> For GCS backend, if the bucket does not exist, KubeStash needs `Storage Object Admin` role permissions to create the bucket. For more details, please check the following [guide](/docs/guides/backends/gcs/index.md). **Create Storage Secret:** At first, let's create a secret called `gcs-secret` with access credentials to our desired GCS bucket, ```bash -$ echo -n 'changeit' > RESTIC_PASSWORD $ echo -n '' > GOOGLE_PROJECT_ID -$ cat downloaded-sa-json.key > GOOGLE_SERVICE_ACCOUNT_JSON_KEY +$ cat /path/to/downloaded/sa_key_file.json > GOOGLE_SERVICE_ACCOUNT_JSON_KEY $ kubectl create secret generic -n demo gcs-secret \ - --from-file=./RESTIC_PASSWORD \ --from-file=./GOOGLE_PROJECT_ID \ --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY secret/gcs-secret created ``` -**Create Repository:** +**Create BackupStorage:** -Now, crete a `Repository` object with the information of your desired bucket. Below is the YAML of `Repository` object we are going to create, +Now, create a `BackupStorage` custom resource specifying the desired bucket, and directory inside the bucket where the backed up data will be stored. + +Below is the YAML of `BackupStorage` object that we are going to create, ```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: cluster-resource-storage + name: gcs-storage namespace: demo spec: - backend: + storage: + provider: gcs gcs: - bucket: stash-testing - prefix: /manifests/cluster - storageSecretName: gcs-secret + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut +``` + +Let's create the `BackupStorage` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/kubedump/namespace/examples/backupstorage.yaml +backupstorage.storage.kubestash.com/gcs-repo created +``` + +Now, we also have to create another secret with an encryption key `RESTIC_PASSWORD` for Restic. This secret will be used by Restic for both encrypting and decrypting the backup data during backup & restore. + +**Create Restic Repository Secret:** + +Let's create a Secret named `encry-secret` with an encryption key `RESTIC_PASSWORD` for Restic. + +```bash +$ echo -n 'changeit' > RESTIC_PASSWORD +$ kubectl create secret generic -n demo encry-secret \ + --from-file=./RESTIC_PASSWORD +secret/encryption-secret created ``` -Let's create the `Repository` we have shown above, +Now, we have to create a `RetentionPolicy` custom resource which specifies how the old `Snapshots` should be cleaned up. + +**Create RetentionPolicy:** + +Below is the YAML of the `RetentionPolicy` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same +``` + +Let's create the `RetentionPolicy` object that we have shown above, ```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/addons/kubedump/cluster/examples/repository.yaml -repository.stash.appscode.com/cluster-resource-storage created +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/kubedump/namespace/examples/retentionpolicy.yaml +retentionpolicy.storage.kubestash.com/demo-retention created ``` + #### Create RBAC -The `kubedump` plugin requires read permission for all the cluster resources. By default, Stash does not grant such cluster-wide permissions. We have to provide the necessary permissions manually. +The `kubedump` plugin requires read permission for all the cluster resources. By default, KubeStash does not grant such cluster-wide permissions. We have to provide the necessary permissions manually. Here, is the YAML of the `ServiceAccount`, `ClusterRole`, and `ClusterRoleBinding` that we are going to use for granting the necessary permissions. @@ -153,39 +225,64 @@ To schedule a backup, we have to create a `BackupConfiguration` object. Then Sta Below is the YAML for `BackupConfiguration` object we care going to use to backup the YAMLs of the cluster resources, ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: - name: cluster-resources-backup + name: kube-system-backup namespace: demo spec: - schedule: "*/5 * * * *" - task: - name: kubedump-backup-0.1.0 - repository: - name: cluster-resource-storage - runtimeSettings: - pod: - serviceAccountName: cluster-resource-reader - retentionPolicy: - name: keep-last-5 - keepLast: 5 - prune: true + target: + apiGroup: "" + kind: Namespace + name: kube-system + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader ``` Here, -- `.spec.schedule` specifies that we want to backup the cluster resources at 5 minutes intervals. -- `.spec.task.name` specifies the name of the Task object that specifies the necessary Functions and their execution order to backup the resource YAMLs. -- `.spec.repository.name` specifies the Repository CR name we have created earlier with backend information. -- `.spec.runtimeSettings.pod.serviceAccountName` specifies the ServiceAccount name that we have created earlier with cluster-wide resource reading permission. -- `.spec.retentionPolicy` specifies a policy indicating how we want to cleanup the old backups. +- `spec.target` specifies the backup target. `apiGroup`, `kind`, `namespace` and `name` refers to the `apiGroup`, `kind`, `namespace` and `name` of the targeted workload respectively. +- `spec.sessions[*].scheduler.schedule` specifies a [cron expression](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) indicates that `BackupSession` will be created at 5 minute interval. +- `spec.sessions[*].repositories[*].name` specifies the name of the `Repository` object that holds the backend information. +- `spec.sessions[*].repositories[*].backend` specifies the name of the backend that holds the `BackupStorage` information. +- `spec.sessions[*].repositories[*].directory` specifies the path of the `Repository` where the backed up data will be stored. +- `spec.sessions[*].repositories[*].encryptionSecret` specifies the encryption secret for `Restic Repository` which will be used to encrypting the backup data. +- `spec.sessions[*].addon.name` specifies the name of the `Addon` object that specifies addon configuration that will be used to perform backup of a stand-alone PVC. +- `spec.sessions[*].addon.tasks[*].name` specifies the name of the `Task` that holds the `Function` and their order of execution to perform backup of a stand-alone PVC. Let's create the `BackupConfiguration` object we have shown above, ```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/addons/kubedump/cluster/examples/backupconfiguration.yaml -backupconfiguration.stash.appscode.com/cluster-resources-backup created +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/kubedump/namespace/examples/backupconfiguration.yaml +backupconfiguration.core.kubestash.com/kube-system-backup created ``` #### Verify Backup Setup Successful @@ -194,51 +291,67 @@ If everything goes well, the phase of the `BackupConfiguration` should be `Ready ```bash ❯ kubectl get backupconfiguration -n demo -NAME TASK SCHEDULE PAUSED PHASE AGE -cluster-resources-backup kubedump-backup-0.1.0 */5 * * * * Ready 18s -``` +NAME PHASE PAUSED AGE +kube-system-backup Ready 79s -#### Verify CronJob +``` -Stash will create a CronJob with the schedule specified in `spec.schedule` field of `BackupConfiguration` object. +**Verify Repository:** -Verify that the CronJob has been created using the following command, +Verify that KubeStash has created `Repositories` that holds the `BackupStorage` information by the following command, ```bash -❯ kubectl get cronjob -n demo -NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE -stash-trigger-cluster-resources-backup */5 * * * * False 0 49s +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository Ready 28s ``` -#### Wait for BackupSession +**Verify CronJob:** -The `stash-trigger-cluster-resources-backup` CronJob will trigger a backup on each scheduled slot by creating a `BackupSession` object. +Verify that KubeStash has created a CronJob to trigger a periodic backup of the targeted PVC by the following command, -Now, wait for a schedule to appear. Run the following command to watch for a `BackupSession` object, +```bash +$ kubectl get cronjob -n demo +NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE +trigger-kube-system-backup-frequent-backup */5 * * * * False 0 45s +``` + +**Wait for BackupSession:** + +Now, wait for the next backup schedule. You can watch for `BackupSession` crd using the following command, ```bash -❯ kubectl get backupsession -n demo -w -NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE -cluster-resources-backup-1652164800 BackupConfiguration cluster-resources-backup 0s -cluster-resources-backup-1652164800 BackupConfiguration cluster-resources-backup Pending 0s -cluster-resources-backup-1652164800 BackupConfiguration cluster-resources-backup Running 0s -cluster-resources-backup-1652164800 BackupConfiguration cluster-resources-backup Running 69s -cluster-resources-backup-1652164800 BackupConfiguration cluster-resources-backup Succeeded 1m9s 69s +$ Every 1.0s: kubectl get backupsession -n demo -l=kubestash.com/invoker-name=kube-system-backup anisur: Fri Feb 23 18:34:53 2024 + +NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE +kube-system-backup-frequent-backup-1708691400 BackupConfiguration kube-system-backup Succeeded 4m53s + ``` -Here, the phase `Succeeded` means that the backup process has been completed successfully. +**Verify Backup:** -#### Verify Backup +When backup session is created, kubestash operator creates `Snapshot` which represents the state of backup run for each `Repository` which are provided in `BackupConfiguration`. -Now, we are going to verify whether the backed-up data is present in the backend or not. Once a backup is completed, Stash will update the respective `Repository` object to reflect the backup completion. Check that the repository `cluster-resource-storage` has been updated by the following command, +Run the following command to check `Snapshot` phase, ```bash -❯ kubectl get repository -n demo -NAME INTEGRITY SIZE SNAPSHOT-COUNT LAST-SUCCESSFUL-BACKUP AGE -cluster-resource-storage true 2.324 MiB 1 70s 54m +$ kubectl get snapshots -n demo +NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE AGE +gcs-repository-kube-system-backup-frequent-backup-1708691400 gcs-repository frequent-backup 2024-02-23T12:30:00Z Delete Succeeded 5m9s +``` + +When backup session is completed, KubeStash will update the respective `Repository` to reflect the latest state of backed up data. + +Run the following command to check if a backup snapshot has been stored in the backend, +```bash +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository true 1 2.262 KiB Ready 103s 8m ``` +From the output above, we can see that `1` snapshot has been stored in the backend specified by Repository `gcs-repository`. + Now, if we navigate to the GCS bucket, we will see the backed up data has been stored in `/manifest/cluster` directory as specified by `.spec.backend.gcs.prefix` field of the `Repository` object.
@@ -250,226 +363,78 @@ Now, if we navigate to the GCS bucket, we will see the backed up data has been s ## Restore -Stash does not provide any automatic mechanism to restore the cluster resources from the backed-up YAMLs. Your application might be managed by Helm or by an operator. In such cases, just applying the YAMLs is not enough to restore the application. Furthermore, there might be an order issue. Some resources must be applied before others. It is difficult to generalize and codify various application-specific logic. +KubeStash does not provide any automatic mechanism to restore the cluster resources from the backed-up YAMLs. Your application might be managed by Helm or by an operator. In such cases, just applying the YAMLs is not enough to restore the application. Furthermore, there might be an order issue. Some resources must be applied before others. It is difficult to generalize and codify various application-specific logic. Therefore, it is the user's responsibility to download the backed-up YAMLs and take the necessary steps based on his application to restore it properly. ### Download the YAMLs -Stash provides a [kubectl plugin](https://stash.run/docs/v2022.05.12/guides/cli/cli/#download-snapshots) for making it easy to download a snapshot locally. +KubeStash provides a [kubectl plugin](https://stash.run/docs/v2022.05.12/guides/cli/cli/#download-snapshots) for making it easy to download a snapshot locally. -Now, let's download the latest Snapshot from our backed-up data into the `$HOME/Downloads/stash` folder of our local machine. +Now, let's download the latest Snapshot from our backed-up data into the `$HOME/Downloads/kubestash` folder of our local machine. ```bash -❯ kubectl stash download -n demo cluster-resource-storage --destination=$HOME/Downloads/stash --snapshots="latest" +❯ kubectl kubestash download --namespace=demo gcs-repository-cluster-resourcesckup-frequent-backup-1708694700 --destination=$HOME/Downloads/kubestash/cluster ``` Now, lets use [tree](https://linux.die.net/man/1/tree) command to inspect downloaded YAMLs files. ```bash -❯ tree $HOME/Downloads/stash -/home/emruz/Downloads/stash -└── latest +❯ /home/anisur/Downloads/kubestash/cluster/gcs-repository-cluster-resourcesckup-frequent-backup-1708694700 +└── manifest └── tmp - └── resources + └── manifest ├── global - │ ├── APIService - │ │ ├── v1.admissionregistration.k8s.io.yaml - │ │ ├── v1alpha1.admission.stash.appscode.com.yaml - │ │ ├── v1alpha1.metrics.appscode.com.yaml - │ │ ├── v1alpha1.repositories.stash.appscode.com.yaml - │ ├── ClusterRole - │ │ ├── admin.yaml - │ │ ├── system:heapster.yaml - │ │ └── view.yaml - │ ├── ClusterRoleBinding - │ │ ├── appscode:license-reader-demo-cluster-resource-reader.yaml - │ │ ├── appscode:stash:garbage-collector.yaml - │ │ ├── cluster-admin.yaml - │ │ ├── cluster-resource-reader.yaml - │ │ ├── kindnet.yaml - │ ├── ComponentStatus - │ │ ├── controller-manager.yaml - │ │ ├── etcd-0.yaml - │ │ └── scheduler.yaml - │ ├── CSINode - │ │ └── kind-control-plane.yaml - │ ├── CustomResourceDefinition - │ │ ├── appbindings.appcatalog.appscode.com.yaml - │ │ ├── backupbatches.stash.appscode.com.yaml - │ │ ├── backupblueprints.stash.appscode.com.yaml - │ │ ├── backupconfigurations.stash.appscode.com.yaml - │ ├── FlowSchema - │ │ ├── catch-all.yaml - │ │ ├── exempt.yaml - │ │ ├── global-default.yaml - │ │ ├── kube-controller-manager.yaml - │ │ ├── kube-scheduler.yaml - │ ├── Function - │ │ ├── elasticsearch-backup-5.6.4.yaml - │ │ ├── mongodb-restore-5.0.3.yaml - │ │ ├── mysql-backup-5.7.25.yaml - │ │ └── update-status.yaml - │ ├── MetricsConfiguration - │ │ ├── stash-appscode-com-backupconfiguration.yaml - │ ├── MutatingWebhookConfiguration - │ │ └── admission.stash.appscode.com.yaml - │ ├── Namespace - │ │ ├── default.yaml - │ │ ├── demo.yaml - │ │ ├── kube-node-lease.yaml - │ │ ├── kube-public.yaml - │ │ ├── kube-system.yaml - │ │ └── local-path-storage.yaml - │ ├── Node - │ │ └── kind-control-plane.yaml - │ ├── StorageClass - │ │ └── standard.yaml - │ ├── Task - │ │ ├── elasticsearch-backup-5.6.4.yaml - │ │ ├── redis-restore-5.0.13.yaml - │ │ └── redis-restore-6.2.5.yaml - │ └── ValidatingWebhookConfiguration - │ └── admission.stash.appscode.com.yaml + │   ├── Addon + │   │   ├── kubedump-addon.yaml + │   │   ├── pvc-addon.yaml + │   │   └── workload-addon.yaml + │   ├── APIService + │   │   ├── v1.admissionregistration.k8s.io.yaml └── namespaces ├── default - │ ├── ConfigMap - │ │ └── kube-root-ca.crt.yaml - │ ├── Endpoints - │ │ └── kubernetes.yaml - │ ├── EndpointSlice - │ │ └── kubernetes.yaml - │ ├── Secret - │ │ └── default-token-7lpm6.yaml - │ ├── Service - │ │ └── kubernetes.yaml - │ └── ServiceAccount - │ └── default.yaml + │   ├── ConfigMap + │   │   └── kube-root-ca.crt.yaml + │   ├── Endpoints + │   │   └── kubernetes.yaml + │   ├── EndpointSlice + │   │   └── kubernetes.yaml + │   ├── Service + │   │   └── kubernetes.yaml + │   └── ServiceAccount + │   └── default.yaml ├── demo - │ ├── BackupConfiguration - │ │ └── cluster-resources-backup.yaml - │ ├── BackupSession - │ │ ├── cluster-resources-backup-1652176500.yaml - │ │ └── cluster-resources-backup-1652176800.yaml - │ ├── ConfigMap - │ │ └── kube-root-ca.crt.yaml - │ ├── CronJob - │ │ └── stash-trigger-cluster-resources-backup.yaml - │ ├── Event - │ │ ├── cluster-resources-backup-1652173200.16edb2d9e21556ba.yaml - │ ├── Job - │ │ ├── stash-backup-cluster-resources-backup-1652176500-0.yaml - │ │ └── stash-backup-cluster-resources-backup-1652176800-0.yaml - │ ├── Pod - │ │ ├── stash-backup-cluster-resources-backup-1652176500-0-vmx94.yaml - │ │ └── stash-backup-cluster-resources-backup-1652176800-0-5v4k9.yaml - │ ├── Repository - │ │ └── cluster-resource-storage.yaml - │ ├── RoleBinding - │ │ ├── backupconfiguration-cluster-resources-backup-0.yaml - │ │ └── stash-trigger-cluster-resources-backup.yaml - │ ├── Secret - │ │ ├── cluster-resource-reader-token-rqf6n.yaml - │ │ ├── default-token-rbrqt.yaml - │ │ └── gcs-secret.yaml - │ ├── ServiceAccount - │ │ ├── cluster-resource-reader.yaml - │ │ └── default.yaml - │ └── Snapshot - │ ├── cluster-resource-storage-5085e316.yaml - │ ├── cluster-resource-storage-6034bdcf.yaml - │ ├── cluster-resource-storage-706e8f46.yaml - ├── kube-node-lease - │ ├── ConfigMap - │ │ └── kube-root-ca.crt.yaml - │ ├── Lease - │ │ └── kind-control-plane.yaml - │ ├── Secret - │ │ └── default-token-69n9s.yaml - │ └── ServiceAccount - │ └── default.yaml - ├── kube-public - │ ├── ConfigMap - │ │ ├── cluster-info.yaml - │ │ └── kube-root-ca.crt.yaml - │ ├── Role - │ │ ├── kubeadm:bootstrap-signer-clusterinfo.yaml - │ │ └── system:controller:bootstrap-signer.yaml - │ ├── RoleBinding - │ │ ├── kubeadm:bootstrap-signer-clusterinfo.yaml - │ │ └── system:controller:bootstrap-signer.yaml - │ ├── Secret - │ │ └── default-token-jqv4n.yaml - │ └── ServiceAccount - │ └── default.yaml - ├── kube-system - │ ├── ConfigMap - │ │ ├── coredns.yaml - │ │ ├── extension-apiserver-authentication.yaml - │ │ ├── kubeadm-config.yaml - │ │ ├── kubelet-config-1.21.yaml - │ │ ├── kube-proxy.yaml - │ │ └── kube-root-ca.crt.yaml - │ ├── ControllerRevision - │ │ ├── kindnet-5b547684d9.yaml - │ │ └── kube-proxy-6bc6858f58.yaml - │ ├── DaemonSet - │ │ ├── kindnet.yaml - │ │ └── kube-proxy.yaml - │ ├── Deployment - │ │ ├── coredns.yaml - │ │ └── stash-stash-enterprise.yaml - │ ├── Endpoints - │ │ ├── kube-dns.yaml - │ │ └── stash-stash-enterprise.yaml - │ ├── EndpointSlice - │ │ ├── kube-dns-m2s5c.yaml - │ │ └── stash-stash-enterprise-k28h6.yaml - │ ├── Lease - │ │ ├── kube-controller-manager.yaml - │ │ └── kube-scheduler.yaml - │ ├── Pod - │ │ ├── coredns-558bd4d5db-hdsw9.yaml - │ │ ├── coredns-558bd4d5db-wk9tx.yaml - │ │ ├── etcd-kind-control-plane.yaml - │ │ └── stash-stash-enterprise-567dd95f5b-6xtxg.yaml - │ ├── ReplicaSet - │ │ ├── coredns-558bd4d5db.yaml - │ │ └── stash-stash-enterprise-567dd95f5b.yaml - │ ├── Role - │ │ ├── extension-apiserver-authentication-reader.yaml - │ │ ├── kubeadm:kubelet-config-1.21.yaml - │ │ └── system::leader-locking-kube-scheduler.yaml - │ ├── RoleBinding - │ │ ├── kubeadm:kubelet-config-1.21.yaml - │ │ ├── kubeadm:nodes-kubeadm-config.yaml - │ │ ├── kube-proxy.yaml - │ ├── Service - │ │ ├── kube-dns.yaml - │ │ └── stash-stash-enterprise.yaml - │ └── ServiceAccount - │ ├── attachdetach-controller.yaml - │ ├── bootstrap-signer.yaml + │   ├── BackupConfiguration + │   │   └── cluster-resources-backup.yaml + │   ├── BackupSession + │   │   └── cluster-resources-backup-frequent-backup-1708694700.yaml + │   ├── BackupStorage + │   │   └── gcs-storage.yaml + │   ├── ConfigMap + │   │   └── kube-root-ca.crt.yaml + │   ├── ReplicaSet + │   │   └── coredns-565d847f94.yaml + │   ├── Role + │   │   ├── extension-apiserver-authentication-reader.yaml + │   │   ├── kubeadm:kubelet-config.yaml + │   ├── RoleBinding + │   │   ├── kubeadm:kubelet-config.yaml + │   │   ├── kubeadm:nodes-kubeadm-config.yaml + │   ├── Service + │   │   └── kube-dns.yaml + │   └── ServiceAccount + │   ├── attachdetach-controller.yaml + │   ├── bootstrap-signer.yaml + │   ├── certificate-controller.yaml + │   ├── cluster-resource-reader.yaml └── local-path-storage ├── ConfigMap - │ ├── kube-root-ca.crt.yaml - │ └── local-path-config.yaml + │   ├── kube-root-ca.crt.yaml + │   └── local-path-config.yaml ├── Deployment - │ └── local-path-provisioner.yaml - ├── Endpoints - │ └── rancher.io-local-path.yaml - ├── Pod - │ └── local-path-provisioner-547f784dff-jb9tq.yaml - ├── ReplicaSet - │ └── local-path-provisioner-547f784dff.yaml - ├── Secret - │ ├── default-token-bnk6x.yaml - │ └── local-path-provisioner-service-account-token-fvkxj.yaml - └── ServiceAccount - ├── default.yaml - └── local-path-provisioner-service-account.yaml - -77 directories, 776 files + │   └── local-path-provisioner.yaml + +84 directories, 405 files ``` Here, the non-namespaced resources have been grouped under the `global` directory and the namespaced resources have been grouped inside the namespace specific folder under the `namespaces` directory. @@ -477,7 +442,7 @@ Here, the non-namespaced resources have been grouped under the `global` director Let's inspect the YAML of `kubeadm-config.yaml` file under `kube-system` namespace. ```yaml -❯ cat $HOME/Downloads/stash/latest/tmp/resources/namespaces/kube-system/ConfigMap/kubeadm-config.yaml +❯ cat /home/anisur/Downloads/kubestash/cluster/gcs-repository-cluster-resourcesckup-frequent-backup-1708694700/manifest/tmp/manifest/namespaces/kube-system/ConfigMap/kubeadm-config.yaml apiVersion: v1 data: ClusterConfiguration: | @@ -489,33 +454,25 @@ data: authorization-mode: Node,RBAC runtime-config: "" timeoutForControlPlane: 4m0s - apiVersion: kubeadm.k8s.io/v1beta2 + apiVersion: kubeadm.k8s.io/v1beta3 certificatesDir: /etc/kubernetes/pki clusterName: kind controlPlaneEndpoint: kind-control-plane:6443 controllerManager: extraArgs: enable-hostpath-provisioner: "true" - dns: - type: CoreDNS + dns: {} etcd: local: dataDir: /var/lib/etcd - imageRepository: k8s.gcr.io + imageRepository: registry.k8s.io kind: ClusterConfiguration - kubernetesVersion: v1.21.1 + kubernetesVersion: v1.25.2 networking: dnsDomain: cluster.local podSubnet: 10.244.0.0/16 serviceSubnet: 10.96.0.0/16 scheduler: {} - ClusterStatus: | - apiEndpoints: - kind-control-plane: - advertiseAddress: 172.18.0.2 - bindPort: 6443 - apiVersion: kubeadm.k8s.io/v1beta2 - kind: ClusterStatus kind: ConfigMap metadata: name: kubeadm-config @@ -530,8 +487,11 @@ To cleanup the Kubernetes resources created by this tutorial, run: ```bash kubectl delete -n demo backupconfiguration cluster-resources-backup -kubectl delete -n demo repository cluster-resource-storage -kubectl delete -n demo serviceaccount cluster-resource-reader -kubectl delete -n demo clusterrole cluster-resource-reader -kubectl delete -n demo clusterrolebinding cluster-resource-reader +kubectl delete -n kubestash serviceaccount cluster-resource-reader +kubectl delete clusterrole cluster-resource-reader +kubectl delete clusterrolebindin[index.md](..%2F..%2F..%2Fguides%2Fkubedump%2Fcluster%2Findex.md)g cluster-resource-reader +kubectl delete retentionPolicy -n demo demo-retention +kubectl delete -n demo backupstorage gcs-storage +kubectl delete secret -n demo encryption-secret +kubectl delete secret -n demo gcs-secret ``` diff --git a/docs/addons/kubedump/customization/examples/filter-resources.yaml b/docs/addons/kubedump/customization/examples/filter-resources.yaml index ba4cf9b..15b772e 100644 --- a/docs/addons/kubedump/customization/examples/filter-resources.yaml +++ b/docs/addons/kubedump/customization/examples/filter-resources.yaml @@ -1,21 +1,38 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: - name: cluster-resources-backup + name: kube-system-backup namespace: demo spec: - schedule: "*/5 * * * *" - task: - name: kubedump-backup-0.1.0 - params: - - name: labelSelector - value: "k8s-app=kube-dns" - repository: - name: cluster-resource-storage - runtimeSettings: - pod: - serviceAccountName: cluster-resource-reader - retentionPolicy: - name: keep-last-5 - keepLast: 5 - prune: true + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + params: + labelSelector: "k8s-app=kube-dns" + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader \ No newline at end of file diff --git a/docs/addons/kubedump/customization/examples/multi-retention-policy.yaml b/docs/addons/kubedump/customization/examples/multi-retention-policy.yaml deleted file mode 100644 index 08be02b..0000000 --- a/docs/addons/kubedump/customization/examples/multi-retention-policy.yaml +++ /dev/null @@ -1,22 +0,0 @@ -apiVersion: stash.appscode.com/v1beta1 -kind: BackupConfiguration -metadata: - name: cluster-resources-backup - namespace: demo -spec: - schedule: "*/5 * * * *" - task: - name: kubedump-backup-0.1.0 - repository: - name: cluster-resource-storage - runtimeSettings: - pod: - serviceAccountName: cluster-resource-reader - retentionPolicy: - name: cluster-resource-retention - keepLast: 5 - keepDaily: 10 - keepWeekly: 20 - keepMonthly: 50 - keepYearly: 100 - prune: true diff --git a/docs/addons/kubedump/customization/examples/passing-args.yaml b/docs/addons/kubedump/customization/examples/passing-args.yaml index 48e35d8..4f22e78 100644 --- a/docs/addons/kubedump/customization/examples/passing-args.yaml +++ b/docs/addons/kubedump/customization/examples/passing-args.yaml @@ -1,21 +1,43 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: - name: cluster-resources-backup + name: application-manifest-backup namespace: demo spec: - schedule: "*/5 * * * *" - task: - name: kubedump-backup-0.1.0 - params: - - name: sanitize - value: "false" - repository: - name: cluster-resource-storage - runtimeSettings: - pod: - serviceAccountName: cluster-resource-reader - retentionPolicy: - name: keep-last-5 - keepLast: 5 - prune: true + target: + apiGroup: apps + kind: Deployment + name: kubestash-kubestash-operator + namespace: kubestash + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /deployment-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + params: + sanitize: "false" + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader \ No newline at end of file diff --git a/docs/addons/kubedump/customization/examples/resource-limit.yaml b/docs/addons/kubedump/customization/examples/resource-limit.yaml index 6984a5e..c4a951e 100644 --- a/docs/addons/kubedump/customization/examples/resource-limit.yaml +++ b/docs/addons/kubedump/customization/examples/resource-limit.yaml @@ -1,26 +1,45 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: - name: cluster-resources-backup + name: kube-system-backup namespace: demo spec: - schedule: "*/5 * * * *" - task: - name: kubedump-backup-0.1.0 - repository: - name: cluster-resource-storage - runtimeSettings: - pod: - serviceAccountName: cluster-resource-reader - container: - resources: - requests: - cpu: "200m" - memory: "1Gi" - limits: - cpu: "200m" - memory: "1Gi" - retentionPolicy: - name: keep-last-5 - keepLast: 5 - prune: true + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/2 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + params: + labelSelector: "k8s-app=kube-dns" + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader + resources: + requests: + cpu: "200m" + memory: "1Gi" + limits: + cpu: "200m" + memory: "1Gi" diff --git a/docs/addons/kubedump/customization/examples/specific-user.yaml b/docs/addons/kubedump/customization/examples/specific-user.yaml index ed58cb9..7e58125 100644 --- a/docs/addons/kubedump/customization/examples/specific-user.yaml +++ b/docs/addons/kubedump/customization/examples/specific-user.yaml @@ -1,21 +1,41 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: - name: cluster-resources-backup + name: kube-system-backup namespace: demo spec: - schedule: "*/5 * * * *" - task: - name: kubedump-backup-0.1.0 - repository: - name: cluster-resource-storage - runtimeSettings: - pod: - serviceAccountName: cluster-resource-reader - securityContext: - runAsUser: 0 - runAsGroup: 0 - retentionPolicy: - name: keep-last-5 - keepLast: 5 - prune: true + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/2 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + params: + labelSelector: "k8s-app=kube-dns" + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader + securityContext: + runAsUser: 0 + runAsGroup: 0 \ No newline at end of file diff --git a/docs/addons/kubedump/customization/index.md b/docs/addons/kubedump/customization/index.md index 391cd6f..c2ebbac 100644 --- a/docs/addons/kubedump/customization/index.md +++ b/docs/addons/kubedump/customization/index.md @@ -14,9 +14,9 @@ section_menu_id: stash-addons # Customizing Backup Process -Stash provides rich customization supports for the backup and restores process to meet the requirements of various cluster configurations. This guide will show you some examples of these customizations. +KubeStash provides rich customization supports for the backup and restores process to meet the requirements of various cluster configurations. This guide will show you some examples of these customizations. -In this section, we are going to show you how to customize the backup process. Here, we are going to show some examples of filtering resources using a label selector, running the backup process as a specific user, using multiple retention policies, etc. +In this section, we are going to show you how to customize the backup process. Here, we are going to show some examples of filtering resources using a label selector, running the backup process as a specific user, etc. > Note: YAML files used in this tutorial are stored [here](https://github.com/kubestash/docs/tree/{{< param "info.version" >}}/docs/addons/kubedump/customization/examples). @@ -25,54 +25,72 @@ In this section, we are going to show you how to customize the backup process. H You can use a label selector to backup the YAML for the resources that have particular labels. You just have to pass the `labelSelector` parameter under `task.params` section with your desired label selector. ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: - name: cluster-resources-backup + name: kube-system-backup namespace: demo spec: - schedule: "*/5 * * * *" - task: - name: kubedump-backup-0.1.0 - params: - - name: labelSelector - value: "k8s-app=kube-dns" - repository: - name: cluster-resource-storage - runtimeSettings: - pod: - serviceAccountName: cluster-resource-reader - retentionPolicy: - name: keep-last-5 - keepLast: 5 - prune: true + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + params: + labelSelector: "k8s-app=kube-dns" + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader ``` The above backup process will backup only the resources that has `k8s-app: kube-dns` label. Here, is a sample of the resources backed up by the above BackupConfiguration. ```bash -❯ tree $HOME/Downloads/stash -/home/emruz/Downloads/stash -└── latest - └── tmp - └── resources - └── namespaces - └── kube-system - ├── Deployment - │ └── coredns.yaml - ├── Endpoints - │ └── kube-dns.yaml - ├── EndpointSlice - │ └── kube-dns-m2s5c.yaml - ├── Pod - │ ├── coredns-558bd4d5db-hdsw9.yaml - │ └── coredns-558bd4d5db-wk9tx.yaml - ├── ReplicaSet - │ └── coredns-558bd4d5db.yaml - └── Service - └── kube-dns.yaml - -11 directories, 7 files +❯ tree /home/anisur/Downloads/kubestash/label-selector +/home/anisur/Downloads/kubestash/label-selector +└── gcs-repository-kube-system-backup-frequent-backup-1708926900 + └── manifest + └── tmp + └── manifest + └── namespaces + └── kube-system + ├── Deployment + │   └── coredns.yaml + ├── Endpoints + │   └── kube-dns.yaml + ├── EndpointSlice + │   └── kube-dns-nv9px.yaml + ├── Pod + │   ├── coredns-565d847f94-78r86.yaml + │   └── coredns-565d847f94-zdtcs.yaml + ├── ReplicaSet + │   └── coredns-565d847f94.yaml + └── Service + └── kube-dns.yaml + +12 directories, 7 files ``` ### Passing arguments @@ -82,27 +100,49 @@ You can pass arguments to the backup process using `task.params` section. The following example shows how passes `sanitize` argument value to `false` which tells the backup process not to remove decorators (i.e. `status`, `managedFields` etc.) from the YAML files. ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: - name: cluster-resources-backup + name: application-manifest-backup namespace: demo spec: - schedule: "*/5 * * * *" - task: - name: kubedump-backup-0.1.0 - params: - - name: sanitize - value: "false" - repository: - name: cluster-resource-storage - runtimeSettings: - pod: - serviceAccountName: cluster-resource-reader - retentionPolicy: - name: keep-last-5 - keepLast: 5 - prune: true + target: + apiGroup: apps + kind: Deployment + name: kubestash-kubestash-operator + namespace: kubestash + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /deployment-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + params: + sanitize: "false" + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader ``` ### Running backup job as a specific user @@ -110,27 +150,45 @@ spec: If your cluster requires running the backup job as a specific user, you can provide `securityContext` under `runtimeSettings.pod` section. The below example shows how you can run the backup job as the root user. ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: - name: cluster-resources-backup + name: kube-system-backup namespace: demo spec: - schedule: "*/5 * * * *" - task: - name: kubedump-backup-0.1.0 - repository: - name: cluster-resource-storage - runtimeSettings: - pod: - serviceAccountName: cluster-resource-reader - securityContext: - runAsUser: 0 - runAsGroup: 0 - retentionPolicy: - name: keep-last-5 - keepLast: 5 - prune: true + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/2 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader + securityContext: + runAsUser: 0 + runAsGroup: 0 ``` ### Specifying Memory/CPU limit/request for the backup job @@ -138,61 +196,49 @@ spec: If you want to specify the Memory/CPU limit/request for your backup job, you can specify `resources` field under `runtimeSettings.container` section. ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: - name: cluster-resources-backup + name: kube-system-backup namespace: demo spec: - schedule: "*/5 * * * *" - task: - name: kubedump-backup-0.1.0 - repository: - name: cluster-resource-storage - runtimeSettings: - pod: - serviceAccountName: cluster-resource-reader - container: - resources: - requests: - cpu: "200m" - memory: "1Gi" - limits: - cpu: "200m" - memory: "1Gi" - retentionPolicy: - name: keep-last-5 - keepLast: 5 - prune: true -``` - -### Using multiple retention policies - -You can also specify multiple retention policies for your backed up data. For example, you may want to keep a few daily snapshots, a few weekly snapshots, and few monthly snapshots, etc. You just need to pass the desired number with the respective key under the `retentionPolicy` section. - -```yaml -apiVersion: stash.appscode.com/v1beta1 -kind: BackupConfiguration -metadata: - name: cluster-resources-backup - namespace: demo -spec: - schedule: "*/5 * * * *" - task: - name: kubedump-backup-0.1.0 - repository: - name: cluster-resource-storage - runtimeSettings: - pod: - serviceAccountName: cluster-resource-reader - retentionPolicy: - name: cluster-resource-retention - keepLast: 5 - keepDaily: 10 - keepWeekly: 20 - keepMonthly: 50 - keepYearly: 100 - prune: true -``` - -To know more about the available options for retention policies, please visit [here](/docs/concepts/crds/backupconfiguration/index.md#specretentionpolicy). + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/2 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + params: + labelSelector: "k8s-app=kube-dns" + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader + resources: + requests: + cpu: "200m" + memory: "1Gi" + limits: + cpu: "200m" + memory: "1Gi" +``` \ No newline at end of file diff --git a/docs/addons/kubedump/namespace/examples/backupconfiguration.yaml b/docs/addons/kubedump/namespace/examples/backupconfiguration.yaml index f60a9d1..612a101 100644 --- a/docs/addons/kubedump/namespace/examples/backupconfiguration.yaml +++ b/docs/addons/kubedump/namespace/examples/backupconfiguration.yaml @@ -1,23 +1,40 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: name: kube-system-backup namespace: demo spec: - schedule: "*/5 * * * *" - task: - name: kubedump-backup-0.1.0 - repository: - name: namespace-resource-storage target: - ref: - apiVersion: v1 - kind: Namespace - name: kube-system - runtimeSettings: - pod: - serviceAccountName: cluster-resource-reader - retentionPolicy: - name: keep-last-5 - keepLast: 5 - prune: true + apiGroup: "" + kind: Namespace + name: kube-system + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader \ No newline at end of file diff --git a/docs/addons/kubedump/namespace/examples/backupstorage.yaml b/docs/addons/kubedump/namespace/examples/backupstorage.yaml new file mode 100644 index 0000000..0461b26 --- /dev/null +++ b/docs/addons/kubedump/namespace/examples/backupstorage.yaml @@ -0,0 +1,17 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/addons/kubedump/namespace/examples/repository.yaml b/docs/addons/kubedump/namespace/examples/repository.yaml deleted file mode 100644 index 3c703b0..0000000 --- a/docs/addons/kubedump/namespace/examples/repository.yaml +++ /dev/null @@ -1,11 +0,0 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: namespace-resource-storage - namespace: demo -spec: - backend: - gcs: - bucket: stash-testing - prefix: /manifests/namespace/kube-system - storageSecretName: gcs-secret diff --git a/docs/addons/kubedump/namespace/examples/retentionpolicy.yaml b/docs/addons/kubedump/namespace/examples/retentionpolicy.yaml new file mode 100644 index 0000000..3af9037 --- /dev/null +++ b/docs/addons/kubedump/namespace/examples/retentionpolicy.yaml @@ -0,0 +1,15 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same \ No newline at end of file diff --git a/docs/addons/kubedump/namespace/images/namespace_manifests_backup.png b/docs/addons/kubedump/namespace/images/namespace_manifests_backup.png index 5d2d405..5777f78 100644 Binary files a/docs/addons/kubedump/namespace/images/namespace_manifests_backup.png and b/docs/addons/kubedump/namespace/images/namespace_manifests_backup.png differ diff --git a/docs/addons/kubedump/namespace/index.md b/docs/addons/kubedump/namespace/index.md index 1bb4e45..2ae0a92 100644 --- a/docs/addons/kubedump/namespace/index.md +++ b/docs/addons/kubedump/namespace/index.md @@ -12,24 +12,25 @@ menu_name: docs_{{ .version }} section_menu_id: stash-addons --- -# Backup resource YAMLs of a Namespace using Stash +# Backup resource YAMLs of a Namespace using KubeStash -Stash `{{< param "info.version" >}}` supports taking backup of the resource YAMLs using `kubedump` plugin. This guide will show you how you can take a backup of the resource YAMLs of a particular namespace using Stash. +KubeStash `{{< param "info.version" >}}` supports taking backup of the resource YAMLs using `kubedump` plugin. This guide will show you how you can take a backup of the resource YAMLs of a particular namespace using KubeStash. ## Before You Begin - At first, you need to have a Kubernetes cluster, and the `kubectl` command-line tool must be configured to communicate with your cluster. -- Install Stash in your cluster following the steps [here](/docs/setup/install/stash/index.md). -- Install Stash `kubectl` plugin in your local machine following the steps [here](/docs/setup/install/kubectl-plugin/index.md). -- If you are not familiar with how Stash backup the resource YAMLs, please check the following guide [here](/docs/addons/kubedump/overview/index.md). +- Install KubeStash in your cluster following the steps [here](/docs/setup/install/kubestash/index.md). +- Install KubeStash `kubectl` plugin in your local machine following the steps [here](/docs/setup/install/kubectl-plugin/index.md). +- If you are not familiar with how KubeStash backup the resource YAMLs, please check the following guide [here](/docs/guides/kubedump/overview/index.md). You have to be familiar with the following custom resources: -- [AppBinding](/docs/concepts/crds/appbinding/index.md) -- [Function](/docs/concepts/crds/function/index.md) -- [Task](/docs/concepts/crds/task/index.md) +- [BackupStorage](/docs/concepts/crds/backupstorage/index.md) - [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md) - [BackupSession](/docs/concepts/crds/backupsession/index.md) +- [SnapShot](/docs/concepts/crds/snapshot/index.md) +- [RestoreSession](/docs/concepts/crds/restoresession/index.md) +- [RetentionPolicy](/docs/concepts/crds/retentionpolicy/index.md) To keep things isolated, we are going to use a separate namespace called `demo` throughout this tutorial. Create the `demo` namespace if you haven't created it already. @@ -46,55 +47,125 @@ In this section, we are going to configure a backup for all the resource YAMLs o #### Ensure `kubedump` Addon -When you install the Stash, it will automatically install all the official addons. Make sure that `kubedump` addon was installed properly using the following command. +**Verify necessary Addons and Functions:** + +When you install KubeStash, it automatically creates the necessary `Addon` and `Function` to backup Kubernetes Resource YAML. + +Let's verify that KubeStash has created the necessary `Function` to backup Kubernetes Resource YAML by the following command, + +```bash +$ kubectl get functions | grep 'kubedump' +kubedump-backup 142m +``` + +Also, verify that the necessary `Addon` has been created, ```bash -❯ kubectl get tasks.stash.appscode.com | grep kubedump -kubedump-backup-0.1.0 23s +❯ kubectl get addons | grep 'kubedump' +kubedump-addon 143m +``` + +Now, you can view all `BackupTasks` and `RestoreTasks` of the `pvc-addon` addon using the following command + +```bash +❯ kubectl get addon kubedump-addon -o=jsonpath='{.spec.backupTasks[*].name}'| tr ' ' '\n' +manifest-backup ``` #### Prepare Backend -We are going to store our backed-up data into a GCS bucket. So, we need to create a Secret with GCS credentials and a `Repository` object with the bucket information. If you want to use a different backend, please read the respective backend configuration doc from [here](/docs/guides/backends/overview/index.md). +Now, we are going to store our backed-up data into a GCS bucket using KubeStash. We have to create a Secret and a `BackupStorage` object with access credentials and backend information respectively. If you want to use a different backend, please read the respective backend configuration doc from [here](/docs/guides/backends/overview/index.md). + +> For GCS backend, if the bucket does not exist, KubeStash needs `Storage Object Admin` role permissions to create the bucket. For more details, please check the following [guide](/docs/guides/backends/gcs/index.md). **Create Storage Secret:** At first, let's create a secret called `gcs-secret` with access credentials to our desired GCS bucket, ```bash -$ echo -n 'changeit' > RESTIC_PASSWORD $ echo -n '' > GOOGLE_PROJECT_ID -$ cat downloaded-sa-json.key > GOOGLE_SERVICE_ACCOUNT_JSON_KEY +$ cat /path/to/downloaded/sa_key_file.json > GOOGLE_SERVICE_ACCOUNT_JSON_KEY $ kubectl create secret generic -n demo gcs-secret \ - --from-file=./RESTIC_PASSWORD \ --from-file=./GOOGLE_PROJECT_ID \ --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY secret/gcs-secret created ``` -**Create Repository:** +**Create BackupStorage:** -Now, crete a `Repository` object with the information of your desired bucket. Below is the YAML of `Repository` object we are going to create, +Now, create a `BackupStorage` custom resource specifying the desired bucket, and directory inside the bucket where the backed up data will be stored. + +Below is the YAML of `BackupStorage` object that we are going to create, ```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: namespace-resource-storage + name: gcs-storage namespace: demo spec: - backend: + storage: + provider: gcs gcs: - bucket: stash-testing - prefix: /manifests/namespace/kube-system - storageSecretName: gcs-secret + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut +``` + +Let's create the `BackupStorage` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/kubedump/application/examples/backupstorage.yaml +backupstorage.storage.kubestash.com/gcs-repo created ``` -Let's create the `Repository` we have shown above, +Now, we also have to create another secret with an encryption key `RESTIC_PASSWORD` for Restic. This secret will be used by Restic for both encrypting and decrypting the backup data during backup & restore. + +**Create Restic Repository Secret:** + +Let's create a Secret named `encry-secret` with an encryption key `RESTIC_PASSWORD` for Restic. ```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/addons/kubedump/namespace/examples/repository.yaml -repository.stash.appscode.com/namespace-resource-storage created +$ echo -n 'changeit' > RESTIC_PASSWORD +$ kubectl create secret generic -n demo encry-secret \ + --from-file=./RESTIC_PASSWORD +secret/encryption-secret created +``` + +Now, we have to create a `RetentionPolicy` custom resource which specifies how the old `Snapshots` should be cleaned up. + +**Create RetentionPolicy:** + +Below is the YAML of the `RetentionPolicy` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same +``` + +Let's create the `RetentionPolicy` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/kubedump/application/examples/retentionpolicy.yaml +retentionpolicy.storage.kubestash.com/demo-retention created ``` #### Create RBAC @@ -155,39 +226,58 @@ To schedule a backup, we have to create a `BackupConfiguration` object. Then Sta Below is the YAML for `BackupConfiguration` object we care going to use to backup the YAMLs of the cluster resources, ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: name: kube-system-backup namespace: demo spec: - schedule: "*/5 * * * *" - task: - name: kubedump-backup-0.1.0 - repository: - name: namespace-resource-storage target: - ref: - apiVersion: v1 - kind: Namespace - name: kube-system - runtimeSettings: - pod: - serviceAccountName: cluster-resource-reader - retentionPolicy: - name: keep-last-5 - keepLast: 5 - prune: true + apiGroup: "" + kind: Namespace + name: kube-system + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader ``` Here, -- `.spec.schedule` specifies that we want to backup the cluster resources at 5 minutes intervals. -- `.spec.task.name` specifies the name of the Task object that specifies the necessary Functions and their execution order to backup the resource YAMLs. -- `.spec.repository.name` specifies the Repository CR name we have created earlier with backend information. -- `.spec.target` specifies the targeted Namespace that we are going to backup. -- `.spec.runtimeSettings.pod.serviceAccountName` specifies the ServiceAccount name that we have created earlier with cluster-wide resource reading permission. -- `.spec.retentionPolicy` specifies a policy indicating how we want to cleanup the old backups. +- `spec.target` specifies the backup target. `apiGroup`, `kind`, `namespace` and `name` refers to the `apiGroup`, `kind`, `namespace` and `name` of the targeted workload respectively. +- `spec.sessions[*].scheduler.schedule` specifies a [cron expression](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) indicates that `BackupSession` will be created at 5 minute interval. +- `spec.sessions[*].repositories[*].name` specifies the name of the `Repository` object that holds the backend information. +- `spec.sessions[*].repositories[*].backend` specifies the name of the backend that holds the `BackupStorage` information. +- `spec.sessions[*].repositories[*].directory` specifies the path of the `Repository` where the backed up data will be stored. +- `spec.sessions[*].repositories[*].encryptionSecret` specifies the encryption secret for `Restic Repository` which will be used to encrypting the backup data. +- `spec.sessions[*].addon.name` specifies the name of the `Addon` object that specifies addon configuration that will be used to perform backup of a stand-alone PVC. +- `spec.sessions[*].addon.tasks[*].name` specifies the name of the `Task` that holds the `Function` and their order of execution to perform backup of a stand-alone PVC. Let's create the `BackupConfiguration` object we have shown above, @@ -202,50 +292,67 @@ If everything goes well, the phase of the `BackupConfiguration` should be `Ready ```bash ❯ kubectl get backupconfiguration -n demo -NAME TASK SCHEDULE PAUSED PHASE AGE -kube-system-backup kubedump-backup-0.1.0 */5 * * * * Ready 8s -``` +NAME PHASE PAUSED AGE +kube-system-backup Ready 79s -#### Verify CronJob +``` -Stash will create a CronJob with the schedule specified in `spec.schedule` field of `BackupConfiguration` object. +**Verify Repository:** -Verify that the CronJob has been created using the following command, +Verify that KubeStash has created `Repositories` that holds the `BackupStorage` information by the following command, ```bash -❯ kubectl get cronjob -n demo -NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE -stash-trigger-kube-system-backup */5 * * * * False 0 25s +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository Ready 28s ``` -#### Wait for BackupSession +**Verify CronJob:** -The `stash-trigger-kube-system-backup` CronJob will trigger a backup on each scheduled slot by creating a `BackupSession` object. +Verify that KubeStash has created a CronJob to trigger a periodic backup of the targeted PVC by the following command, -Now, wait for a schedule to appear. Run the following command to watch for a `BackupSession` object, +```bash +$ kubectl get cronjob -n demo +NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE +trigger-kube-system-backup-frequent-backup */5 * * * * False 0 45s +``` + +**Wait for BackupSession:** + +Now, wait for the next backup schedule. You can watch for `BackupSession` crd using the following command, ```bash -❯ kubectl get backupsession -n demo -w -NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE -kube-system-backup-1652247300 BackupConfiguration kube-system-backup 0s -kube-system-backup-1652247300 BackupConfiguration kube-system-backup Pending 0s -kube-system-backup-1652247300 BackupConfiguration kube-system-backup Running 0s -kube-system-backup-1652247300 BackupConfiguration kube-system-backup Running 17s -kube-system-backup-1652247300 BackupConfiguration kube-system-backup Succeeded 1m11s 71s +$ Every 1.0s: kubectl get backupsession -n demo -l=kubestash.com/invoker-name=kube-system-backup anisur: Fri Feb 23 18:34:53 2024 + +NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE +kube-system-backup-frequent-backup-1708691400 BackupConfiguration kube-system-backup Succeeded 4m53s + ``` -Here, the phase `Succeeded` means that the backup process has been completed successfully. +**Verify Backup:** -#### Verify Backup +When backup session is created, kubestash operator creates `Snapshot` which represents the state of backup run for each `Repository` which are provided in `BackupConfiguration`. -Now, we are going to verify whether the backed-up data is present in the backend or not. Once a backup is completed, Stash will update the respective `Repository` object to reflect the backup completion. Check that the repository `namespace-resource-storage` has been updated by the following command, +Run the following command to check `Snapshot` phase, ```bash -❯ kubectl get repository -n demo -NAME INTEGRITY SIZE SNAPSHOT-COUNT LAST-SUCCESSFUL-BACKUP AGE -namespace-resource-storage true 407.949 KiB 1 95s 13m +$ kubectl get snapshots -n demo +NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE AGE +gcs-repository-kube-system-backup-frequent-backup-1708691400 gcs-repository frequent-backup 2024-02-23T12:30:00Z Delete Succeeded 5m9s ``` +When backup session is completed, KubeStash will update the respective `Repository` to reflect the latest state of backed up data. + +Run the following command to check if a backup snapshot has been stored in the backend, + +```bash +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository true 1 2.262 KiB Ready 103s 8m +``` + +From the output above, we can see that `1` snapshot has been stored in the backend specified by Repository `gcs-repository`. + Now, if we navigate to the GCS bucket, we will see the backed up data has been stored in `/manifest/namespace/kube-system` directory as specified by `.spec.backend.gcs.prefix` field of the `Repository` object.
@@ -253,99 +360,81 @@ Now, if we navigate to the GCS bucket, we will see the backed up data has been s
Fig: Backup data in GCS Bucket
-> Note: Stash keeps all the backed-up data encrypted. So, data in the backend will not make any sense until they are decrypted. +> Note: KubeStash keeps all the backed-up data encrypted. So, data in the backend will not make any sense until they are decrypted. ## Restore -Stash does not provide any automatic mechanism to restore the cluster resources from the backed-up YAMLs. Your application might be managed by Helm or by an operator. In such cases, just applying the YAMLs is not enough to restore the application. Furthermore, there might be an order issue. Some resources must be applied before others. It is difficult to generalize and codify various application-specific logic. +KubeStash does not provide any automatic mechanism to restore the cluster resources from the backed-up YAMLs. Your application might be managed by Helm or by an operator. In such cases, just applying the YAMLs is not enough to restore the application. Furthermore, there might be an order issue. Some resources must be applied before others. It is difficult to generalize and codify various application-specific logic. Therefore, it is the user's responsibility to download the backed-up YAMLs and take the necessary steps based on his application to restore it properly. ### Download the YAMLs -Stash provides a [kubectl plugin](https://stash.run/docs/v2022.05.12/guides/cli/cli/#download-snapshots) for making it easy to download a snapshot locally. +KubeStash provides a [kubectl plugin](https://stash.run/docs/v2022.05.12/guides/cli/cli/#download-snapshots) for making it easy to download a snapshot locally. -Now, let's download the latest Snapshot from our backed-up data into the `$HOME/Downloads/stash/namespace/kube-system` folder of our local machine. +Now, let's download the latest Snapshot from our backed-up data into the `$HOME/Downloads/kubestash/namespace/kube-system` folder of our local machine. ```bash -❯ kubectl stash download -n demo namespace-resource-storage --destination=$HOME/Downloads/stash/namespace/kube-system --snapshots="latest" +❯ kubectl kubestash download --namespace=demo gcs-repository-kube-system-backup-frequent-backup-1708691400 --destination=$HOME/Downloads/kubestash/namespace/kube-system/ ``` Now, lets use [tree](https://linux.die.net/man/1/tree) command to inspect downloaded YAMLs files. ```bash -❯ tree $HOME/Downloads/stash/namespace/kube-system -/home/emruz/Downloads/stash/namespace/kube-system -└── latest - └── tmp - └── resources - ├── ConfigMap - │   ├── coredns.yaml - │   ├── extension-apiserver-authentication.yaml - │   ├── kubeadm-config.yaml - │   ├── kubelet-config-1.21.yaml - │   ├── kube-proxy.yaml - │   └── kube-root-ca.crt.yaml - ├── ControllerRevision - │   ├── kindnet-5b547684d9.yaml - │   └── kube-proxy-6bc6858f58.yaml - ├── DaemonSet - │   ├── kindnet.yaml - │   └── kube-proxy.yaml - ├── Deployment - │   ├── coredns.yaml - │   └── stash-stash-enterprise.yaml - ├── Endpoints - │   ├── kube-dns.yaml - │   └── stash-stash-enterprise.yaml - ├── EndpointSlice - │   ├── kube-dns-m2s5c.yaml - │   └── stash-stash-enterprise-k28h6.yaml - ├── Lease - │   ├── kube-controller-manager.yaml - │   └── kube-scheduler.yaml - ├── Pod - │   ├── coredns-558bd4d5db-hdsw9.yaml - │   ├── coredns-558bd4d5db-wk9tx.yaml - │   ├── etcd-kind-control-plane.yaml - │   ├── kindnet-69whw.yaml - │   ├── kube-apiserver-kind-control-plane.yaml - │   ├── kube-controller-manager-kind-control-plane.yaml - │   ├── kube-proxy-p7j9f.yaml - │   ├── kube-scheduler-kind-control-plane.yaml - │   └── stash-stash-enterprise-567dd95f5b-6xtxg.yaml - ├── ReplicaSet - │   ├── coredns-558bd4d5db.yaml - │   └── stash-stash-enterprise-567dd95f5b.yaml - ├── Role - │   ├── extension-apiserver-authentication-reader.yaml - │   ├── kubeadm:kubelet-config-1.21.yaml - │   ├── kubeadm:nodes-kubeadm-config.yaml - │   ├── kube-proxy.yaml - ├── RoleBinding - │   ├── kubeadm:kubelet-config-1.21.yaml - │   ├── kubeadm:nodes-kubeadm-config.yaml - │   ├── kube-proxy.yaml - ├── Secret - │   ├── attachdetach-controller-token-w68zv.yaml - │   ├── bootstrap-signer-token-j6q2c.yaml - │   ├── certificate-controller-token-d5dvw.yaml - │   ├── clusterrole-aggregation-controller-token-77x8n.yaml - ├── Service - │   ├── kube-dns.yaml - │   └── stash-stash-enterprise.yaml - └── ServiceAccount - ├── attachdetach-controller.yaml - ├── bootstrap-signer.yaml - ├── certificate-controller.yaml - ├── clusterrole-aggregation-controller.yaml - ├── coredns.yaml - ├── cronjob-controller.yaml - ├── daemon-set-controller.yaml - ├── default.yaml - ├── deployment-controller.yaml - -17 directories, 131 files +❯ tree /home/anisur/Downloads/kubestash/namespace/kube-system +/home/anisur/Downloads/kubestash/namespace/kube-system +└── gcs-repository-kube-system-backup-frequent-backup-1708691400 + └── manifest + └── tmp + └── manifest + ├── ConfigMap + │   ├── coredns.yaml + │   ├── extension-apiserver-authentication.yaml + │   ├── kubeadm-config.yaml + ├── ControllerRevision + │   ├── kindnet-685dcc8c4c.yaml + │   └── kube-proxy-7ccf78d585.yaml + ├── DaemonSet + │   ├── kindnet.yaml + │   └── kube-proxy.yaml + ├── Deployment + │   └── coredns.yaml + ├── Endpoints + │   └── kube-dns.yaml + ├── EndpointSlice + │   └── kube-dns-nv9px.yaml + ├── Lease + │   ├── kube-controller-manager.yaml + │   └── kube-scheduler.yaml + ├── Pod + │   ├── coredns-565d847f94-78r86.yaml + │   ├── coredns-565d847f94-zdtcs.yaml + │   ├── etcd-kind-control-plane.yaml + │   ├── kindnet-qb7b8.yaml + ├── ReplicaSet + │   └── coredns-565d847f94.yaml + ├── Role + │   ├── extension-apiserver-authentication-reader.yaml + │   ├── kubeadm:kubelet-config.yaml + │   ├── kubeadm:nodes-kubeadm-config.yaml + │   ├── kube-proxy.yaml + ├── RoleBinding + │   ├── kubeadm:kubelet-config.yaml + │   ├── kubeadm:nodes-kubeadm-config.yaml + │   ├── kube-proxy.yaml + │   ├── system:controller:bootstrap-signer.yaml + │   ├── system:controller:cloud-provider.yaml + ├── Service + │   └── kube-dns.yaml + └── ServiceAccount + ├── attachdetach-controller.yaml + ├── bootstrap-signer.yaml + ├── certificate-controller.yaml + ├── cluster-resource-reader.yaml + ├── clusterrole-aggregation-controller.yaml + ├── coredns.yaml + +17 directories, 80 files ``` Here, the resources has been grouped under the respective Kind folder. @@ -353,7 +442,7 @@ Here, the resources has been grouped under the respective Kind folder. Let's inspect the YAML of `coredns.yaml` file under ConfigMap folder, ```yaml -❯ cat $HOME/Downloads/stash/namespace/kube-system/latest/tmp/resources/ConfigMap/coredns.yaml +❯ cat /home/anisur/Downloads/kubestash/namespace/kube-system/gcs-repository-kube-system-backup-frequent-backup-1708691400/manifest/tmp/manifest/ConfigMap/coredns.yaml apiVersion: v1 data: Corefile: | diff --git a/docs/addons/kubedump/overview/images/kubedump-backup.svg b/docs/addons/kubedump/overview/images/kubedump-backup.svg deleted file mode 100644 index 2eac1ce..0000000 --- a/docs/addons/kubedump/overview/images/kubedump-backup.svg +++ /dev/null @@ -1,914 +0,0 @@ - - - - - - image/svg+xml - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/docs/addons/kubedump/overview/index.md b/docs/addons/kubedump/overview/index.md deleted file mode 100644 index 7030d3a..0000000 --- a/docs/addons/kubedump/overview/index.md +++ /dev/null @@ -1,56 +0,0 @@ ---- -title: KubeDump Backup Overview | Stash -description: How KubeDump Backup Works in Stash -menu: - docs_{{ .version }}: - identifier: stash-kubedump-overview - name: How does it work? - parent: stash-kubedump - weight: 10 -product_name: kubestash -menu_name: docs_{{ .version }} -section_menu_id: stash-addons ---- - -# How Stash Backups Kubernetes Resources - -Stash `{{< param "info.version" >}}` supports taking backup of Kubernetes resource YAMLs. You can backup the YAML definition of the resources of entire cluster, a particular namespaces, or only an application etc. In this guide, we are going to show you how Kubernetes resource backup works in Stash. - -## How Backup Works - -The following diagram shows how Stash takes backup of the Kubernetes resources. Open the image in a new tab to see the enlarged version. - -
-  KubeDump Backup Overview -
Fig: KubeDump Backup Overview
-
- -The backup process consists of the following steps: - -1. At first, a user creates a secret with access credentials of the backend where the backed up data will be stored. - -2. Then, she creates a `Repository` crd that specifies the backend information along with the secret that holds the credentials to access the backend. - -3. Then, she creates a `BackupConfiguration` crd which specifies the `Task` to use to backup the the resources. - -4. Stash operator watches for `BackupConfiguration` crd. - -5. Once Stash operator finds a `BackupConfiguration` crd, it creates a CronJob with the schedule specified in `BackupConfiguration` object to trigger backup periodically. - -6. On the next scheduled slot, the CronJob triggers a backup by creating a `BackupSession` crd. - -7. Stash operator also watches for `BackupSession` crd. - -8. When it finds a `BackupSession` object, it resolves the respective `Task` and `Function` and prepares a Job definition to backup. - -9. Then, it creates the Job to backup the desired resource YAMLs. - -10. Then, the Job dumps the Kubernetes Resource YAML, and store them temporarily in a directory.Then, it upload the content of the directory to the cloud backend. - -11. Finally, when the backup is complete, the Job sends Prometheus metrics to the Pushgateway running inside Stash operator pod. It also updates the `BackupSession` and `Repository` status to reflect the backup procedure. - -## Next Steps - -- Backup the YAMLs of your entire Kubernetes cluster using Stash following the guide from [here](/docs/addons/kubedump/cluster/index.md). -- Backup the YAMLs of an entire Namespace using Stash following the guide from [here](/docs/addons/kubedump/namespace/index.md). -- Backup the YAMLs of a particular application using Stash following the guide from [here](/docs/addons/kubedump/application/index.md). diff --git a/docs/guides/backends/azure/examples/azure.yaml b/docs/guides/backends/azure/examples/azure.yaml index ef092cf..798badc 100644 --- a/docs/guides/backends/azure/examples/azure.yaml +++ b/docs/guides/backends/azure/examples/azure.yaml @@ -1,11 +1,18 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: azure-repo + name: azure-storage namespace: demo spec: - backend: + storage: + provider: azure azure: - container: stash-backup - prefix: /demo/deployment/my-deploy - storageSecretName: azure-secret + storageAccount: kubestash + container: kubestash-demo + prefix: /backup/demo/deployment/kubestash-demo + secretName: azure-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/backends/azure/index.md b/docs/guides/backends/azure/index.md index 91544b0..d5c5179 100644 --- a/docs/guides/backends/azure/index.md +++ b/docs/guides/backends/azure/index.md @@ -1,6 +1,6 @@ --- -title: Azure Backend | Stash -description: Configure Stash to use Microsoft Azure Storage as Backend. +title: Azure Backend | KubeStash +description: Configure KubeStash to use Microsoft Azure Storage as Backend. menu: docs_{{ .version }}: identifier: backend-azure @@ -14,9 +14,9 @@ section_menu_id: guides # Microsoft Azure Storage -Stash supports Microsoft's [Azure Blob Storage](https://azure.microsoft.com/en-us/services/storage/blobs/) as a backend. This tutorial will show you how to use this backend. +KubeStash supports Microsoft's [Azure Blob Storage](https://azure.microsoft.com/en-us/services/storage/blobs/) as a backend. This tutorial will show you how to use this backend. -In order to use Azure Blob Storage as backend, you have to create a `Secret` and a `Repository` object pointing to the desired blob container. +In order to use Azure Blob Storage as backend, you have to create a `Secret` and a `BackupStorage` object pointing to the desired blob container. #### Create Storage Secret @@ -24,62 +24,65 @@ To configure storage secret for this backend, following secret keys are needed: | Key | Type | Description | | -------------------- | ---------- | ---------------------------------------------------------- | -| `RESTIC_PASSWORD` | `Required` | Password that will be used to encrypt the backup snapshots | -| `AZURE_ACCOUNT_NAME` | `Required` | Azure Storage account name | | `AZURE_ACCOUNT_KEY` | `Required` | Azure Storage account key | Create storage secret as below, ```bash -$ echo -n 'changeit' > RESTIC_PASSWORD -$ echo -n '' > AZURE_ACCOUNT_NAME $ echo -n '' > AZURE_ACCOUNT_KEY $ kubectl create secret generic -n demo azure-secret \ - --from-file=./RESTIC_PASSWORD \ - --from-file=./AZURE_ACCOUNT_NAME \ --from-file=./AZURE_ACCOUNT_KEY secret/azure-secret created ``` -### Create Repository +### Create BackupStorage -Now, you have to create a `Repository` crd. You have to provide the storage secret that we have created earlier in `spec.backend.storageSecretName` field. +Now, you have to create a `BackupStorage` crd. You have to provide the storage secret that we have created earlier in `spec.storage.azure.secret` field. Following parameters are available for `azure` backend. -| Parameter | Type | Description | -| ---------------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------- | -| `azure.container` | `Required` | Name of Storage container. | -| `azure.prefix` | `Optional` | Path prefix inside the container where backed up data will be stored. | -| `azure.maxConnections` | `Optional` | Maximum number of parallel connections to use for uploading backup data. By default, Stash will use maximum 5 parallel connections. | +| Parameter | Type | Description | +|------------------------| ---------- |-----------------------------------------------------------------------------------------------------------------------------------------| +| `azure.storageAccount` | `Required` | Name of the Storage account. | +| `azure.container` | `Required` | Name of Storage container. | +| `azure.secret` | `Required` | Specify the name of the Secret that contains the access credential for this storage. | | +| `azure.prefix` | `Optional` | Path prefix inside the container where backed up data will be stored. | +| `azure.maxConnections` | `Optional` | Maximum number of parallel connections to use for uploading backup data. By default, KubeStash will use maximum 5 parallel connections. | -Below, the YAML of a sample `Repository` crd that uses an Azure Blob container as a backend. +Below, the YAML of a sample `BackupStorage` crd that uses an Azure Blob container as a backend. ```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: azure-repo + name: azure-storage namespace: demo spec: - backend: + storage: + provider: azure azure: - container: stash-backup - prefix: /demo/deployment/my-deploy - storageSecretName: azure-secret + storageAccount: kubestash + container: kubestash-demo + prefix: /backup/demo/deployment/kubestash-demo + secretName: azure-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut ``` -Create the `Repository` we have shown above using the following command, +Create the `BackupStorage` we have shown above using the following command, ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/azure/examples/azure.yaml -repository/azure-repo created +backupstorage.storage.kubestash.com/azure-storaqge created ``` -Now, we are ready to use this backend to backup our desired data using Stash. +Now, we are ready to use this backend to backup our desired data using KubeStash. ## Next Steps -- Learn how to use Stash to backup workloads data from [here](/docs/guides/workloads/overview/index.md). -- Learn how to use Stash to backup databases from [here](/docs/guides/addons/overview/index.md). -- Learn how to use Stash to backup stand-alone PVC from [here](/docs/guides/volumes/overview/index.md). +- Learn how to use KubeStash to backup workloads data from [here](/docs/guides/workloads/overview/index.md). +- Learn how to use KubeStash to backup databases from [here](/docs/guides/addons/overview/index.md). +- Learn how to use KubeStash to backup stand-alone PVC from [here](/docs/guides/volumes/overview/index.md). diff --git a/docs/guides/backends/b2/examples/b2.yaml b/docs/guides/backends/b2/examples/b2.yaml deleted file mode 100644 index e03411c..0000000 --- a/docs/guides/backends/b2/examples/b2.yaml +++ /dev/null @@ -1,11 +0,0 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: b2-repo - namespace: demo -spec: - backend: - b2: - bucket: stash-backup - prefix: /demo/deployment/my-deploy - storageSecretName: b2-secret diff --git a/docs/guides/backends/b2/index.md b/docs/guides/backends/b2/index.md deleted file mode 100644 index cbb021f..0000000 --- a/docs/guides/backends/b2/index.md +++ /dev/null @@ -1,87 +0,0 @@ ---- -title: Backblaze B2 Backend | Stash -description: Configure Stash to use Backblaze B2 as Backend. -menu: - docs_{{ .version }}: - identifier: backend-b2 - name: Backblaze B2 - parent: backend - weight: 70 -product_name: kubestash -menu_name: docs_{{ .version }} -section_menu_id: guides ---- - -# Backblaze B2 - -Stash supports Backblaze's [B2 Cloud Storage](https://www.backblaze.com/b2/cloud-storage.html) as a backend. This tutorial will show you how to use this backend. - -In order to use Backblaze B2 Cloud Storage as backend, you have to create a `Secret` and a `Repository` object pointing to the desired B2 bucket. - ->If the bucket does not exist yet and the credentials you have provided have the privilege to create bucket, it will be created automatically during the first backup. In this case, you have to make sure that the bucket name is unique across all B2 buckets. - -#### Create Storage Secret - -To configure storage secret for this backend, following secret keys are needed: - -| Key | Type | Description | -| ----------------- | ---------- | ----------------------------------------------------------- | -| `RESTIC_PASSWORD` | `Required` | Password that will be used to encrypt the backup snapshots. | -| `B2_ACCOUNT_ID` | `Required` | Backblaze B2 account id. | -| `B2_ACCOUNT_KEY` | `Required` | Backblaze B2 account key. | - -Create storage secret as below, - -```bash -$ echo -n 'changeit' > RESTIC_PASSWORD -$ echo -n '' > B2_ACCOUNT_ID -$ echo -n '' > B2_ACCOUNT_KEY -$ kubectl create secret generic -n demo b2-secret \ - --from-file=./RESTIC_PASSWORD \ - --from-file=./B2_ACCOUNT_ID \ - --from-file=./B2_ACCOUNT_KEY -secret/b2-secret created -``` - -### Create Repository - -Now, you have to create a `Repository` crd. You have to provide the storage secret that we have created earlier in `spec.backend.storageSecretName` field. - -Following parameters are available for `b2` backend, - -| Parameter | Type | Description | -| ------------------- | ---------- | ----------------------------------------------------------------------------------------------------------------------------------- | -| `b2.bucket` | `Required` | Name of the B2 bucket. | -| `b2.prefix` | `Optional` | Path prefix inside the bucket where the backed up data will be stored. | -| `b2.maxConnections` | `Optional` | Maximum number of parallel connections to use for uploading backup data. By default, Stash will use maximum 5 parallel connections. | - -Below, the YAML of a sample `Repository` crd that uses a B2 bucket as a backend. - -```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: b2-repo - namespace: demo -spec: - backend: - b2: - bucket: stash-backup - prefix: /demo/deployment/my-deploy - storageSecretName: b2-secret -``` - -Create the `Repository` we have shown above using the following command, - -```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/b2/examples/b2.yaml -repository/b2-repo created -``` - -Now, we are ready to use this backend to backup our desired data using Stash. - -## Next Steps - -- Learn how to use Stash to backup workloads data from [here](/docs/guides/workloads/overview/index.md). -- Learn how to use Stash to backup databases from [here](/docs/guides/addons/overview/index.md). -- Learn how to use Stash to backup stand-alone PVC from [here](/docs/guides/volumes/overview/index.md). diff --git a/docs/guides/backends/gcs/examples/gcs.yaml b/docs/guides/backends/gcs/examples/gcs.yaml index 8dadcee..25de7be 100644 --- a/docs/guides/backends/gcs/examples/gcs.yaml +++ b/docs/guides/backends/gcs/examples/gcs.yaml @@ -1,11 +1,19 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: gcs-repo + name: s3-storage namespace: demo spec: - backend: - gcs: - bucket: stash-backup - prefix: /demo/deployment/my-deploy - storageSecretName: gcs-secret + storage: + provider: s3 + s3: + endpoint: s3.amazonaws.com # use server URL for s3 compatible other storage service + bucket: kubestash-demo + region: us-west-1 + prefix: /backup/demo/deployment/kubestash-demo + secretName: s3-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/backends/gcs/index.md b/docs/guides/backends/gcs/index.md index 8c2ccdc..35d5b7e 100644 --- a/docs/guides/backends/gcs/index.md +++ b/docs/guides/backends/gcs/index.md @@ -1,6 +1,6 @@ --- -title: GCS Backend | Stash -description: Configure Stash to use Google Cloud Storage (GCS) as Backend. +title: GCS Backend | KubeStash +description: Configure KubeStash to use Google Cloud Storage (GCS) as Backend. menu: docs_{{ .version }}: identifier: backend-gcs @@ -14,11 +14,11 @@ section_menu_id: guides # Google Cloud Storage (GCS) -Stash supports [Google Cloud Storage(GCS)](https://cloud.google.com/storage/) as a backend. This tutorial will show you how to use this backend. +KubeStash supports [Google Cloud Storage(GCS)](https://cloud.google.com/storage/) as a backend. This tutorial will show you how to use this backend. -In order to use Google Cloud Storage as backend, you have to create a `Secret` and a `Repository` object pointing to the desired GCS bucket. +In order to use Google Cloud Storage as backend, you have to create a `Secret` and a `BackupStorage` object pointing to the desired GCS bucket. -> If the bucket already exists, the Google Cloud service account you provide to Stash only needs `Storage Object Creator` role permission. However, if the bucket does not exist, Stash will create the bucket during the first backup. In this case, the Google Cloud service account key used for Stash must have `Storage Object Admin` role permission. To avoid giving this elevated level of permission to Stash, create the bucket manually (either from GCP console or gcloud cli) ahead of time. +> If the bucket already exists, the Google Cloud service account you provide to KubeStash only needs `Storage Object Creator` role permission. However, if the bucket does not exist, KubeStash will create the bucket during the first backup. In this case, the Google Cloud service account key used for KubeStash must have `Storage Object Admin` role permission. To avoid giving this elevated level of permission to KubeStash, create the bucket manually (either from GCP console or gcloud cli) ahead of time. #### Create Storage Secret @@ -26,62 +26,66 @@ To configure storage secret for this backend, following secret keys are needed: | Key | Type | Description | | --------------------------------- | ---------- | ----------------------------------------------------------- | -| `RESTIC_PASSWORD` | `Required` | Password that will be used to encrypt the backup snapshots. | | `GOOGLE_PROJECT_ID` | `Required` | Google Cloud project ID. | | `GOOGLE_SERVICE_ACCOUNT_JSON_KEY` | `Required` | Google Cloud service account json key. | Create storage secret as below, ```bash -$ echo -n 'changeit' > RESTIC_PASSWORD $ echo -n '' > GOOGLE_PROJECT_ID $ mv downloaded-sa-json.key GOOGLE_SERVICE_ACCOUNT_JSON_KEY $ kubectl create secret generic -n demo gcs-secret \ - --from-file=./RESTIC_PASSWORD \ --from-file=./GOOGLE_PROJECT_ID \ --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY secret/gcs-secret created ``` -### Create Repository +### Create BackupStorage -Now, you have to create a `Repository` crd. You have to provide the storage secret that we have created earlier in `spec.backend.storageSecretName` field. +Now, you have to create a `BackupStorage` crd. You have to provide the storage secret that we have created earlier in `spec.backend.storageSecretName` field. Following parameters are available for `gcs` backend. -| Parameter | Type | Description | -| -------------------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `gcs.bucket` | `Required` | Name of Bucket. If the bucket does not exist yet, it will be created in the default location (US). It is not possible at the moment for Stash to create a new bucket in a different location, so you need to create it using Google cloud console. | -| `gcs.prefix` | `Optional` | Path prefix inside the bucket where backed up data will be stored. | -| `gcs.maxConnections` | `Optional` | Maximum number of parallel connections to use for uploading backup data. By default, Stash will use maximum 5 parallel connections. | +| Parameter | Type | Description | +| -------------------- | ---------- |--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `gcs.bucket` | `Required` | Name of Bucket. If the bucket does not exist yet, it will be created in the default location (US). It is not possible at the moment for KubeStash to create a new bucket in a different location, so you need to create it using Google cloud console. | +| `gcs.prefix` | `Optional` | Path prefix inside the bucket where backed up data will be stored. | +| `gcs.maxConnections` | `Optional` | Maximum number of parallel connections to use for uploading backup data. By default, KubeStash will use maximum 5 parallel connections. | -Below, the YAML of a sample `Repository` crd that uses a GCS bucket as a backend. +Below, the YAML of a sample `BackupStorage` crd that uses a GCS bucket as a backend. ```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: gcs-repo + name: gcs-storage namespace: demo spec: - backend: + storage: + provider: gcs gcs: - bucket: stash-backup - prefix: /demo/deployment/my-deploy - storageSecretName: gcs-secret + bucket: kubestash-demo + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true # Use this BackupStorage as the default + deletionPolicy: WipeOut # One of: WipeOut, Delete ``` -Create the `Repository` we have shown above using the following command, +Create the `BackupStorage` we have shown above using the following command, ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/gcs/examples/gcs.yaml -repository/gcs-repo created +backupstorage.storage.kubestash.com/gcs-storaqge created + ``` -Now, we are ready to use this backend to backup our desired data using Stash. +Now, we are ready to use this backend to backup our desired data using KubeStash. ## Next Steps -- Learn how to use Stash to backup workloads data from [here](/docs/guides/workloads/overview/index.md). -- Learn how to use Stash to backup databases from [here](/docs/guides/addons/overview/index.md). -- Learn how to use Stash to backup stand-alone PVC from [here](/docs/guides/volumes/overview/index.md). +- Learn how to use KubeStash to backup workloads data from [here](/docs/guides/workloads/overview/index.md). +- Learn how to use KubeStash to backup databases from [here](/docs/guides/addons/overview/index.md). +- Learn how to use KubeStash to backup stand-alone PVC from [here](/docs/guides/volumes/overview/index.md). diff --git a/docs/guides/backends/local/examples/awsElasticBolckStore.yaml b/docs/guides/backends/local/examples/awsElasticBolckStore.yaml deleted file mode 100644 index b561256..0000000 --- a/docs/guides/backends/local/examples/awsElasticBolckStore.yaml +++ /dev/null @@ -1,13 +0,0 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: local-repo-with-awsebs - namespace: demo -spec: - backend: - local: - mountPath: /safe/data - awsElasticBlockStore: # This AWS EBS volume must already exist. - volumeID: - fsType: ext4 - storageSecretName: local-secret diff --git a/docs/guides/backends/local/examples/azureDisk.yaml b/docs/guides/backends/local/examples/azureDisk.yaml deleted file mode 100644 index fb4a839..0000000 --- a/docs/guides/backends/local/examples/azureDisk.yaml +++ /dev/null @@ -1,13 +0,0 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: local-repo-with-azuredisk - namespace: demo -spec: - backend: - local: - mountPath: /safe/data - azureDisk: - diskName: stash.vhd - diskURI: https://someaccount.blob.microsoft.net/vhds/stash.vhd - storageSecretName: local-secret diff --git a/docs/guides/backends/local/examples/emptyDir.yaml b/docs/guides/backends/local/examples/emptyDir.yaml deleted file mode 100644 index cac13c8..0000000 --- a/docs/guides/backends/local/examples/emptyDir.yaml +++ /dev/null @@ -1,11 +0,0 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: local-repo-with-emptydir - namespace: demo -spec: - backend: - local: - mountPath: /safe/data - emptyDir: {} - storageSecretName: local-secret diff --git a/docs/guides/backends/local/examples/gcePersistentDisk.yaml b/docs/guides/backends/local/examples/gcePersistentDisk.yaml deleted file mode 100644 index 9409337..0000000 --- a/docs/guides/backends/local/examples/gcePersistentDisk.yaml +++ /dev/null @@ -1,13 +0,0 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: local-repo-with-gcepersistentdisk - namespace: demo -spec: - backend: - local: - mountPath: /safe/data - gcePersistentDisk: - pdName: stash-repo - fsType: ext4 - storageSecretName: local-secret diff --git a/docs/guides/backends/local/examples/hostPath.yaml b/docs/guides/backends/local/examples/hostPath.yaml index aad9716..ce5c3a4 100644 --- a/docs/guides/backends/local/examples/hostPath.yaml +++ b/docs/guides/backends/local/examples/hostPath.yaml @@ -1,12 +1,21 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: local-repo-with-hostpath + name: local-storage-with-hostpath namespace: demo spec: - backend: + storage: + provider: local local: mountPath: /safe/data hostPath: - path: /data/stash-test/repo - storageSecretName: local-secret + path: /data/kubestash-test/storage + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut + runtimeSettings: + pod: + securityContext: + runAsUser: 0 # Give run as a root permission \ No newline at end of file diff --git a/docs/guides/backends/local/examples/nfs.yaml b/docs/guides/backends/local/examples/nfs.yaml index 5d2bdb5..bce0074 100644 --- a/docs/guides/backends/local/examples/nfs.yaml +++ b/docs/guides/backends/local/examples/nfs.yaml @@ -1,13 +1,22 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: local-repo-with-nfs + name: local-storage-with-nfs namespace: demo spec: - backend: + storage: + provider: local local: mountPath: /safe/data nfs: server: "nfs-service.storage.svc.cluster.local" # use you own NFS server address path: "/" # this path is relative to "/exports" path of NFS server - storageSecretName: local-secret + usagePolicy: + allowedNamespaces: + from: All + default: false + deletionPolicy: WipeOut + runtimeSettings: + pod: + securityContext: + fsGroup: 65535 # Give file system group permission \ No newline at end of file diff --git a/docs/guides/backends/local/examples/pvc.yaml b/docs/guides/backends/local/examples/pvc.yaml index aa1d9f3..44f962e 100644 --- a/docs/guides/backends/local/examples/pvc.yaml +++ b/docs/guides/backends/local/examples/pvc.yaml @@ -1,12 +1,21 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: local-repo-with-pvc + name: local-storage-with-pvc namespace: demo spec: - backend: + storage: + provider: local local: mountPath: /safe/data persistentVolumeClaim: - claimName: repo-pvc - storageSecretName: local-secret + claimName: storage-pvc + usagePolicy: + allowedNamespaces: + from: All + default: false + deletionPolicy: WipeOut + runtimeSettings: + pod: + securityContext: + fsGroup: 65535 # Give file system group permission diff --git a/docs/guides/backends/local/examples/storageOS.yaml b/docs/guides/backends/local/examples/storageOS.yaml deleted file mode 100644 index 2905a5c..0000000 --- a/docs/guides/backends/local/examples/storageOS.yaml +++ /dev/null @@ -1,13 +0,0 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: local-repo-with-storageos - namespace: demo -spec: - backend: - local: - mountPath: /safe/data - storageos: - volumeName: stash-vol01 # The `stash-vol01` volume must already exist within StorageOS in the `demo` namespace. - fsType: ext4 - storageSecretName: local-secret diff --git a/docs/guides/backends/local/index.md b/docs/guides/backends/local/index.md index 4e47034..4b7b073 100644 --- a/docs/guides/backends/local/index.md +++ b/docs/guides/backends/local/index.md @@ -1,6 +1,6 @@ --- -title: Local Backend | Stash -description: Configure Stash to Use Local Backend. +title: Local Backend | KubeStash +description: Configure KubeStash to Use Local Backend. menu: docs_{{ .version }}: identifier: backend-local @@ -14,261 +14,139 @@ section_menu_id: guides # Local Backend -`Local` backend refers to a local path inside `stash` sidecar container. Any Kubernetes supported [persistent volume](https://kubernetes.io/docs/concepts/storage/volumes/) such as [PersistentVolumeClaim](https://kubernetes.io/docs/concepts/storage/volumes/#persistentvolumeclaim), [HostPath](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath), [EmptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) (for testing only), [NFS](https://kubernetes.io/docs/concepts/storage/volumes/#nfs), [gcePersistentDisk](https://kubernetes.io/docs/concepts/storage/volumes/#gcepersistentdisk) etc. can be used as local backend. +### What is Local Backend -In order to use Kubernetes volumes as backend, you have to create a `Secret` and a `Repository` object pointing to the desired volume. +`Local` backend refers to a local path inside `kubestash` backup container. Any Kubernetes supported [persistent volume](https://kubernetes.io/docs/concepts/storage/volumes/) such as [PersistentVolumeClaim](https://kubernetes.io/docs/concepts/storage/volumes/#persistentvolumeclaim), [HostPath](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath), [EmptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) (for testing only), [NFS](https://kubernetes.io/docs/concepts/storage/volumes/#nfs), [gcePersistentDisk](https://kubernetes.io/docs/concepts/storage/volumes/#gcepersistentdisk) etc. can be used as local backend. -### Create Storage Secret +### Create BackupStorage -To configure storage secret for local backend, following secret keys are needed: +Now, In this section, we are going to create `BackupStorage` object that uses Kubernetes volumes as a backend. -| Key | Type | Description | -| ----------------- | ---------- | ---------------------------------------------------------- | -| `RESTIC_PASSWORD` | `Required` | Password that will be used to encrypt the backup snapshots | - -Create storage secret as below, - -```bash -$ echo -n 'changeit' > RESTIC_PASSWORD -$ kubectl create secret generic -n demo local-secret --from-file=./RESTIC_PASSWORD -secret/local-secret created -``` - -### Create Repository +Following parameters are available for `Local` backend. -Now, you have to create a `Repository` crd that uses Kubernetes volume as a backend. You have to provide the storage secret that we have created earlier in `spec.backend.storageSecretName` field. +| Parameter | Type | Description | +| -------------------- | ---------- |----------------------------------------------------------------------------------------------------| +| `local.mountPath` | `Required` | Path where this volume will be mounted inside the backup job container. Example: `/safe/data`. | +| `local.subPath` | `Optional` | Sub-path inside the referenced volume where the backed up data will be stored instead of its root. | +| `local.VolumeSource` | `Required` | Any Kubernetes volume. Can be specified inlined. Example: `hostPath`. | -Following parameters are available for `Local` backend. -| Parameter | Type | Description | -| -------------------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `local.mountPath` | `Required` | Path where this volume will be mounted inside the sidecar container. Example: `/safe/data`.
We have put `stash` binary in the root directory. Hence, you can not use `/stash` or `/stash/*` as `local.mountPath` | -| `local.subPath` | `Optional` | Sub-path inside the referenced volume where the backed up snapshot will be stored instead of its root. | -| `local.VolumeSource` | `Required` | Any Kubernetes volume. Can be specified inlined. Example: `hostPath`. | +> Note that by default, for local backend KubeStash run an initializer job, which doesn’t have file write permission. So, in order to achieve +that you must give file system group permission, achieved by specifying `spec.securityContext.pod.fsGroup` in the BackupStorage configuration. -Here, we are going to show some sample `Repository` crds that uses different Kubernetes volume as a backend. +Here, we are going to show some sample `BackupStorage` objects that uses different Kubernetes volume as a backend. -##### HostPath volume as Backend +### HostPath volume as Backend -Below, the YAML of a sample `Repository` crd that uses a `hostPath` volume as a backend. +Below, the YAML of a sample `BackupStorage` object that uses a `hostPath` volume as a backend. ```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: local-repo-with-hostpath + name: local-storage-with-hostpath namespace: demo spec: - backend: + storage: + provider: local local: mountPath: /safe/data hostPath: - path: /data/stash-test/repo - storageSecretName: local-secret + path: /data/kubestash-test/storage + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut + runtimeSettings: + pod: + securityContext: + runAsUser: 0 ``` -Create the `Repository` we have shown above using the following command, +Create the `BackupStoage` we have shown above using the following command, ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/local/examples/hostPath.yaml -repository/local-repo-with-hostpath created +backupstorage.storage.kubestash.com/local-storage-with-hostpath created ``` ->Note that by default, Stash runs as `non-root` user. `hostPath` volume is writable only for `root` user. So, in order to use `hostPath` volume as backend, either you have to run Stash as `root` user using securityContext or you have to change the permission of the `hostPath` to make it writable for `non-root` users. +> Note that by default, Kubestash run backupStorage initializer job with a `non-root` user. `hostPath` volume is writable only for the `root` user. +So, in order to use `hostPath` volume as a backend, you must either run initializer job as the `root` user, achieved by specifying +`spec.securityContext.pod.runAsUser` in the BackupStorage configuration, or adjust the permissions of the `hostPath` to allow write access for `non-root` users. -##### PersistentVolumeClaim as Backend +### PersistentVolumeClaim as Backend -Below, the YAML of a sample `Repository` crd that uses a `PersistentVolumeClaim` as a backend. +Below, the YAML of a sample `BackupStorage` crd that uses a `PersistentVolumeClaim` as a backend. ```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: local-repo-with-pvc + name: local-storage-with-pvc namespace: demo spec: - backend: + storage: + provider: local local: mountPath: /safe/data persistentVolumeClaim: - claimName: repo-pvc - storageSecretName: local-secret + claimName: storage-pvc + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut + runtimeSettings: + pod: + securityContext: + fsGroup: 65535 ``` -Create the `Repository` we have shown above using the following command, +Create the `BackupStorage` we have shown above using the following command, ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/local/examples/pvc.yaml -repository/local-repo-with-pvc created +backupstorage.storage.kubestash.com/local-storage-with-pvc created ``` -##### NFS volume as Backend +### NFS volume as Backend -Below, the YAML of a sample `Repository` crd that uses an `NFS` volume as a backend. +Below, the YAML of a sample `BackupStorage` crd that uses an `NFS` volume as a backend. ```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: local-repo-with-nfs + name: local-storage-with-nfs namespace: demo spec: - backend: + storage: + provider: local local: mountPath: /safe/data nfs: server: "nfs-service.storage.svc.cluster.local" # use you own NFS server address path: "/" # this path is relative to "/exports" path of NFS server - storageSecretName: local-secret + usagePolicy: + allowedNamespaces: + from: All + default: false + deletionPolicy: WipeOut + runtimeSettings: + pod: + securityContext: + fsGroup: 65535 ``` -Create the `Repository` we have shown above using the following command, +Create the `BackupStorage` we have shown above using the following command, ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/local/examples/nfs.yaml -repository/local-repo-with-nfs created -``` - ->For NFS backend, Stash may have to run the network volume accessor deployments in privileged mode to provide Snapshot listing facility. In this case, please configure network volume accessors by following the instruction [here](/docs/setup/install/troubleshooting/index.md#configuring-network-volume-accessor). - -##### GCE PersitentDisk as Backend - -Below, the YAML of a sample `Repository` crd that uses a [gcePersistentDisk](https://kubernetes.io/docs/concepts/storage/volumes/#gcepersistentdisk) as a backend. - -```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: local-repo-with-gcepersistentdisk - namespace: demo -spec: - backend: - local: - mountPath: /safe/data - gcePersistentDisk: - pdName: stash-repo - fsType: ext4 - storageSecretName: local-secret -``` - -Create the `Repository` we have shown above using the following command, - -```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/local/examples/gcePersistentDisk.yaml -repository/local-repo-with-gcepersistentdisk created -``` - ->In order to use `gcePersistentDisk` volume as backend, the node where stash container is running must be a GCE VM and the VM must be in same GCE project and zone as the Persistent Disk. - -##### AWS EBS volume as Backend - -Below, the YAML of a sample `Repository` crd that uses an [awsElasticBlockStore](https://kubernetes.io/docs/concepts/storage/volumes/#awselasticblockstore) as a backend. - -```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: local-repo-with-awsebs - namespace: demo -spec: - backend: - local: - mountPath: /safe/data - awsElasticBlockStore: # This AWS EBS volume must already exist. - volumeID: - fsType: ext4 - storageSecretName: local-secret -``` - -Create the `Repository` we have shown above using the following command, - -```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/local/examples/awsElasticBlockStore.yaml -repository/local-repo-with-awsebs created -``` - ->In order to use `awsElasticBlockStore` volume as backend, the pod where stash container is running must be running on an AWS EC2 instance and the instance must be in the same region and availability-zone as the EBS volume. - -##### Azure Disk as Backend - -Below, the YAML of a sample `Repository` crd that uses an [azureDisk](https://kubernetes.io/docs/concepts/storage/volumes/#azuredisk) as a backend. - -```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: local-repo-with-azuredisk - namespace: demo -spec: - backend: - local: - mountPath: /safe/data - azureDisk: - diskName: stash.vhd - diskURI: https://someaccount.blob.microsoft.net/vhds/stash.vhd - storageSecretName: local-secret -``` - -Create the `Repository` we have shown above using the following command, - -```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/local/examples/azureDisk.yaml -repository/local-repo-with-azuredisk created -``` - -##### StorageOS as Backend - -Below, the YAML of a sample `Repository` crd that uses a [storageOS](https://kubernetes.io/docs/concepts/storage/volumes/#storageos) volume as a backend. - -```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: local-repo-with-storageos - namespace: demo -spec: - backend: - local: - mountPath: /safe/data - storageos: - volumeName: stash-vol01 # The `stash-vol01` volume must already exist within StorageOS in the `demo` namespace. - fsType: ext4 - storageSecretName: local-secret -``` - -Create the `Repository` we have shown above using the following command, - -```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/local/examples/storageOS.yaml -repository/local-repo-with-storageos created -``` - -##### EmptyDir volume as Backend - -Below, the YAML of a sample `Repository` crd that uses an [emptyDir](https://kubernetes.io/docs/concepts/storage/volumes/#emptydir) as a backend. - -```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: local-repo-with-emptydir - namespace: demo -spec: - backend: - local: - mountPath: /safe/data - emptyDir: {} - storageSecretName: local-secret -``` - -Create the `Repository` we have shown above using the following command, - -```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/local/examples/emptyDir.yaml -repository/local-repo-with-emptydir created +backupstorage.storage.kubestash.com/local-storage-with-nfs created ``` ->**Warning:** Data of an `emptyDir` volume is not persistent. If you delete the pod that runs the respective stash container, you will lose all the backed up data. You should use this kind of volumes only to test backup process. +>For NFS backend, KubeStash may have to run the network volume accessor deployments in privileged mode to provide Snapshot listing facility. In this case, please configure network volume accessors by following the instruction [here](/docs/setup/install/troubleshooting/index.md#configuring-network-volume-accessor). ## Next Steps diff --git a/docs/guides/backends/overview/images/backend_overview.svg b/docs/guides/backends/overview/images/backend_overview.svg deleted file mode 100644 index 7106b39..0000000 --- a/docs/guides/backends/overview/images/backend_overview.svg +++ /dev/null @@ -1,527 +0,0 @@ - - - - - - image/svg+xml - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/docs/guides/backends/overview/images/backupstorage-initialize.png b/docs/guides/backends/overview/images/backupstorage-initialize.png new file mode 100644 index 0000000..fab8a75 Binary files /dev/null and b/docs/guides/backends/overview/images/backupstorage-initialize.png differ diff --git a/docs/guides/backends/overview/images/gcs_repository.png b/docs/guides/backends/overview/images/gcs_repository.png new file mode 100644 index 0000000..d39a904 Binary files /dev/null and b/docs/guides/backends/overview/images/gcs_repository.png differ diff --git a/docs/guides/backends/overview/images/kubestash_backend_overview.svg b/docs/guides/backends/overview/images/kubestash_backend_overview.svg new file mode 100644 index 0000000..51501d5 --- /dev/null +++ b/docs/guides/backends/overview/images/kubestash_backend_overview.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/guides/backends/overview/images/s3_repository.png b/docs/guides/backends/overview/images/s3_repository.png deleted file mode 100644 index 47cdab9..0000000 Binary files a/docs/guides/backends/overview/images/s3_repository.png and /dev/null differ diff --git a/docs/guides/backends/overview/index.md b/docs/guides/backends/overview/index.md index 5063a50..e4b8dcc 100644 --- a/docs/guides/backends/overview/index.md +++ b/docs/guides/backends/overview/index.md @@ -1,6 +1,6 @@ --- -title: Backend Overview | Stash -description: An overview of the backends used by Stash to store backed up data. +title: Backend Overview | KubeStash +description: An overview of the backends used by KubeStash to store backed up data. menu: docs_{{ .version }}: identifier: backend-overview @@ -12,40 +12,46 @@ menu_name: docs_{{ .version }} section_menu_id: guides --- -> New to Stash? Please start [here](/docs/concepts/README.md). +> New to KubeStash? Please start [here](/docs/concepts/README.md). -# Stash Backends +# KubeStash Backends -Stash supports various backends for storing data snapshots. It can be a cloud storage like GCS bucket, AWS S3, Azure Blob Storage etc. or a Kubernetes persistent volume like [HostPath](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath), [PersistentVolumeClaim](https://kubernetes.io/docs/concepts/storage/volumes/#persistentvolumeclaim), [NFS](https://kubernetes.io/docs/concepts/storage/volumes/#nfs) etc. +## BackupStorage -The following diagram shows how Stash sidecar container accesses and backs up data into a backend. +KubeStash supports various backends for use as a BackupStorage. It can be a cloud storage like GCS bucket, AWS S3, Azure Blob Storage etc. or a Kubernetes persistent volume like [HostPath](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath), [PersistentVolumeClaim](https://kubernetes.io/docs/concepts/storage/volumes/#persistentvolumeclaim), [NFS](https://kubernetes.io/docs/concepts/storage/volumes/#nfs) etc. + +The following diagram shows how kubestash backup container accesses and backs up data into a backend.
-  Stash Backend Overview -
Fig: Stash Backend Overview
+ KubeStash Backend Overview +
Fig: KubeStash Backend Overview
-You have to create a [Repository](/docs/concepts/crds/repository/index.md) object which contains backend information and a `Secret` which contains necessary credentials to access the backend. +You need to create a [BackupStorage]() object that contains backend information along with a Secret object containing the corresponding backend credentials required for accessing the backend. + +When you create the BackupStorage object, the Kubestash operator reads backend information from this object and retrieves access credentials from the associated Secret and initializes the BackupStorage. -Stash sidecar/backup job reads backend information from the `Repository` and retrieves access credentials from the `Secret`. Then on the first backup session, Stash will initialize a repository in the backend. +Below, a screenshot that shows initialization of a BackupStorage in a GCS bucket named `kubestash-qa`: -Below, a screenshot that shows a repository created in AWS S3 bucket named `stash-qa`:
-  Repository in AWS S3 Backend -
Fig: Repository in AWS S3 Backend
+  BackupStorage initialization in GCS Backend +
Fig: BackupStorage initialization in GCS Backend
-You will see all snapshots taken by Stash at `/snapshot` directory of this repository. +Here, `kubestash-qa` serves as the bucket name, and the presence of `metadata.yaml` indicates the successful initialization of the BackupStorage. -> Note: Stash stores data encrypted at rest. So, snapshot files in the bucket will not contain any meaningful data until they are decrypted. +## Repository +Once the BackupStoarge is initialize and in ready phase then the next steps is creating [BackupConfiguration](). When you create a BackupConfiguration, then KubeStash operator retrieves [Repository]() information from it and create Repository. This Repository object serve as a container for effectively managing and storing the backups data. -## Next Steps -- Learn how to configure `Kubernetes Volume` as backend from [here](/docs/guides/backends/local/index.md). -- Learn how to configure `AWS S3/Minio/Rook` backend from [here](/docs/guides/backends/s3/index.md). -- Learn how to configure `Google Cloud Storage (GCS)` backend from [here](/docs/guides/backends/gcs/index.md). -- Learn how to configure `Microsoft Azure Storage` backend from [here](/docs/guides/backends/azure/index.md). -- Learn how to configure `OpenStack Swift` backend from [here](/docs/guides/backends/swift/index.md). -- Learn how to configure `Backblaze B2` backend from [here](/docs/guides/backends/b2/index.md). -- Learn how to configure `REST` backend from [here](/docs/guides/backends/rest/index.md). +Below, a screenshot the shows a `Repository` named `mongodb-backup` with backup data under a `BackupStorage` named `demo`: + +
+  Repository with backup data under a `demo` BackupStorage +
Fig: Repository with backup data under a BackupStorage
+
+ + +## Next Steps +- Learn how to configure `Kubernetes Volume` as backend from [here](). diff --git a/docs/guides/backends/rest/examples/rest.yaml b/docs/guides/backends/rest/examples/rest.yaml deleted file mode 100644 index de74aae..0000000 --- a/docs/guides/backends/rest/examples/rest.yaml +++ /dev/null @@ -1,10 +0,0 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: rest-repo - namespace: demo -spec: - backend: - rest: - url: http://rest-server.demo.svc:8000/stash-backup-demo - storageSecretName: rest-secret diff --git a/docs/guides/backends/rest/index.md b/docs/guides/backends/rest/index.md deleted file mode 100644 index 9d92887..0000000 --- a/docs/guides/backends/rest/index.md +++ /dev/null @@ -1,83 +0,0 @@ ---- -title: REST Backend | Stash -description: Configure Stash to REST Server as Backend. -menu: - docs_{{ .version }}: - identifier: backend-rest - name: REST Server - parent: backend - weight: 80 -product_name: kubestash -menu_name: docs_{{ .version }} -section_menu_id: guides ---- - -# REST Backend - -Stash supports restic's [REST Server](https://github.com/restic/rest-server) as a backend. This tutorial will show you how to use this backend. - -In order to use REST Server as backend, you have to create a `Secret` and a `Repository` object pointing to the desired REST Server address. - -#### Create Storage Secret - -To configure storage secret for this backend, following secret keys are needed: - -| Key | Type | Description | -| ---------------------- | ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `RESTIC_PASSWORD` | `Required` | Password that will be used to encrypt the backup snapshots. | -| `REST_SERVER_USERNAME` | `Optional` | Username for basic authentication in the REST server. | -| `REST_SERVER_PASSWORD` | `Optional` | Password for basic authentication in the REST Server | -| `CA_CERT_DATA` | `optional` | CA certificate used by storage backend. This can be used to pass the root certificate that has been used to sign the server certificate of a TLS secured REST Server. | - -Create storage secret as below, - -```bash -$ echo -n 'changeit' > RESTIC_PASSWORD -$ echo -n '' > REST_SERVER_USERNAME -$ echo -n '' > REST_SERVER_PASSWORD -$ kubectl create secret generic -n demo rest-secret \ - --from-file=./RESTIC_PASSWORD \ - --from-file=./REST_SERVER_USERNAME \ - --from-file=./REST_SERVER_PASSWORD -secret/rest-secret created -``` - -### Create Repository - -Now, you have to create a `Repository` crd. You have to provide the storage secret that we have created earlier in `spec.backend.storageSecretName` field. - -Following parameters are available for `rest` backend, - -| Parameter | Type | Description | -| ---------- | ---------- | ------------------------------------------------------------------------------------------------------------- | -| `rest.url` | `Required` | URL of the REST Server along with an optional path inside the server where backed up snapshot will be stored. | - -Below, the YAML of a sample `Repository` crd that uses a REST Server as a backend. - -```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: rest-repo - namespace: demo -spec: - backend: - rest: - url: http://rest-server.demo.svc:8000/stash-backup-demo - storageSecretName: rest-secret -``` - -Create the `Repository` we have shown above using the following command, - -```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/rest/examples/rest.yaml -repository/rest-repo created -``` - -Now, we are ready to use this backend to backup our desired data using Stash. - -## Next Steps - -- Learn how to use Stash to backup workloads data from [here](/docs/guides/workloads/overview/index.md). -- Learn how to use Stash to backup databases from [here](/docs/guides/addons/overview/index.md). -- Learn how to use Stash to backup stand-alone PVC from [here](/docs/guides/volumes/overview/index.md). diff --git a/docs/guides/backends/s3/examples/minio.yaml b/docs/guides/backends/s3/examples/minio.yaml index b8ccb26..79814e5 100644 --- a/docs/guides/backends/s3/examples/minio.yaml +++ b/docs/guides/backends/s3/examples/minio.yaml @@ -1,12 +1,18 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: minio-repo + name: minio-storage namespace: demo spec: - backend: + storage: + provider: s3 s3: endpoint: https://my-minio-service.minio-namespace.svc - bucket: stash-demo - prefix: /backup/demo/deployment/stash-demo - storageSecretName: minio-secret + bucket: kubestash-demo + prefix: /backup/demo/deployment/kubestash-demo + secretName: minio-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/backends/s3/examples/s3.yaml b/docs/guides/backends/s3/examples/s3.yaml index ec4559c..25de7be 100644 --- a/docs/guides/backends/s3/examples/s3.yaml +++ b/docs/guides/backends/s3/examples/s3.yaml @@ -1,13 +1,19 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: s3-repo + name: s3-storage namespace: demo spec: - backend: + storage: + provider: s3 s3: endpoint: s3.amazonaws.com # use server URL for s3 compatible other storage service - bucket: stash-demo + bucket: kubestash-demo region: us-west-1 - prefix: /backup/demo/deployment/stash-demo - storageSecretName: s3-secret + prefix: /backup/demo/deployment/kubestash-demo + secretName: s3-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/backends/s3/index.md b/docs/guides/backends/s3/index.md index 227c101..e814a91 100644 --- a/docs/guides/backends/s3/index.md +++ b/docs/guides/backends/s3/index.md @@ -1,6 +1,6 @@ --- -title: AWS S3/Minio/Rook Backend | Stash -description: Configure Stash to use AWS S3/Minio/Rook as Backend. +title: AWS S3/Minio/Rook Backend | KubeStash +description: Configure KubeStash to use AWS S3/Minio/Rook as Backend. menu: docs_{{ .version }}: identifier: backend-s3 @@ -14,11 +14,11 @@ section_menu_id: guides # AWS S3 -Stash supports AWS S3 or S3 compatible storage services like [Minio](https://minio.io/) servers, [Rook Object Store](https://rook.io/docs/rook/v0.9/ceph-object.html), [DigitalOceans Space](https://www.digitalocean.com/products/spaces/) as a backend. This tutorial will show you how to use this backend. +KubeStash supports AWS S3 or S3 compatible storage services like [Minio](https://minio.io/) servers, [Rook Object Store](https://rook.io/docs/rook/v0.9/ceph-object.html), [DigitalOceans Space](https://www.digitalocean.com/products/spaces/) as a backend. This tutorial will show you how to use this backend. -In order to use S3 or S3 compatible storage service as backend, you have to create a `Secret` and a `Repository` object pointing to the desired bucket. +In order to use S3 or S3 compatible storage service as backend, you have to create a `Secret` and a `BackupStorage` object pointing to the desired bucket. ->If the bucket does not exist yet, Stash will create it automatically in the default region (`us-east-1`) during the first backup. In this case, you have to make sure that the bucket name is unique across all S3 buckets. Currently, it is not possible for Stash to create bucket in different region. You have to create the bucket in your desired region before using it in Stash. +>If the bucket does not exist yet, KubeStash will create it automatically in the default region (`us-east-1`) during the first backup. In this case, you have to make sure that the bucket name is unique across all S3 buckets. Currently, it is not possible for KubeStash to create bucket in different region. You have to create the bucket in your desired region before using it in KubeStash. #### Create Storage Secret @@ -26,7 +26,6 @@ To configure storage secret for this backend, following secret keys are needed: | Key | Type | Description | | ----------------------- | ---------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `RESTIC_PASSWORD` | `Required` | Password that will be used to encrypt the backup snapshots. | | `AWS_ACCESS_KEY_ID` | `Required` | AWS / Minio / Rook / DigitalOcean Spaces access key ID | | `AWS_SECRET_ACCESS_KEY` | `Required` | AWS / Minio / Rook / DigitalOcean Spaces secret access key | | `CA_CERT_DATA` | `optional` | CA certificate used by storage backend. This can be used to pass the root certificate that has been used to sign the server certificate of a TLS secured Minio server. | @@ -34,11 +33,9 @@ To configure storage secret for this backend, following secret keys are needed: Create storage secret as below, ```bash -$ echo -n 'changeit' > RESTIC_PASSWORD $ echo -n '' > AWS_ACCESS_KEY_ID $ echo -n '' > AWS_SECRET_ACCESS_KEY $ kubectl create secret generic -n demo s3-secret \ - --from-file=./RESTIC_PASSWORD \ --from-file=./AWS_ACCESS_KEY_ID \ --from-file=./AWS_SECRET_ACCESS_KEY secret/s3-secret created @@ -47,12 +44,10 @@ secret/s3-secret created For TLS secured Minio Server, create secret as below, ```bash -$ echo -n 'changeit' > RESTIC_PASSWORD $ echo -n '' > AWS_ACCESS_KEY_ID $ echo -n '' > AWS_SECRET_ACCESS_KEY $ cat ./directory/of/root/certificate/ca.crt > CA_CERT_DATA $ kubectl create secret generic -n demo minio-secret \ - --from-file=./RESTIC_PASSWORD \ --from-file=./AWS_ACCESS_KEY_ID \ --from-file=./AWS_SECRET_ACCESS_KEY \ --from-file=./CA_CERT_DATA @@ -61,65 +56,82 @@ secret/minio-secret created {{< notice type="warning" message="If you are using a Minio backend, make sure that you are using `AWS_ACCESS_KEY_ID` instead of `MINIO_ACCESS_KEY` and `AWS_SECRET_ACCESS_KEY` instead of `MINIO_SECRET_KEY`." >}} -### Create Repository +### Create BackupStorage -Now, you have to create a `Repository` crd. You have to provide the storage secret that we have created earlier in `spec.backend.storageSecretName` field. +Now, you have to create a `BackupStorage` object. You have to provide the storage secret that we have created earlier in `spec.storage.s3.secret` field. Following parameters are available for `S3` backend. -| Parameter | Type | Description | -| ------------- | ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `s3.endpoint` | `Required` | For S3, use `s3.amazonaws.com`. If your bucket is in a different location, S3 server (s3.amazonaws.com) will redirect Stash to the correct endpoint. For DigitalOCean, use `nyc3.digitaloceanspaces.com` etc. depending on your bucket region. For S3-compatible other storage services like Minio / Rook use URL of the server. | -| `s3.bucket` | `Required` | Name of Bucket. If the bucket does not exist yet it will be created in the default location (`us-east-1` for S3). It is not possible at the moment for Stash to create a new bucket in a different location, so you need to create it using a different program. | -| `s3.region` | `Optional` | Specify the region of your bucket. | -| `s3.prefix` | `Optional` | Path prefix inside the bucket where the backed up data will be stored. | +| Parameter | Type | Description | +|------------------| ---------- |--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `s3.endpoint` | `Required` | For S3, use `s3.amazonaws.com`. If your bucket is in a different location, S3 server (s3.amazonaws.com) will redirect KubeStash to the correct endpoint. For DigitalOCean, use `nyc3.digitaloceanspaces.com` etc. depending on your bucket region. For S3-compatible other storage services like Minio / Rook use URL of the server. | +| `s3.bucket` | `Required` | Name of Bucket. If the bucket does not exist yet it will be created in the default location (`us-east-1` for S3). It is not possible at the moment for KubeStash to create a new bucket in a different location, so you need to create it using a different program. | +| `s3.secret` | `Required` | Specify the name of the Secret that contains the access credential for this storage. | +| `s3.region` | `Optional` | Specify the region of your bucket. | +| `s3.prefix` | `Optional` | Path prefix inside the bucket where the backed up data will be stored. | +| `s3.insecureTLS` | `Optional` | Specify whether to skip TLS certificate verification. Setting this field to `true` disables verification, which might be necessary in cases where the server uses self-signed certificates or certificates from an untrusted CA. Use this option with caution, as it can expose the client to man-in-the-middle attacks and other security risks. Only use it when absolutely necessary. | Below, the YAML of a sample `Repository` crd that uses an `S3` bucket as a backend. ```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: s3-repo + name: s3-storage namespace: demo spec: - backend: + storage: + provider: s3 s3: endpoint: s3.amazonaws.com # use server URL for s3 compatible other storage service - bucket: stash-demo + bucket: kubestash-demo region: us-west-1 - prefix: /backup/demo/deployment/stash-demo - storageSecretName: s3-secret + prefix: /backup/demo/deployment/kubestash-demo + secretName: s3-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut +``` +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/s3/examples/s3.yaml +backupstorage.storage.kubestash.com/s3-storage created ``` - For S3 compatible Minio and other storage services, specify the endpoint with connection scheme (`http`, or `https`), ```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: minio-repo + name: minio-storage namespace: demo spec: - backend: + storage: + provider: s3 s3: endpoint: https://my-minio-service.minio-namespace.svc - bucket: stash-demo - prefix: /backup/demo/deployment/stash-demo - storageSecretName: s3-secret + bucket: kubestash-demo + prefix: /backup/demo/deployment/kubestash-demo + secretName: minio-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut ``` Create the `s3-repo` Repository we have shown above using the following command, ```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/s3/examples/s3.yaml -repository/s3-repo created +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/s3/examples/minio.yaml +backupstorage.storage.kubestash.com/minio-storage created ``` -Now, we are ready to use this backend to backup our desired data using Stash. +Now, we are ready to use this backend to backup our desired data using KubeStash. ## Next Steps -- Learn how to use Stash to backup workloads data from [here](/docs/guides/workloads/overview/index.md). -- Learn how to use Stash to backup databases from [here](/docs/guides/addons/overview/index.md). -- Learn how to use Stash to backup stand-alone PVC from [here](/docs/guides/volumes/overview/index.md). +- Learn how to use KubeStash to backup workloads data from [here](/docs/guides/workloads/overview/index.md). +- Learn how to use KubeStash to backup databases from [here](/docs/guides/addons/overview/index.md). +- Learn how to use KubeStash to backup stand-alone PVC from [here](/docs/guides/volumes/overview/index.md). diff --git a/docs/guides/backends/swift/examples/swift.yaml b/docs/guides/backends/swift/examples/swift.yaml deleted file mode 100644 index 9a817f3..0000000 --- a/docs/guides/backends/swift/examples/swift.yaml +++ /dev/null @@ -1,11 +0,0 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: swift-repo - namespace: demo -spec: - backend: - swift: - container: stash-backup - prefix: /demo/deployment/my-deploy - storageSecretName: swift-secret diff --git a/docs/guides/backends/swift/index.md b/docs/guides/backends/swift/index.md deleted file mode 100644 index ce490f4..0000000 --- a/docs/guides/backends/swift/index.md +++ /dev/null @@ -1,158 +0,0 @@ ---- -title: Swift Backend | Stash -description: Configure Stash to use OpenStack Swift as Backend. -menu: - docs_{{ .version }}: - identifier: backend-swift - name: OpenStack Swift - parent: backend - weight: 60 -product_name: kubestash -menu_name: docs_{{ .version }} -section_menu_id: guides ---- - -# OpenStack Swift - -Stash supports [OpenStack Swift](https://docs.openstack.org/swift/latest/) as a backend. This tutorial will show you how to use this backend. - -In order to use OpenStack Swift as backend, you have to create a `Secret` and a `Repository` object pointing to the desired Swift container. - ->If the Swift container does not exist yet, Stash will automatically create it during the first backup. - -#### Create Storage Secret - -Stash supports Swift's Keystone v1, v2, v3 authentication as well as token-based authentication. - -**Keystone v1 authentication:** - -For keystone v1 authentication, following secret keys are needed: - -| Key | Description | -|--------------------------|------------------------------------------------------------| -| `RESTIC_PASSWORD` | Password used that will be used to encrypt the backup snapshots.| -| `ST_AUTH` | URL of the Keystone server. | -| `ST_USER` | Username. | -| `ST_KEY` | Password. | - -**Keystone v2 authentication:** - -For keystone v2 authentication, following secret keys are needed: - -| Key | Description | -|--------------------------|------------------------------------------------------------| -| `RESTIC_PASSWORD` | Password used that will be used to encrypt the backup snapshots.| -| `OS_AUTH_URL` | URL of the Keystone server. | -| `OS_REGION_NAME` | Storage region name | -| `OS_USERNAME` | Username | -| `OS_PASSWORD` | Password | -| `OS_TENANT_ID` | Tenant ID | -| `OS_TENANT_NAME` | Tenant Name | - -**Keystone v3 authentication:** - -For keystone v3 authentication, following secret keys are needed: - -| Key | Description | -|--------------------------|------------------------------------------------------------| -| `RESTIC_PASSWORD` | Password used that will be used to encrypt the backup snapshots.| -| `OS_AUTH_URL` | URL of the Keystone server. | -| `OS_REGION_NAME` | Storage region name | -| `OS_USERNAME` | Username | -| `OS_PASSWORD` | Password | -| `OS_USER_DOMAIN_NAME` | User domain name | -| `OS_PROJECT_NAME` | Project name | -| `OS_PROJECT_DOMAIN_NAME` | Project domain name | - -For keystone v3 application credential authentication (application credential id): - -| Key | Description | -|--------------------------|------------------------------------------------------------| -| `RESTIC_PASSWORD` | Password used that will be used to encrypt the backup snapshots.| -| `OS_AUTH_URL` | URL of the Keystone server. | -| `OS_APPLICATION_CREDENTIAL_ID` | The ID of the application credential used for authentication. If not provided, the application credential must be identified by its name and its owning user.| -| `OS_APPLICATION_CREDENTIAL_SECRET` | The secret for authenticating the application credential. | - -For keystone v3 application credential authentication (application credential name): - -| Key | Description | -|--------------------------|------------------------------------------------------------| -| `RESTIC_PASSWORD` | Password used that will be used to encrypt the backup snapshots.| -| `OS_AUTH_URL` | URL of the Keystone server. | -| `OS_USERNAME` | User name| -| `OS_USER_DOMAIN_NAME` | User domain name| -| `OS_APPLICATION_CREDENTIAL_NAME` | The name of the application credential used for authentication. If provided, must be accompanied by a user object. | -| `OS_APPLICATION_CREDENTIAL_SECRET` | The secret for authenticating the application credential. | - -**Token-based authentication:** - -For token-based authentication, following secret keys are needed: - -| Key | Description | -|--------------------------|------------------------------------------------------------| -| `RESTIC_PASSWORD` | Password used that will be used to encrypt the backup snapshots.| -| `OS_STORAGE_URL` | Storage URL | -| `OS_AUTH_TOKEN` | Authentication token | - -A sample storage secret creation for keystone v2 authentication is shown below, - -```bash -$ echo -n 'changeit' > RESTIC_PASSWORD -$ echo -n '' > OS_AUTH_URL -$ echo -n '' > OS_TENANT_ID -$ echo -n '' > OS_TENANT_NAME -$ echo -n '' > OS_USERNAME -$ echo -n '' > OS_PASSWORD -$ echo -n '' > OS_REGION_NAME -$ kubectl create secret generic swift-secret \ - --from-file=./RESTIC_PASSWORD \ - --from-file=./OS_AUTH_URL \ - --from-file=./OS_TENANT_ID \ - --from-file=./OS_TENANT_NAME \ - --from-file=./OS_USERNAME \ - --from-file=./OS_PASSWORD \ - --from-file=./OS_REGION_NAME -secret/swift-secret created -``` - -### Create Repository - -Now, you have to create a `Repository` crd. You have to provide the storage secret that we have created earlier in `spec.backend.storageSecretName` field. - -Following parameters are available for `Swift` backend. - -| Parameter | Description | -| ----------------- | ------------------------------------------------------------------------------ | -| `swift.container` | `Required`. Name of Storage container | -| `swift.prefix` | `Optional`. Path prefix inside the container where backed up data will be stored. | - -Below, the YAML of a sample `Repository` crd that uses a Swift container as a backend. - -```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: swift-repo - namespace: demo -spec: - backend: - swift: - container: stash-backup - prefix: /demo/deployment/my-deploy - storageSecretName: swift-secret -``` - -Create the `Repository` we have shown above using the following command, - -```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/swift/examples/swift.yaml -repository/swift-repo created -``` - -Now, we are ready to use this backend to backup our desired data using Stash. - -## Next Steps - -- Learn how to use Stash to backup workloads data from [here](/docs/guides/workloads/overview/index.md). -- Learn how to use Stash to backup databases from [here](/docs/guides/addons/overview/index.md). -- Learn how to use Stash to backup stand-alone PVC from [here](/docs/guides/volumes/overview/index.md). diff --git a/docs/guides/kubedump/README.md b/docs/guides/kubedump/README.md new file mode 100644 index 0000000..1e5ffc0 --- /dev/null +++ b/docs/guides/kubedump/README.md @@ -0,0 +1,16 @@ +--- +title: KubeDump Backup Addon Overview | Stash +description: KubeDump Backup Addon Overview | Stash +menu: + docs_{{ .version }}: + identifier: stash-kubedump-readme + name: Readme + parent: stash-kubedump + weight: -1 +product_name: kubestash +menu_name: docs_{{ .version }} +section_menu_id: stash-addons +url: /docs/{{ .version }}/addons/kubedump/ +aliases: + - /docs/{{ .version }}/addons/kubedump/README/ +--- diff --git a/docs/guides/kubedump/_index.md b/docs/guides/kubedump/_index.md new file mode 100644 index 0000000..fbf8c78 --- /dev/null +++ b/docs/guides/kubedump/_index.md @@ -0,0 +1,10 @@ +--- +title: KubeDump | KubeStash +menu: + docs_{{ .version }}: + identifier: kubestash-kubedump + name: KubeDump + parent: guides + weight: 80 +menu_name: docs_{{ .version }} +--- diff --git a/docs/guides/kubedump/application/examples/backupconfiguration.yaml b/docs/guides/kubedump/application/examples/backupconfiguration.yaml new file mode 100644 index 0000000..65fe333 --- /dev/null +++ b/docs/guides/kubedump/application/examples/backupconfiguration.yaml @@ -0,0 +1,43 @@ +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: application-manifest-backup + namespace: demo +spec: + target: + apiGroup: apps + kind: Deployment + name: kubestash-kubestash-operator + namespace: kubestash + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /deployment-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + params: + includeDependants: "true" + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader \ No newline at end of file diff --git a/docs/guides/kubedump/application/examples/backupstorage.yaml b/docs/guides/kubedump/application/examples/backupstorage.yaml new file mode 100644 index 0000000..0461b26 --- /dev/null +++ b/docs/guides/kubedump/application/examples/backupstorage.yaml @@ -0,0 +1,17 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/kubedump/application/examples/rbac.yaml b/docs/guides/kubedump/application/examples/rbac.yaml new file mode 100644 index 0000000..1d79b32 --- /dev/null +++ b/docs/guides/kubedump/application/examples/rbac.yaml @@ -0,0 +1,27 @@ +apiVersion: v1 +kind: ServiceAccount +metadata: + name: cluster-resource-reader + namespace: kubestash +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: cluster-resource-reader +rules: +- apiGroups: ["*"] + resources: ["*"] + verbs: ["get","list"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: cluster-resource-reader +subjects: +- kind: ServiceAccount + name: cluster-resource-reader + namespace: kubestash +roleRef: + kind: ClusterRole + name: cluster-resource-reader + apiGroup: rbac.authorization.k8s.io diff --git a/docs/guides/kubedump/application/examples/retentionpolicy.yaml b/docs/guides/kubedump/application/examples/retentionpolicy.yaml new file mode 100644 index 0000000..3af9037 --- /dev/null +++ b/docs/guides/kubedump/application/examples/retentionpolicy.yaml @@ -0,0 +1,15 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same \ No newline at end of file diff --git a/docs/guides/kubedump/application/images/application_manifest_backup.png b/docs/guides/kubedump/application/images/application_manifest_backup.png new file mode 100644 index 0000000..a5fb985 Binary files /dev/null and b/docs/guides/kubedump/application/images/application_manifest_backup.png differ diff --git a/docs/guides/kubedump/application/index.md b/docs/guides/kubedump/application/index.md new file mode 100644 index 0000000..e0369ab --- /dev/null +++ b/docs/guides/kubedump/application/index.md @@ -0,0 +1,563 @@ +--- +title: Backup resource YAMLs of an Application | KubeStash +description: Backup application manifests along with it's dependant using KubeStash +menu: + docs_{{ .version }}: + identifier: kubestash-kubedump-application + name: Application Manifest Backup + parent: kubestash-kubedump + weight: 40 +product_name: kubestash +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +# Backup YAMLs of an Application using KubeStash + +KubeStash `{{< param "info.version" >}}` supports taking backup of the resource YAMLs using `kubedump` plugin. This guide will show you how you can take a backup of the YAMLs of an application along with it's dependant using KubeStash. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the `kubectl` command-line tool must be configured to communicate with your cluster. +- Install KubeStash in your cluster following the steps [here](/docs/setup/install/kubestash/index.md). +- Install KubeStash `kubectl` plugin in your local machine following the steps [here](/docs/setup/install/kubectl-plugin/index.md). +- If you are not familiar with how KubeStash backup the resource YAMLs, please check the following guide [here](/docs/guides/kubedump/overview/index.md). + +You have to be familiar with the following custom resources: + +- [BackupStorage](/docs/concepts/crds/backupstorage/index.md) +- [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md) +- [BackupSession](/docs/concepts/crds/backupsession/index.md) +- [SnapShot](/docs/concepts/crds/snapshot/index.md) +- [RestoreSession](/docs/concepts/crds/restoresession/index.md) +- [RetentionPolicy](/docs/concepts/crds/retentionpolicy/index.md) + + +To keep things isolated, we are going to use a separate namespace called `demo` throughout this tutorial. Create the `demo` namespace if you haven't created it already. + +```bash +❯ kubectl create ns demo +namespace/demo created +``` + +> Note: YAML files used in this tutorial are stored [here](https://github.com/kubestash/docs/tree/{{< param "info.version" >}}/docs/addons/kubedump/application/examples). + +#### Ensure `kubedump` Addon + +**Verify necessary Addons and Functions:** + +When you install KubeStash, it automatically creates the necessary `Addon` and `Function` to backup Kubernetes Resource YAML. + +Let's verify that KubeStash has created the necessary `Function` to backup Kubernetes Resource YAML by the following command, + +```bash +$ kubectl get functions | grep 'kubedump' +kubedump-backup 142m +``` + +Also, verify that the necessary `Addon` has been created, + +```bash +❯ kubectl get addons | grep 'kubedump' +kubedump-addon 143m +``` + +Now, you can view all `BackupTasks` and `RestoreTasks` of the `pvc-addon` addon using the following command + +```bash +❯ kubectl get addon kubedump-addon -o=jsonpath='{.spec.backupTasks[*].name}'| tr ' ' '\n' +manifest-backup +``` + +#### Prepare Backend + +Now, we are going to store our backed-up data into a GCS bucket using KubeStash. We have to create a Secret and a `BackupStorage` object with access credentials and backend information respectively. If you want to use a different backend, please read the respective backend configuration doc from [here](/docs/guides/backends/overview/index.md). + +> For GCS backend, if the bucket does not exist, KubeStash needs `Storage Object Admin` role permissions to create the bucket. For more details, please check the following [guide](/docs/guides/backends/gcs/index.md). + +**Create Storage Secret:** + +At first, let's create a secret called `gcs-secret` with access credentials to our desired GCS bucket, + +```bash +$ echo -n '' > GOOGLE_PROJECT_ID +$ cat /path/to/downloaded/sa_key_file.json > GOOGLE_SERVICE_ACCOUNT_JSON_KEY +$ kubectl create secret generic -n demo gcs-secret \ + --from-file=./GOOGLE_PROJECT_ID \ + --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY +secret/gcs-secret created +``` + +**Create BackupStorage:** + +Now, create a `BackupStorage` custom resource specifying the desired bucket, and directory inside the bucket where the backed up data will be stored. + +Below is the YAML of `BackupStorage` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut +``` + +Let's create the `BackupStorage` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/kubedump/application/examples/backupstorage.yaml +backupstorage.storage.kubestash.com/gcs-repo created +``` + +Now, we also have to create another secret with an encryption key `RESTIC_PASSWORD` for Restic. This secret will be used by Restic for both encrypting and decrypting the backup data during backup & restore. + +**Create Restic Repository Secret:** + +Let's create a Secret named `encryption-secret` with an encryption key `RESTIC_PASSWORD` for Restic. + +```bash +$ echo -n 'changeit' > RESTIC_PASSWORD +$ kubectl create secret generic -n demo encryption-secret \ + --from-file=./RESTIC_PASSWORD +secret/encryption-secret created +``` + +Now, we have to create a `RetentionPolicy` custom resource which specifies how the old `Snapshots` should be cleaned up. + +**Create RetentionPolicy:** + +Below is the YAML of the `RetentionPolicy` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same +``` + +Let's create the `RetentionPolicy` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/kubedump/application/examples/retentionpolicy.yaml +retentionpolicy.storage.kubestash.com/demo-retention created +``` + +#### Create RBAC + +The `kubedump` plugin requires read permission for the application resources. By default, KubeStash does not grant such permissions. We have to provide the necessary permissions manually. + +Here, is the YAML of the `ServiceAccount`, `ClusterRole`, and `ClusterRoleBinding` that we are going to use for granting the necessary permissions. + +```yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: cluster-resource-reader + namespace: kubestash +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: cluster-resource-reader +rules: + - apiGroups: ["*"] + resources: ["*"] + verbs: ["get","list"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: cluster-resource-reader +subjects: + - kind: ServiceAccount + name: cluster-resource-reader + namespace: kubestash +roleRef: + kind: ClusterRole + name: cluster-resource-reader + apiGroup: rbac.authorization.k8s.io +``` + +Here, we have give permission to read all the cluster resources. You can restrict this permission to a particular application resources only. + +Let's create the RBAC resources we have shown above, + +```bash +❯ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/kubedump/application/examples/rbac.yaml +serviceaccount/cluster-resource-reader created +clusterrole.rbac.authorization.k8s.io/cluster-resource-reader created +clusterrolebinding.rbac.authorization.k8s.io/cluster-resource-reader created +``` + +Now, we are ready for backup. In the next section, we are going to schedule a backup for our cluster resources. + +### Backup + +To schedule a backup, we have to create a `BackupConfiguration` object. Then KubeStash will create a CronJob to periodically backup the database. + +At first, lets list available Deployment in `kubestash` namespace. + +```bash +❯ kubectl get deployments -n kubestash +NAME READY UP-TO-DATE AVAILABLE AGE +kubestash-kubestash-operator 1/1 1 1 19h +``` + +Here, we are going to setup backup YAMLs for `kubestash-kubestash-operator` Deployment. + +#### Create BackupConfiguration + +Below is the YAML for `BackupConfiguration` object we care going to use to backup the YAMLs of the cluster resources, + +```yaml +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: application-manifest-backup + namespace: demo +spec: + target: + apiGroup: apps + kind: Deployment + name: kubestash-kubestash-operator + namespace: kubestash + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /deployment-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + params: + includeDependants: "true" + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader +``` + +Here, + +- `spec.target` specifies the backup target. `apiGroup`, `kind`, `namespace` and `name` refers to the `apiGroup`, `kind`, `namespace` and `name` of the targeted workload respectively. +- `spec.sessions[*].scheduler.schedule` specifies a [cron expression](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) indicates that `BackupSession` will be created at 5 minute interval. +- `spec.sessions[*].repositories[*].name` specifies the name of the `Repository` object that holds the backend information. +- `spec.sessions[*].repositories[*].backend` specifies the name of the backend that holds the `BackupStorage` information. +- `spec.sessions[*].repositories[*].directory` specifies the path of the `Repository` where the backed up data will be stored. +- `spec.sessions[*].repositories[*].encryptionSecret` specifies the encryption secret for `Restic Repository` which will be used to encrypting the backup data. +- `spec.sessions[*].addon.name` specifies the name of the `Addon` object that specifies addon configuration that will be used to perform backup of a stand-alone PVC. +- `spec.sessions[*].addon.tasks[*].name` specifies the name of the `Task` that holds the `Function` and their order of execution to perform backup of a stand-alone PVC. +- `spec.sessions[*].addon.jobTemplate.runtimeSettings.pod.serviceAccountName` specifies the ServiceAccount name that we have created earlier with cluster-wide resource reading permission. + +Let's create the `BackupConfiguration` object we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/kubedump/application/examples/backupconfiguration.yaml +backupconfiguration.core.kubestash.com/application-manifest-backup created +``` + +#### Verify Backup Setup Successful + +If everything goes well, the phase of the `BackupConfiguration` should be `Ready`. The `Ready` phase indicates that the backup setup is successful. Let's verify the `Phase` of the BackupConfiguration, + +```bash +❯ kubectl get backupconfiguration -n demo +NAME PHASE PAUSED AGE +application-manifest-backup Ready 19s +``` + +**Verify Repository:** + +Verify that KubeStash has created `Repositories` that holds the `BackupStorage` information by the following command, + +```bash +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository Ready 28s +``` + +**Verify CronJob:** + +Verify that KubeStash has created a CronJob to trigger a periodic backup of the targeted PVC by the following command, + +```bash +$ kubectl get cronjob -n demo +NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE +trigger-application-manifest-backup-frequent-backup */5 * * * * False 0 45s +``` + +**Wait for BackupSession:** + +Now, wait for the next backup schedule. You can watch for `BackupSession` crd using the following command, + +```bash +$ watch -n 1 kubectl get backupsession -n demo -l=kubestash.com/invoker-name=application-manifest-backup +Every 1.0s: kubectl get backupsession -n demo -l=kubestash.com/invoker-name=application-manifest-backup anisur: Fri Feb 23 14:35:27 2024 + +NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE +application-manifest-backup-frequent-backup-1708677300 BackupConfiguration application-manifest-backup Succeeded 27s + +``` + +**Verify Backup:** + +When backup session is created, kubestash operator creates `Snapshot` which represents the state of backup run for each `Repository` which are provided in `BackupConfiguration`. + +Run the following command to check `Snapshot` phase, + +```bash +$ kubectl get snapshots -n demo +NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE AGE +gcs-repository-application-manifckup-frequent-backup-1708677300 gcs-repository frequent-backup 2024-02-23T08:35:00Z Delete Succeeded 43s +``` + +When backup session is completed, KubeStash will update the respective `Repository` to reflect the latest state of backed up data. + +Run the following command to check if a backup snapshot has been stored in the backend, + +```bash +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository true 1 2.262 KiB Ready 103s 8m +``` + +From the output above, we can see that `1` snapshot has been stored in the backend specified by Repository `gcs-repository`. + +If we navigate to `kubestash-qa/demo/deployment-manifests/repository/v1/frequent-backup/manifest` directory of our GCS bucket, we are going to see that the snapshot has been stored there. + +
+  Backup YAMLs data of an Application in GCS storage +
Fig: Backup YAMLs data of an Application in GCS backend
+
+ +> KubeStash keeps all backup data encrypted. So, snapshot files in the bucket will not contain any meaningful data until they are decrypted. + + +## Restore + +KubeStash does not provide any automatic mechanism to restore the cluster resources from the backed-up YAMLs. Your application might be managed by Helm or by an operator. In such cases, just applying the YAMLs is not enough to restore the application. Furthermore, there might be an order issue. Some resources must be applied before others. It is difficult to generalize and codify various application-specific logic. + +Therefore, it is the user's responsibility to download the backed-up YAMLs and take the necessary steps based on his application to restore it properly. + +### Download the YAMLs + +KubeStash provides a [kubectl plugin](https://stash.run/docs/v2022.05.12/guides/cli/cli/#download-snapshots) for making it easy to download a snapshot locally. + +Now, let's download the latest Snapshot from our backed-up data into the `$HOME/Downloads/stash/applications/kube-system/stash-enterprise` folder of our local machine. + +```bash +❯ kubectl kubestash download gcs-repository-application-manifckup-frequent-backup-1708677300 --namespace=demo --destination=$HOME/Downloads/kubestash/applications/kubestash/kubestash-kubestash-operator +``` + +Now, lets use [tree](https://linux.die.net/man/1/tree) command to inspect downloaded YAMLs files. + +```bash +❯ tree $HOME/Downloads/kubestash/applications/kubestash/kubestash-kubestash-operator +/home/anisur/Downloads/kubestash/applications/kubestash/kubestash-kubestash-operator +└── gcs-repository-application-manifckup-frequent-backup-1708677300 + └── manifest + └── tmp + └── manifest + ├── kubestash-kubestash-operator.yaml + └── ReplicaSet + └── kubestash-kubestash-operator-7bc7564b69 + ├── kubestash-kubestash-operator-7bc7564b69.yaml + └── Pod + └── kubestash-kubestash-operator-7bc7564b69-2frcq + └── kubestash-kubestash-operator-7bc7564b69-2frcq.yaml + +8 directories, 3 files + +``` + +As you can see that the Deployment has been backed up along with it's ReplicaSet and Pods. + +Let's inspect the YAML of `kubestash-kubestash-operator.yaml` file, + +```yaml +❯ cat /home/anisur/Downloads/kubestash/applications/kubestash/kubestash-kubestash-operator/gcs-repository-application-manifckup-frequent-backup-1708677300/manifest/tmp/manifest/kubestash-kubestash-operator.yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + annotations: + deployment.kubernetes.io/revision: "1" + meta.helm.sh/release-name: kubestash + meta.helm.sh/release-namespace: kubestash + labels: + app.kubernetes.io/instance: kubestash + app.kubernetes.io/managed-by: Helm + app.kubernetes.io/name: kubestash-operator + app.kubernetes.io/version: v0.5.0 + helm.sh/chart: kubestash-operator-v0.5.0 + name: kubestash-kubestash-operator + namespace: kubestash +spec: + progressDeadlineSeconds: 600 + replicas: 1 + revisionHistoryLimit: 10 + selector: + matchLabels: + app.kubernetes.io/instance: kubestash + app.kubernetes.io/name: kubestash-operator + strategy: + rollingUpdate: + maxSurge: 25% + maxUnavailable: 25% + type: RollingUpdate + template: + metadata: + annotations: + checksum/apiregistration.yaml: 62af9aba894e98a7dc849e63a31ef52d6c3b459df8d2242e71cc72e458553d11 + labels: + app.kubernetes.io/instance: kubestash + app.kubernetes.io/name: kubestash-operator + spec: + containers: + - args: + - run + - --config=/var/config/config.yaml + - --license-file=/var/run/secrets/appscode/license/key.txt + env: + - name: POD_NAME + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + apiVersion: v1 + fieldPath: metadata.namespace + image: ghcr.io/kubestash/kubestash:v0.5.0 + imagePullPolicy: IfNotPresent + name: operator + ports: + - containerPort: 9443 + name: webhook-server + protocol: TCP + resources: + requests: + cpu: 100m + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 65534 + terminationMessagePolicy: File + volumeMounts: + - mountPath: /kubestash-tmp + name: kubestash-tmp-volume + - mountPath: /var/serving-cert + name: serving-cert + readOnly: true + - mountPath: /var/config + name: config + - mountPath: /var/run/secrets/appscode/license + name: license + - args: + - --secure-listen-address=0.0.0.0:8443 + - --upstream=http://127.0.0.1:8080/ + - --logtostderr=true + - --v=10 + image: ghcr.io/appscode/kube-rbac-proxy:v0.11.0 + imagePullPolicy: IfNotPresent + name: kube-rbac-proxy + ports: + - containerPort: 8443 + name: https + protocol: TCP + resources: {} + securityContext: + allowPrivilegeEscalation: false + capabilities: + drop: + - ALL + readOnlyRootFilesystem: true + runAsNonRoot: true + runAsUser: 65534 + terminationMessagePolicy: File + nodeSelector: + kubernetes.io/os: linux + restartPolicy: Always + schedulerName: default-scheduler + securityContext: + fsGroup: 65535 + serviceAccount: kubestash-kubestash-operator + serviceAccountName: kubestash-kubestash-operator + volumes: + - emptyDir: {} + name: kubestash-tmp-volume + - name: serving-cert + secret: + defaultMode: 420 + secretName: kubestash-kubestash-operator-webhook-cert + - configMap: + defaultMode: 420 + name: kubestash-kubestash-operator-config + name: config + - name: license + secret: + defaultMode: 420 + secretName: kubestash-kubestash-operator-license +``` + +Now, you can use these YAML files to re-create your desired application. + +## Cleanup + +To cleanup the Kubernetes resources created by this tutorial, run: + +```bash +kubectl delete -n demo backupconfiguration application-manifest-backup +kubectl delete -n kubestash serviceaccount cluster-resource-reader +kubectl delete clusterrole cluster-resource-reader +kubectl delete clusterrolebinding cluster-resource-reader +kubectl delete retentionPolicy -n demo demo-retention +kubectl delete -n demo backupstorage gcs-storage +kubectl delete secret -n demo encryption-secret +kubectl delete secret -n demo gcs-secret +``` diff --git a/docs/guides/kubedump/cluster/examples/backupconfiguration.yaml b/docs/guides/kubedump/cluster/examples/backupconfiguration.yaml new file mode 100644 index 0000000..9ed1ea8 --- /dev/null +++ b/docs/guides/kubedump/cluster/examples/backupconfiguration.yaml @@ -0,0 +1,36 @@ +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: cluster-resources-backup + namespace: demo +spec: + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /cluster-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader \ No newline at end of file diff --git a/docs/guides/kubedump/cluster/examples/backupstorage.yaml b/docs/guides/kubedump/cluster/examples/backupstorage.yaml new file mode 100644 index 0000000..0461b26 --- /dev/null +++ b/docs/guides/kubedump/cluster/examples/backupstorage.yaml @@ -0,0 +1,17 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/kubedump/cluster/examples/retentionpolicy.yaml b/docs/guides/kubedump/cluster/examples/retentionpolicy.yaml new file mode 100644 index 0000000..3af9037 --- /dev/null +++ b/docs/guides/kubedump/cluster/examples/retentionpolicy.yaml @@ -0,0 +1,15 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same \ No newline at end of file diff --git a/docs/guides/kubedump/cluster/images/cluster_manifests_backup.png b/docs/guides/kubedump/cluster/images/cluster_manifests_backup.png new file mode 100644 index 0000000..830e412 Binary files /dev/null and b/docs/guides/kubedump/cluster/images/cluster_manifests_backup.png differ diff --git a/docs/guides/kubedump/cluster/index.md b/docs/guides/kubedump/cluster/index.md new file mode 100644 index 0000000..1412d1a --- /dev/null +++ b/docs/guides/kubedump/cluster/index.md @@ -0,0 +1,492 @@ +--- +title: Backup resource YAMLs of entire cluster | KubeStash +description: Take backup of cluster resources YAMLs using KubeStash +menu: + docs_{{ .version }}: + identifier: kubestash-kubedump-cluster + name: Cluster Backup + parent: kubestash-kubedump + weight: 20 +product_name: kubestash +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +# Backup resource YAMLs of entire cluster using KubeStash + +KubeStash `{{< param "info.version" >}}` supports taking backup of the resource YAMLs using `kubedump` plugin. This guide will show you how you can take a backup of the resource YAMLs of your entire cluster using KubeStash. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the `kubectl` command-line tool must be configured to communicate with your cluster. +- Install KubeStash in your cluster following the steps [here](/docs/setup/install/kubestash/index.md). +- Install KubeStash `kubectl` plugin in your local machine following the steps [here](/docs/setup/install/kubectl-plugin/index.md). +- If you are not familiar with how KubeStash backup the resource YAMLs, please check the following guide [here](/docs/guides/kubedump/overview/index.md). + +You have to be familiar with the following custom resources: + +- [BackupStorage](/docs/concepts/crds/backupstorage/index.md) +- [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md) +- [BackupSession](/docs/concepts/crds/backupsession/index.md) +- [SnapShot](/docs/concepts/crds/snapshot/index.md) +- [RestoreSession](/docs/concepts/crds/restoresession/index.md) +- [RetentionPolicy](/docs/concepts/crds/retentionpolicy/index.md) + +To keep things isolated, we are going to use a separate namespace called `demo` throughout this tutorial. Create the `demo` namespace if you haven't created it already. + +```bash +$ kubectl create ns demo +namespace/demo created +``` + +> Note: YAML files used in this tutorial are stored [here](https://github.com/kubestash/docs/tree/{{< param "info.version" >}}/docs/guides/kubedump/cluster/examples). + +### Prepare for Backup + +In this section, we are going to configure a backup for all the resource YAMLs of `kube-system` namespace. + +#### Ensure `kubedump` Addon + +**Verify necessary Addons and Functions:** + +When you install KubeStash, it automatically creates the necessary `Addon` and `Function` to backup Kubernetes Resource YAML. + +Let's verify that KubeStash has created the necessary `Function` to backup Kubernetes Resource YAML by the following command, + +```bash +$ kubectl get functions | grep 'kubedump' +kubedump-backup 142m +``` + +Also, verify that the necessary `Addon` has been created, + +```bash +❯ kubectl get addons | grep 'kubedump' +kubedump-addon 143m +``` + +Now, you can view all `BackupTasks` and `RestoreTasks` of the `pvc-addon` addon using the following command + +```bash +❯ kubectl get addon kubedump-addon -o=jsonpath='{.spec.backupTasks[*].name}'| tr ' ' '\n' +manifest-backup +``` + +#### Prepare Backend + +Now, we are going to store our backed-up data into a GCS bucket using KubeStash. We have to create a Secret and a `BackupStorage` object with access credentials and backend information respectively. If you want to use a different backend, please read the respective backend configuration doc from [here](/docs/guides/backends/overview/index.md). + +> For GCS backend, if the bucket does not exist, KubeStash needs `Storage Object Admin` role permissions to create the bucket. For more details, please check the following [guide](/docs/guides/backends/gcs/index.md). + +**Create Storage Secret:** + +At first, let's create a secret called `gcs-secret` with access credentials to our desired GCS bucket, + +```bash +$ echo -n '' > GOOGLE_PROJECT_ID +$ cat /path/to/downloaded/sa_key_file.json > GOOGLE_SERVICE_ACCOUNT_JSON_KEY +$ kubectl create secret generic -n demo gcs-secret \ + --from-file=./GOOGLE_PROJECT_ID \ + --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY +secret/gcs-secret created +``` + +**Create BackupStorage:** + +Now, create a `BackupStorage` custom resource specifying the desired bucket, and directory inside the bucket where the backed up data will be stored. + +Below is the YAML of `BackupStorage` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut +``` + +Let's create the `BackupStorage` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/kubedump/cluster/examples/backupstorage.yaml +backupstorage.storage.kubestash.com/gcs-repo created +``` + +Now, we also have to create another secret with an encryption key `RESTIC_PASSWORD` for Restic. This secret will be used by Restic for both encrypting and decrypting the backup data during backup & restore. + +**Create Restic Repository Secret:** + +Let's create a Secret named `encryption-secret` with an encryption key `RESTIC_PASSWORD` for Restic. + +```bash +$ echo -n 'changeit' > RESTIC_PASSWORD +$ kubectl create secret generic -n demo encryption-secret \ + --from-file=./RESTIC_PASSWORD +secret/encryption-secret created +``` + +Now, we have to create a `RetentionPolicy` custom resource which specifies how the old `Snapshots` should be cleaned up. + +**Create RetentionPolicy:** + +Below is the YAML of the `RetentionPolicy` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same +``` + +Let's create the `RetentionPolicy` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/kubedump/cluster/examples/retentionpolicy.yaml +retentionpolicy.storage.kubestash.com/demo-retention created +``` + + +#### Create RBAC + +The `kubedump` plugin requires read permission for all the cluster resources. By default, KubeStash does not grant such cluster-wide permissions. We have to provide the necessary permissions manually. + +Here, is the YAML of the `ServiceAccount`, `ClusterRole`, and `ClusterRoleBinding` that we are going to use for granting the necessary permissions. + +```yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: cluster-resource-reader + namespace: demo +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: cluster-resource-reader +rules: +- apiGroups: ["*"] + resources: ["*"] + verbs: ["get","list"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: cluster-resource-reader +subjects: +- kind: ServiceAccount + name: cluster-resource-reader + namespace: demo +roleRef: + kind: ClusterRole + name: cluster-resource-reader + apiGroup: rbac.authorization.k8s.io +``` + +Let's create the RBAC resources we have shown above, + +```bash +❯ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/addons/kubedump/cluster/examples/rbac.yaml +serviceaccount/cluster-resource-reader created +clusterrole.rbac.authorization.k8s.io/cluster-resource-reader created +clusterrolebinding.rbac.authorization.k8s.io/cluster-resource-reader created +``` + +Now, we are ready for backup. In the next section, we are going to schedule a backup for our cluster resources. + +### Backup + +To schedule a backup, we have to create a `BackupConfiguration` object. Then KubeStash will create a CronJob to periodically backup the database. + +#### Create BackupConfiguration + +Below is the YAML for `BackupConfiguration` object we care going to use to backup the YAMLs of the cluster resources, + +```yaml +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: cluster-resources-backup + namespace: demo +spec: + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /cluster-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader +``` + +Here, + +- `spec.target` specifies the backup target. `apiGroup`, `kind`, `namespace` and `name` refers to the `apiGroup`, `kind`, `namespace` and `name` of the targeted workload respectively. +- `spec.sessions[*].scheduler.schedule` specifies a [cron expression](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) indicates that `BackupSession` will be created at 5 minute interval. +- `spec.sessions[*].repositories[*].name` specifies the name of the `Repository` object that holds the backend information. +- `spec.sessions[*].repositories[*].backend` specifies the name of the backend that holds the `BackupStorage` information. +- `spec.sessions[*].repositories[*].directory` specifies the path of the `Repository` where the backed up data will be stored. +- `spec.sessions[*].repositories[*].encryptionSecret` specifies the encryption secret for `Restic Repository` which will be used to encrypting the backup data. +- `spec.sessions[*].addon.name` specifies the name of the `Addon` object that specifies addon configuration that will be used to perform backup of a stand-alone PVC. +- `spec.sessions[*].addon.tasks[*].name` specifies the name of the `Task` that holds the `Function` and their order of execution to perform backup of a stand-alone PVC. + +Let's create the `BackupConfiguration` object we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/kubedump/cluster/examples/backupconfiguration.yaml +backupconfiguration.core.kubestash.com/cluster-resources-backup created +``` + +#### Verify Backup Setup Successful + +If everything goes well, the phase of the `BackupConfiguration` should be `Ready`. The `Ready` phase indicates that the backup setup is successful. Let's verify the `Phase` of the BackupConfiguration, + +```bash +❯ kubectl get backupconfiguration -n demo +NAME PHASE PAUSED AGE +cluster-resources-backup Ready 79s + +``` + +**Verify Repository:** + +Verify that KubeStash has created `Repositories` that holds the `BackupStorage` information by the following command, + +```bash +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository Ready 28s +``` + +**Verify CronJob:** + +Verify that KubeStash has created a CronJob to trigger a periodic backup of the targeted PVC by the following command, + +```bash +$ kubectl get cronjob -n demo +NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE +trigger-cluster-resources-backup-frequent-backup */5 * * * * False 0 45s +``` + +**Wait for BackupSession:** + +Now, wait for the next backup schedule. You can watch for `BackupSession` crd using the following command, + +```bash +$ watch -n 1 kubectl get backupsession -n demo -l=kubestash.com/invoker-name=cluster-resources-backup anisur: Fri Feb 23 19:46:11 2024 + +NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE +cluster-resources-backup-frequent-backup-1708694700 BackupConfiguration cluster-resources-backup Succeeded 21m +``` + +**Verify Backup:** + +When backup session is created, kubestash operator creates `Snapshot` which represents the state of backup run for each `Repository` which are provided in `BackupConfiguration`. + +Run the following command to check `Snapshot` phase, + +```bash +$ kubectl get snapshots -n demo +NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE AGE +gcs-repository-cluster-resourcesckup-frequent-backup-1708694700 gcs-repository frequent-backup 2024-02-23T13:25:00Z Delete Succeeded 22m +``` + +When backup session is completed, KubeStash will update the respective `Repository` to reflect the latest state of backed up data. + +Run the following command to check if a backup snapshot has been stored in the backend, + +```bash +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository true 1 2.262 KiB Ready 103s 8m +``` + +From the output above, we can see that `1` snapshot has been stored in the backend specified by Repository `gcs-repository`. + +Now, if we navigate to the GCS bucket, we will see the backed up data has been stored in `/manifest/cluster` directory as specified by `.spec.backend.gcs.prefix` field of the `Repository` object. + +
+ Backup data in GCS Bucket +
Fig: Backup data in GCS Bucket
+
+ +> Note: KubeStash keeps all the backed-up data encrypted. So, data in the backend will not make any sense until they are decrypted. + +## Restore + +KubeStash does not provide any automatic mechanism to restore the cluster resources from the backed-up YAMLs. Your application might be managed by Helm or by an operator. In such cases, just applying the YAMLs is not enough to restore the application. Furthermore, there might be an order issue. Some resources must be applied before others. It is difficult to generalize and codify various application-specific logic. + +Therefore, it is the user's responsibility to download the backed-up YAMLs and take the necessary steps based on his application to restore it properly. + +### Download the YAMLs + +KubeStash provides a [kubectl plugin](/docs/guides/cli/cli/#download-snapshots) for making it easy to download a snapshot locally. + +Now, let's download the latest Snapshot from our backed-up data into the `$HOME/Downloads/kubestash` folder of our local machine. + +```bash +❯ kubectl kubestash download --namespace=demo gcs-repository-cluster-resourcesckup-frequent-backup-1708694700 --destination=$HOME/Downloads/kubestash/cluster +``` + +Now, lets use [tree](https://linux.die.net/man/1/tree) command to inspect downloaded YAMLs files. + +```bash +❯ /home/anisur/Downloads/kubestash/cluster/gcs-repository-cluster-resourcesckup-frequent-backup-1708694700 +└── manifest + └── tmp + └── manifest + ├── global + │   ├── Addon + │   │   ├── kubedump-addon.yaml + │   │   ├── pvc-addon.yaml + │   │   └── workload-addon.yaml + │   ├── APIService + │   │   ├── v1.admissionregistration.k8s.io.yaml + └── namespaces + ├── default + │   ├── ConfigMap + │   │   └── kube-root-ca.crt.yaml + │   ├── Endpoints + │   │   └── kubernetes.yaml + │   ├── EndpointSlice + │   │   └── kubernetes.yaml + │   ├── Service + │   │   └── kubernetes.yaml + │   └── ServiceAccount + │   └── default.yaml + ├── demo + │   ├── BackupConfiguration + │   │   └── cluster-resources-backup.yaml + │   ├── BackupSession + │   │   └── cluster-resources-backup-frequent-backup-1708694700.yaml + │   ├── BackupStorage + │   │   └── gcs-storage.yaml + │   ├── ConfigMap + │   │   └── kube-root-ca.crt.yaml + │   ├── ReplicaSet + │   │   └── coredns-565d847f94.yaml + │   ├── Role + │   │   ├── extension-apiserver-authentication-reader.yaml + │   │   ├── kubeadm:kubelet-config.yaml + │   ├── RoleBinding + │   │   ├── kubeadm:kubelet-config.yaml + │   │   ├── kubeadm:nodes-kubeadm-config.yaml + │   ├── Service + │   │   └── kube-dns.yaml + │   └── ServiceAccount + │   ├── attachdetach-controller.yaml + │   ├── bootstrap-signer.yaml + │   ├── certificate-controller.yaml + │   ├── cluster-resource-reader.yaml + └── local-path-storage + ├── ConfigMap + │   ├── kube-root-ca.crt.yaml + │   └── local-path-config.yaml + ├── Deployment + │   └── local-path-provisioner.yaml + +84 directories, 405 files +``` + +Here, the non-namespaced resources have been grouped under the `global` directory and the namespaced resources have been grouped inside the namespace specific folder under the `namespaces` directory. + +Let's inspect the YAML of `kubeadm-config.yaml` file under `kube-system` namespace. + +```yaml +❯ cat /home/anisur/Downloads/kubestash/cluster/gcs-repository-cluster-resourcesckup-frequent-backup-1708694700/manifest/tmp/manifest/namespaces/kube-system/ConfigMap/kubeadm-config.yaml +apiVersion: v1 +data: + ClusterConfiguration: | + apiServer: + certSANs: + - localhost + - 127.0.0.1 + extraArgs: + authorization-mode: Node,RBAC + runtime-config: "" + timeoutForControlPlane: 4m0s + apiVersion: kubeadm.k8s.io/v1beta3 + certificatesDir: /etc/kubernetes/pki + clusterName: kind + controlPlaneEndpoint: kind-control-plane:6443 + controllerManager: + extraArgs: + enable-hostpath-provisioner: "true" + dns: {} + etcd: + local: + dataDir: /var/lib/etcd + imageRepository: registry.k8s.io + kind: ClusterConfiguration + kubernetesVersion: v1.25.2 + networking: + dnsDomain: cluster.local + podSubnet: 10.244.0.0/16 + serviceSubnet: 10.96.0.0/16 + scheduler: {} +kind: ConfigMap +metadata: + name: kubeadm-config + namespace: kube-system +``` + +Now, you can use these YAML files to re-create your desired application. + +## Cleanup + +To cleanup the Kubernetes resources created by this tutorial, run: + +```bash +kubectl delete -n demo backupconfiguration cluster-resources-backup +kubectl delete -n kubestash serviceaccount cluster-resource-reader +kubectl delete clusterrole cluster-resource-reader +kubectl delete clusterrolebinding cluster-resource-reader +kubectl delete retentionPolicy -n demo demo-retention +kubectl delete -n demo backupstorage gcs-storage +kubectl delete secret -n demo encryption-secret +kubectl delete secret -n demo gcs-secret +``` diff --git a/docs/guides/kubedump/customization/examples/filter-resources.yaml b/docs/guides/kubedump/customization/examples/filter-resources.yaml new file mode 100644 index 0000000..15b772e --- /dev/null +++ b/docs/guides/kubedump/customization/examples/filter-resources.yaml @@ -0,0 +1,38 @@ +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: kube-system-backup + namespace: demo +spec: + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + params: + labelSelector: "k8s-app=kube-dns" + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader \ No newline at end of file diff --git a/docs/guides/kubedump/customization/examples/passing-args.yaml b/docs/guides/kubedump/customization/examples/passing-args.yaml new file mode 100644 index 0000000..4f22e78 --- /dev/null +++ b/docs/guides/kubedump/customization/examples/passing-args.yaml @@ -0,0 +1,43 @@ +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: application-manifest-backup + namespace: demo +spec: + target: + apiGroup: apps + kind: Deployment + name: kubestash-kubestash-operator + namespace: kubestash + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /deployment-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + params: + sanitize: "false" + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader \ No newline at end of file diff --git a/docs/guides/kubedump/customization/examples/resource-limit.yaml b/docs/guides/kubedump/customization/examples/resource-limit.yaml new file mode 100644 index 0000000..c4a951e --- /dev/null +++ b/docs/guides/kubedump/customization/examples/resource-limit.yaml @@ -0,0 +1,45 @@ +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: kube-system-backup + namespace: demo +spec: + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/2 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + params: + labelSelector: "k8s-app=kube-dns" + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader + resources: + requests: + cpu: "200m" + memory: "1Gi" + limits: + cpu: "200m" + memory: "1Gi" diff --git a/docs/guides/kubedump/customization/examples/specific-user.yaml b/docs/guides/kubedump/customization/examples/specific-user.yaml new file mode 100644 index 0000000..7e58125 --- /dev/null +++ b/docs/guides/kubedump/customization/examples/specific-user.yaml @@ -0,0 +1,41 @@ +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: kube-system-backup + namespace: demo +spec: + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/2 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + params: + labelSelector: "k8s-app=kube-dns" + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader + securityContext: + runAsUser: 0 + runAsGroup: 0 \ No newline at end of file diff --git a/docs/guides/kubedump/customization/index.md b/docs/guides/kubedump/customization/index.md new file mode 100644 index 0000000..8021767 --- /dev/null +++ b/docs/guides/kubedump/customization/index.md @@ -0,0 +1,244 @@ +--- +title: Backup Customization | KubeStash +description: Customizing Backup Process with KubeStash +menu: + docs_{{ .version }}: + identifier: kubestash-kubedump-customization + name: Customizing Backup Process + parent: kubestash-kubedump + weight: 50 +product_name: kubestash +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +# Customizing Backup Process + +KubeStash provides rich customization supports for the backup and restores process to meet the requirements of various cluster configurations. This guide will show you some examples of these customizations. + +In this section, we are going to show you how to customize the backup process. Here, we are going to show some examples of filtering resources using a label selector, running the backup process as a specific user, etc. + +> Note: YAML files used in this tutorial are stored [here](https://github.com/kubestash/docs/tree/{{< param "info.version" >}}/docs/addons/kubedump/customization/examples). + +### Filtering resources + +You can use a label selector to backup the YAML for the resources that have particular labels. You just have to pass the `labelSelector` parameter under `task.params` section with your desired label selector. + +```yaml +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: kube-system-backup + namespace: demo +spec: + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + params: + labelSelector: "k8s-app=kube-dns" + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader +``` + +The above backup process will backup only the resources that has `k8s-app: kube-dns` label. Here, is a sample of the resources backed up by the above BackupConfiguration. + +```bash +❯ tree /home/anisur/Downloads/kubestash/label-selector +/home/anisur/Downloads/kubestash/label-selector +└── gcs-repository-kube-system-backup-frequent-backup-1708926900 + └── manifest + └── tmp + └── manifest + └── namespaces + └── kube-system + ├── Deployment + │   └── coredns.yaml + ├── Endpoints + │   └── kube-dns.yaml + ├── EndpointSlice + │   └── kube-dns-nv9px.yaml + ├── Pod + │   ├── coredns-565d847f94-78r86.yaml + │   └── coredns-565d847f94-zdtcs.yaml + ├── ReplicaSet + │   └── coredns-565d847f94.yaml + └── Service + └── kube-dns.yaml + +12 directories, 7 files +``` + +### Passing arguments + +You can pass arguments to the backup process using `task.params` section. + +The following example shows how passes `sanitize` argument value to `false` which tells the backup process not to remove decorators (i.e. `status`, `managedFields` etc.) from the YAML files. + +```yaml +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: application-manifest-backup + namespace: demo +spec: + target: + apiGroup: apps + kind: Deployment + name: kubestash-kubestash-operator + namespace: kubestash + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /deployment-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + params: + sanitize: "false" + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader +``` + +### Running backup job as a specific user + +If your cluster requires running the backup job as a specific user, you can provide `securityContext` under `runtimeSettings.pod` section. The below example shows how you can run the backup job as the root user. + +```yaml +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: kube-system-backup + namespace: demo +spec: + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/2 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader + securityContext: + runAsUser: 0 + runAsGroup: 0 +``` + +### Specifying Memory/CPU limit/request for the backup job + +If you want to specify the Memory/CPU limit/request for your backup job, you can specify `resources` field under `runtimeSettings.container` section. + +```yaml +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: kube-system-backup + namespace: demo +spec: + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/2 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + params: + labelSelector: "k8s-app=kube-dns" + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader + resources: + requests: + cpu: "200m" + memory: "1Gi" + limits: + cpu: "200m" + memory: "1Gi" +``` \ No newline at end of file diff --git a/docs/guides/kubedump/namespace/examples/backupconfiguration.yaml b/docs/guides/kubedump/namespace/examples/backupconfiguration.yaml new file mode 100644 index 0000000..612a101 --- /dev/null +++ b/docs/guides/kubedump/namespace/examples/backupconfiguration.yaml @@ -0,0 +1,40 @@ +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: kube-system-backup + namespace: demo +spec: + target: + apiGroup: "" + kind: Namespace + name: kube-system + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader \ No newline at end of file diff --git a/docs/guides/kubedump/namespace/examples/backupstorage.yaml b/docs/guides/kubedump/namespace/examples/backupstorage.yaml new file mode 100644 index 0000000..0461b26 --- /dev/null +++ b/docs/guides/kubedump/namespace/examples/backupstorage.yaml @@ -0,0 +1,17 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/addons/kubedump/application/examples/rbac.yaml b/docs/guides/kubedump/namespace/examples/rbac.yaml similarity index 100% rename from docs/addons/kubedump/application/examples/rbac.yaml rename to docs/guides/kubedump/namespace/examples/rbac.yaml diff --git a/docs/guides/kubedump/namespace/examples/retentionpolicy.yaml b/docs/guides/kubedump/namespace/examples/retentionpolicy.yaml new file mode 100644 index 0000000..3af9037 --- /dev/null +++ b/docs/guides/kubedump/namespace/examples/retentionpolicy.yaml @@ -0,0 +1,15 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same \ No newline at end of file diff --git a/docs/guides/kubedump/namespace/images/namespace_manifests_backup.png b/docs/guides/kubedump/namespace/images/namespace_manifests_backup.png new file mode 100644 index 0000000..5777f78 Binary files /dev/null and b/docs/guides/kubedump/namespace/images/namespace_manifests_backup.png differ diff --git a/docs/guides/kubedump/namespace/index.md b/docs/guides/kubedump/namespace/index.md new file mode 100644 index 0000000..44555f7 --- /dev/null +++ b/docs/guides/kubedump/namespace/index.md @@ -0,0 +1,491 @@ +--- +title: Backup resource YAMLs of a Namespace | KubeStash +description: Take backup of resources YAMLs of a Namespace using KubeStash +menu: + docs_{{ .version }}: + identifier: kubestash-kubedump-namespace + name: Namespace Backup + parent: kubestash-kubedump + weight: 30 +product_name: kubestash +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +# Backup resource YAMLs of a Namespace using KubeStash + +KubeStash `{{< param "info.version" >}}` supports taking backup of the resource YAMLs using `kubedump` plugin. This guide will show you how you can take a backup of the resource YAMLs of a particular namespace using KubeStash. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the `kubectl` command-line tool must be configured to communicate with your cluster. +- Install KubeStash in your cluster following the steps [here](/docs/setup/install/kubestash/index.md). +- Install KubeStash `kubectl` plugin in your local machine following the steps [here](/docs/setup/install/kubectl-plugin/index.md). +- If you are not familiar with how KubeStash backup the resource YAMLs, please check the following guide [here](/docs/guides/kubedump/overview/index.md). + +You have to be familiar with the following custom resources: + +- [BackupStorage](/docs/concepts/crds/backupstorage/index.md) +- [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md) +- [BackupSession](/docs/concepts/crds/backupsession/index.md) +- [SnapShot](/docs/concepts/crds/snapshot/index.md) +- [RestoreSession](/docs/concepts/crds/restoresession/index.md) +- [RetentionPolicy](/docs/concepts/crds/retentionpolicy/index.md) + +To keep things isolated, we are going to use a separate namespace called `demo` throughout this tutorial. Create the `demo` namespace if you haven't created it already. + +```bash +$ kubectl create ns demo +namespace/demo created +``` + +> Note: YAML files used in this tutorial are stored [here](https://github.com/kubestash/docs/tree/{{< param "info.version" >}}/docs/guides/kubedump/namespace/examples). + +### Prepare for Backup + +In this section, we are going to configure a backup for all the resource YAMLs of `kube-system` namespace. + +#### Ensure `kubedump` Addon + +**Verify necessary Addons and Functions:** + +When you install KubeStash, it automatically creates the necessary `Addon` and `Function` to backup Kubernetes Resource YAML. + +Let's verify that KubeStash has created the necessary `Function` to backup Kubernetes Resource YAML by the following command, + +```bash +$ kubectl get functions | grep 'kubedump' +kubedump-backup 142m +``` + +Also, verify that the necessary `Addon` has been created, + +```bash +❯ kubectl get addons | grep 'kubedump' +kubedump-addon 143m +``` + +Now, you can view all `BackupTasks` and `RestoreTasks` of the `pvc-addon` addon using the following command + +```bash +❯ kubectl get addon kubedump-addon -o=jsonpath='{.spec.backupTasks[*].name}'| tr ' ' '\n' +manifest-backup +``` + +#### Prepare Backend + +Now, we are going to store our backed-up data into a GCS bucket using KubeStash. We have to create a Secret and a `BackupStorage` object with access credentials and backend information respectively. If you want to use a different backend, please read the respective backend configuration doc from [here](/docs/guides/backends/overview/index.md). + +> For GCS backend, if the bucket does not exist, KubeStash needs `Storage Object Admin` role permissions to create the bucket. For more details, please check the following [guide](/docs/guides/backends/gcs/index.md). + +**Create Storage Secret:** + +At first, let's create a secret called `gcs-secret` with access credentials to our desired GCS bucket, + +```bash +$ echo -n '' > GOOGLE_PROJECT_ID +$ cat /path/to/downloaded/sa_key_file.json > GOOGLE_SERVICE_ACCOUNT_JSON_KEY +$ kubectl create secret generic -n demo gcs-secret \ + --from-file=./GOOGLE_PROJECT_ID \ + --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY +secret/gcs-secret created +``` + +**Create BackupStorage:** + +Now, create a `BackupStorage` custom resource specifying the desired bucket, and directory inside the bucket where the backed up data will be stored. + +Below is the YAML of `BackupStorage` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut +``` + +Let's create the `BackupStorage` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/kubedump/namespace/examples/backupstorage.yaml +backupstorage.storage.kubestash.com/gcs-repo created +``` + +Now, we also have to create another secret with an encryption key `RESTIC_PASSWORD` for Restic. This secret will be used by Restic for both encrypting and decrypting the backup data during backup & restore. + +**Create Restic Repository Secret:** + +Let's create a Secret named `encryption-secret` with an encryption key `RESTIC_PASSWORD` for Restic. + +```bash +$ echo -n 'changeit' > RESTIC_PASSWORD +$ kubectl create secret generic -n demo encryption-secret \ + --from-file=./RESTIC_PASSWORD +secret/encryption-secret created +``` + +Now, we have to create a `RetentionPolicy` custom resource which specifies how the old `Snapshots` should be cleaned up. + +**Create RetentionPolicy:** + +Below is the YAML of the `RetentionPolicy` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same +``` + +Let's create the `RetentionPolicy` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/kubedump/namespace/examples/retentionpolicy.yaml +retentionpolicy.storage.kubestash.com/demo-retention created +``` + +#### Create RBAC + +The `kubedump` plugin requires read permission for all the resources of the desired namespace. By default, KubeStash does not grant such permissions. We have to provide the necessary permissions manually. + +Here, is the YAML of the `ServiceAccount`, `ClusterRole`, and `ClusterRoleBinding` that we are going to use for granting the necessary permissions. + +```yaml +apiVersion: v1 +kind: ServiceAccount +metadata: + name: cluster-resource-reader + namespace: demo +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + name: cluster-resource-reader +rules: +- apiGroups: ["*"] + resources: ["*"] + verbs: ["get","list"] +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: cluster-resource-reader +subjects: +- kind: ServiceAccount + name: cluster-resource-reader + namespace: demo +roleRef: + kind: ClusterRole + name: cluster-resource-reader + apiGroup: rbac.authorization.k8s.io +``` + +Here, we have give permission to read all the cluster resources. You can restrict this permission to a particular namespace only. + +Let's create the RBAC resources we have shown above, + +```bash +❯ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/kubedump/namespace/examples/rbac.yaml +serviceaccount/cluster-resource-reader created +clusterrole.rbac.authorization.k8s.io/cluster-resource-reader created +clusterrolebinding.rbac.authorization.k8s.io/cluster-resource-reader created +``` + +Now, we are ready for backup. In the next section, we are going to schedule a backup for our cluster resources. + +### Backup + +To schedule a backup, we have to create a `BackupConfiguration` object. Then KubeStash will create a CronJob to periodically backup the database. + +#### Create BackupConfiguration + +Below is the YAML for `BackupConfiguration` object we care going to use to backup the YAMLs of the cluster resources, + +```yaml +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: kube-system-backup + namespace: demo +spec: + target: + apiGroup: "" + kind: Namespace + name: kube-system + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /kube-system-manifests + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: kubedump-addon + tasks: + - name: manifest-backup + jobTemplate: + spec: + serviceAccountName: cluster-resource-reader +``` + +Here, + +- `spec.target` specifies the backup target. `apiGroup`, `kind`, `namespace` and `name` refers to the `apiGroup`, `kind`, `namespace` and `name` of the targeted workload respectively. +- `spec.sessions[*].scheduler.schedule` specifies a [cron expression](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) indicates that `BackupSession` will be created at 5 minute interval. +- `spec.sessions[*].repositories[*].name` specifies the name of the `Repository` object that holds the backend information. +- `spec.sessions[*].repositories[*].backend` specifies the name of the backend that holds the `BackupStorage` information. +- `spec.sessions[*].repositories[*].directory` specifies the path of the `Repository` where the backed up data will be stored. +- `spec.sessions[*].repositories[*].encryptionSecret` specifies the encryption secret for `Restic Repository` which will be used to encrypting the backup data. +- `spec.sessions[*].addon.name` specifies the name of the `Addon` object that specifies addon configuration that will be used to perform backup of a stand-alone PVC. +- `spec.sessions[*].addon.tasks[*].name` specifies the name of the `Task` that holds the `Function` and their order of execution to perform backup of a stand-alone PVC. + +Let's create the `BackupConfiguration` object we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/kubedump/namespace/examples/backupconfiguration.yaml +backupconfiguration.core.kubestash.com/kube-system-backup created +``` + +#### Verify Backup Setup Successful + +If everything goes well, the phase of the `BackupConfiguration` should be `Ready`. The `Ready` phase indicates that the backup setup is successful. Let's verify the `Phase` of the BackupConfiguration, + +```bash +❯ kubectl get backupconfiguration -n demo +NAME PHASE PAUSED AGE +kube-system-backup Ready 79s + +``` + +**Verify Repository:** + +Verify that KubeStash has created `Repositories` that holds the `BackupStorage` information by the following command, + +```bash +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository Ready 28s +``` + +**Verify CronJob:** + +Verify that KubeStash has created a CronJob to trigger a periodic backup of the targeted PVC by the following command, + +```bash +$ kubectl get cronjob -n demo +NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE +trigger-kube-system-backup-frequent-backup */5 * * * * False 0 45s +``` + +**Wait for BackupSession:** + +Now, wait for the next backup schedule. You can watch for `BackupSession` crd using the following command, + +```bash +$ Every 1.0s: kubectl get backupsession -n demo -l=kubestash.com/invoker-name=kube-system-backup anisur: Fri Feb 23 18:34:53 2024 + +NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE +kube-system-backup-frequent-backup-1708691400 BackupConfiguration kube-system-backup Succeeded 4m53s + +``` + +**Verify Backup:** + +When backup session is created, kubestash operator creates `Snapshot` which represents the state of backup run for each `Repository` which are provided in `BackupConfiguration`. + +Run the following command to check `Snapshot` phase, + +```bash +$ kubectl get snapshots -n demo +NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE AGE +gcs-repository-kube-system-backup-frequent-backup-1708691400 gcs-repository frequent-backup 2024-02-23T12:30:00Z Delete Succeeded 5m9s +``` + +When backup session is completed, KubeStash will update the respective `Repository` to reflect the latest state of backed up data. + +Run the following command to check if a backup snapshot has been stored in the backend, + +```bash +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository true 1 2.262 KiB Ready 103s 8m +``` + +From the output above, we can see that `1` snapshot has been stored in the backend specified by Repository `gcs-repository`. + +Now, if we navigate to the GCS bucket, we will see the backed up data has been stored in `/manifest/namespace/kube-system` directory as specified by `.spec.backend.gcs.prefix` field of the `Repository` object. + +
+ Backup data in GCS Bucket +
Fig: Backup data in GCS Bucket
+
+ +> Note: KubeStash keeps all the backed-up data encrypted. So, data in the backend will not make any sense until they are decrypted. + +## Restore + +KubeStash does not provide any automatic mechanism to restore the cluster resources from the backed-up YAMLs. Your application might be managed by Helm or by an operator. In such cases, just applying the YAMLs is not enough to restore the application. Furthermore, there might be an order issue. Some resources must be applied before others. It is difficult to generalize and codify various application-specific logic. + +Therefore, it is the user's responsibility to download the backed-up YAMLs and take the necessary steps based on his application to restore it properly. + +### Download the YAMLs + +KubeStash provides a [kubectl plugin](/docs/guides/cli/cli/#download-snapshots) for making it easy to download a snapshot locally. + +Now, let's download the latest Snapshot from our backed-up data into the `$HOME/Downloads/kubestash/namespace/kube-system` folder of our local machine. + +```bash +❯ kubectl kubestash download --namespace=demo gcs-repository-kube-system-backup-frequent-backup-1708691400 --destination=$HOME/Downloads/kubestash/namespace/kube-system/ +``` + +Now, lets use [tree](https://linux.die.net/man/1/tree) command to inspect downloaded YAMLs files. + +```bash +❯ tree /home/anisur/Downloads/kubestash/namespace/kube-system +/home/anisur/Downloads/kubestash/namespace/kube-system +└── gcs-repository-kube-system-backup-frequent-backup-1708691400 + └── manifest + └── tmp + └── manifest + ├── ConfigMap + │   ├── coredns.yaml + │   ├── extension-apiserver-authentication.yaml + │   ├── kubeadm-config.yaml + ├── ControllerRevision + │   ├── kindnet-685dcc8c4c.yaml + │   └── kube-proxy-7ccf78d585.yaml + ├── DaemonSet + │   ├── kindnet.yaml + │   └── kube-proxy.yaml + ├── Deployment + │   └── coredns.yaml + ├── Endpoints + │   └── kube-dns.yaml + ├── EndpointSlice + │   └── kube-dns-nv9px.yaml + ├── Lease + │   ├── kube-controller-manager.yaml + │   └── kube-scheduler.yaml + ├── Pod + │   ├── coredns-565d847f94-78r86.yaml + │   ├── coredns-565d847f94-zdtcs.yaml + │   ├── etcd-kind-control-plane.yaml + │   ├── kindnet-qb7b8.yaml + ├── ReplicaSet + │   └── coredns-565d847f94.yaml + ├── Role + │   ├── extension-apiserver-authentication-reader.yaml + │   ├── kubeadm:kubelet-config.yaml + │   ├── kubeadm:nodes-kubeadm-config.yaml + │   ├── kube-proxy.yaml + ├── RoleBinding + │   ├── kubeadm:kubelet-config.yaml + │   ├── kubeadm:nodes-kubeadm-config.yaml + │   ├── kube-proxy.yaml + │   ├── system:controller:bootstrap-signer.yaml + │   ├── system:controller:cloud-provider.yaml + ├── Service + │   └── kube-dns.yaml + └── ServiceAccount + ├── attachdetach-controller.yaml + ├── bootstrap-signer.yaml + ├── certificate-controller.yaml + ├── cluster-resource-reader.yaml + ├── clusterrole-aggregation-controller.yaml + ├── coredns.yaml + +17 directories, 80 files +``` + +Here, the resources has been grouped under the respective Kind folder. + +Let's inspect the YAML of `coredns.yaml` file under ConfigMap folder, + +```yaml +❯ cat /home/anisur/Downloads/kubestash/namespace/kube-system/gcs-repository-kube-system-backup-frequent-backup-1708691400/manifest/tmp/manifest/ConfigMap/coredns.yaml +apiVersion: v1 +data: + Corefile: | + .:53 { + errors + health { + lameduck 5s + } + ready + kubernetes cluster.local in-addr.arpa ip6.arpa { + pods insecure + fallthrough in-addr.arpa ip6.arpa + ttl 30 + } + prometheus :9153 + forward . /etc/resolv.conf { + max_concurrent 1000 + } + cache 30 + loop + reload + loadbalance + } +kind: ConfigMap +metadata: + name: coredns + namespace: kube-system + +``` + +Now, you can use these YAML files to re-create your desired application. + +## Cleanup + +To cleanup the Kubernetes resources created by this tutorial, run: + +```bash +kubectl delete -n demo backupconfiguration kube-system-backup +kubectl delete -n kubestash serviceaccount cluster-resource-reader +kubectl delete clusterrole cluster-resource-reader +kubectl delete clusterrolebinding cluster-resource-reader +kubectl delete retentionPolicy -n demo demo-retention +kubectl delete -n demo backupstorage gcs-storage +kubectl delete secret -n demo encryption-secret +kubectl delete secret -n demo gcs-secret +``` diff --git a/docs/guides/kubedump/overview/images/kubedump-backup.svg b/docs/guides/kubedump/overview/images/kubedump-backup.svg new file mode 100644 index 0000000..dbaa26f --- /dev/null +++ b/docs/guides/kubedump/overview/images/kubedump-backup.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/guides/kubedump/overview/index.md b/docs/guides/kubedump/overview/index.md new file mode 100644 index 0000000..87a6c5b --- /dev/null +++ b/docs/guides/kubedump/overview/index.md @@ -0,0 +1,50 @@ +--- +title: KubeDump Backup Overview | KubeStash +description: How KubeDump Backup Works in KubeStash +menu: + docs_{{ .version }}: + identifier: kubestash-kubedump-overview + name: How does it work? + parent: kubestash-kubedump + weight: 10 +product_name: stash +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +# How KubeStash Backups Kubernetes Resources + +KubeStash `{{< param "info.version" >}}` supports taking backup of Kubernetes resource YAMLs. You can backup the YAML definition of the resources of entire cluster, a particular namespaces, or only an application etc. In this guide, we are going to show you how Kubernetes resource backup works in KubeStash. + +## How Backup Works + +The following diagram shows how KubeStash takes backup of the Kubernetes resources. Open the image in a new tab to see the enlarged version. + +
+  KubeDump Backup Overview +
Fig: KubeDump Backup Overview
+
+ +The backup process consists of the following steps: + +1. At first, a user creates a `Secret` with access credentials of the backend where the backed-up data will be stored. +2. Then, she creates a `BackupStorage` custom resource that specifies the backend information, along with the `Secret` containing the credentials needed to access the backend. +3. KubeStash operator watches for `BackupStorage` custom resources. When it finds a `BackupStorage` object, it initializes the `BackupStorage` by uploading the `metadata.yaml` file into the target storage. +4. Next, she creates a `BackupConfiguration` custom resource, targeting a kubernetes `resource`, `namespace` or `empty`. Target `empty` signifies targeting the entire cluster. The `BackupConfiguration` object specifies the `Repository` pointing to a `BackupStorage` that contains backend information, indicating where to upload backup data. It also defines the `Addon` information with a specified `Task` to be used for backing up the volume. +5. KubeStash operator watches for `BackupConfiguration` custom resources. +6. Once the KubeStash operator finds a `BackupConfiguration` object, it creates `Repository` with the information specified in the BackupConfiguration. +7. KubeStash operator watches for `Repository` custom resources. When it finds the `Repository` object, it Initializes `Repository` by uploading `repository.yaml` file into the `spec.sessions[*].repositories[*].directory` path specified in `BackupConfiguration`. +8. Then, it creates a `CronJob` with the schedule specified in `BackupConfiguration` to trigger backup periodically. +9. On the next scheduled slot, the `CronJob` triggers a backup by creating a `BackupSession` custom resource. +10. KubeStash operator watches for `BackupSession` custom resources. +11. When it finds a `BackupSession` object, it creates a `Snapshot` custom resource for each `Repository` specified in the `BackupConfiguration`. +12. Then it resolves the respective `Addon` and `Function` and prepares a backup `Job` definition. +13. Then, it creates the `Job` to backup the desired resource YAMLs. +14. Then, the Job dumps the Kubernetes Resource YAML, and store them temporarily in a directory.Then, it upload the content of the directory to the cloud backend. +15. After the backup process is completed, the backup `Job` updates the `status.components[*]` field of the `Snapshot` resources with each target volume information. It also updates the `status.phase` field of the `BackupSession` to reflect backup completion. + +## Next Steps + +- Backup the YAMLs of your entire Kubernetes cluster using KubeStash following the guide from [here](/docs/guides/kubedump/cluster/index.md). +- Backup the YAMLs of an entire Namespace using KubeStash following the guide from [here](/docs/guides/kubedump/namespace/index.md). +- Backup the YAMLs of a particular application using KubeStash following the guide from [here](/docs/guides/kubedump/application/index.md). diff --git a/docs/guides/platforms/minio/examples/backupconfiguration.yaml b/docs/guides/platforms/minio/examples/backupconfiguration.yaml index 1a075b8..0d031c1 100644 --- a/docs/guides/platforms/minio/examples/backupconfiguration.yaml +++ b/docs/guides/platforms/minio/examples/backupconfiguration.yaml @@ -1,23 +1,46 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: - name: deployment-backup + name: sample-backup-dep namespace: demo spec: - repository: - name: minio-repo - schedule: "*/5 * * * *" target: - ref: - apiVersion: apps/v1 - kind: Deployment - name: stash-demo - volumeMounts: - - name: source-data - mountPath: /source/data - paths: - - /source/data - retentionPolicy: - name: 'keep-last-5' - keepLast: 5 - prune: true + apiGroup: apps + kind: Deployment + name: kubestash-demo + namespace: demo + backends: + - name: minio-backend + storageRef: + name: minio-storage + namespace: demo + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: demo-session + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: minio-demo-repo + backend: minio-backend + directory: /dep + encryptionSecret: + name: encryption-secret + namespace: demo + addon: + name: workload-addon + tasks: + - name: logical-backup + targetVolumes: + volumeMounts: + - name: source-data + mountPath: /source/data + params: + paths: /source/data + exclude: /source/data/lost+found + retryConfig: + maxRetry: 2 + delay: 1m diff --git a/docs/guides/platforms/minio/examples/backupstorage.yaml b/docs/guides/platforms/minio/examples/backupstorage.yaml new file mode 100644 index 0000000..2184f70 --- /dev/null +++ b/docs/guides/platforms/minio/examples/backupstorage.yaml @@ -0,0 +1,19 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: minio-storage + namespace: demo +spec: + storage: + provider: s3 + s3: + secretName: minio-secret + endpoint: http://minio.storage.svc.cluster.local:443 + bucket: minio-bucket + region: us-east-1 + prefix: /demo + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/platforms/minio/examples/deployment.yaml b/docs/guides/platforms/minio/examples/deployment.yaml index b919066..282ea5e 100644 --- a/docs/guides/platforms/minio/examples/deployment.yaml +++ b/docs/guides/platforms/minio/examples/deployment.yaml @@ -2,31 +2,31 @@ apiVersion: apps/v1 kind: Deployment metadata: labels: - app: stash-demo - name: stash-demo + app: kubestash-demo + name: kubestash-demo namespace: demo spec: replicas: 3 selector: matchLabels: - app: stash-demo + app: kubestash-demo template: metadata: labels: - app: stash-demo + app: kubestash-demo name: busybox spec: containers: - - args: ["echo sample_data > /source/data/data.txt && sleep 3000"] - command: ["/bin/sh", "-c"] - image: busybox - imagePullPolicy: IfNotPresent - name: busybox - volumeMounts: - - mountPath: /source/data - name: source-data + - args: ["echo sample_data > /source/data/data.txt && sleep 3000"] + command: ["/bin/sh", "-c"] + image: busybox + imagePullPolicy: IfNotPresent + name: busybox + volumeMounts: + - mountPath: /source/data + name: source-data restartPolicy: Always volumes: - - name: source-data - persistentVolumeClaim: - claimName: source-pvc + - name: source-data + persistentVolumeClaim: + claimName: source-pvc \ No newline at end of file diff --git a/docs/guides/platforms/minio/examples/recovered_deployment.yaml b/docs/guides/platforms/minio/examples/recovered_deployment.yaml index ae12652..2a33cb7 100644 --- a/docs/guides/platforms/minio/examples/recovered_deployment.yaml +++ b/docs/guides/platforms/minio/examples/recovered_deployment.yaml @@ -5,7 +5,7 @@ metadata: namespace: demo spec: accessModes: - - ReadWriteOnce + - ReadWriteOnce storageClassName: standard resources: requests: @@ -15,32 +15,32 @@ apiVersion: apps/v1 kind: Deployment metadata: labels: - app: stash-recovered - name: stash-recovered + app: kubestash-recovered + name: kubestash-recovered namespace: demo spec: replicas: 3 selector: matchLabels: - app: stash-recovered + app: kubestash-recovered template: metadata: labels: - app: stash-recovered + app: kubestash-recovered name: busybox spec: containers: - - args: - - sleep - - "3600" - image: busybox - imagePullPolicy: IfNotPresent - name: busybox - volumeMounts: - - mountPath: /restore/data - name: restore-data + - args: + - sleep + - "3600" + image: busybox + imagePullPolicy: IfNotPresent + name: busybox + volumeMounts: + - mountPath: /source/data + name: restore-data restartPolicy: Always volumes: - - name: restore-data - persistentVolumeClaim: - claimName: restore-pvc + - name: restore-data + persistentVolumeClaim: + claimName: restore-pvc \ No newline at end of file diff --git a/docs/guides/platforms/minio/examples/repository.yaml b/docs/guides/platforms/minio/examples/repository.yaml deleted file mode 100644 index fa0d958..0000000 --- a/docs/guides/platforms/minio/examples/repository.yaml +++ /dev/null @@ -1,12 +0,0 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: minio-repo - namespace: demo -spec: - backend: - s3: - endpoint: 'https://minio-nodeport-svc.storage.svc' - bucket: minio-bucket - prefix: /source/data - storageSecretName: minio-secret diff --git a/docs/guides/platforms/minio/examples/restoresession.yaml b/docs/guides/platforms/minio/examples/restoresession.yaml index ac8613e..a8de3bd 100644 --- a/docs/guides/platforms/minio/examples/restoresession.yaml +++ b/docs/guides/platforms/minio/examples/restoresession.yaml @@ -1,19 +1,21 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: RestoreSession metadata: - name: deployment-restore + name: sample-restore namespace: demo spec: - repository: - name: minio-repo - target: # target indicates where the recovered data will be stored - ref: - apiVersion: apps/v1 - kind: Deployment - name: stash-recovered - volumeMounts: - - name: restore-data - mountPath: /source/data - rules: - - paths: - - /source/data/ + target: + apiGroup: apps + kind: Deployment + name: kubestash-recovered + namespace: demo + dataSource: + repository: minio-demo-repo + snapshot: latest + encryptionSecret: + name: encryption-secret + namespace: demo + addon: + name: workload-addon + tasks: + - name: logical-backup-restore \ No newline at end of file diff --git a/docs/guides/platforms/minio/examples/retentionpolicy.yaml b/docs/guides/platforms/minio/examples/retentionpolicy.yaml new file mode 100644 index 0000000..4591562 --- /dev/null +++ b/docs/guides/platforms/minio/examples/retentionpolicy.yaml @@ -0,0 +1,15 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: All \ No newline at end of file diff --git a/docs/guides/platforms/minio/images/minio.png b/docs/guides/platforms/minio/images/minio.png index 9a25079..ad9c395 100644 Binary files a/docs/guides/platforms/minio/images/minio.png and b/docs/guides/platforms/minio/images/minio.png differ diff --git a/docs/guides/platforms/minio/index.md b/docs/guides/platforms/minio/index.md index b7e0b36..e835ffc 100644 --- a/docs/guides/platforms/minio/index.md +++ b/docs/guides/platforms/minio/index.md @@ -1,6 +1,6 @@ --- -title: Minio | Stash -description: Using Stash with TLS secured Minio Server +title: Minio | KubeStash +description: Using KubeStash with TLS secured Minio Server menu: docs_{{ .version }}: identifier: platforms-minio @@ -12,21 +12,22 @@ menu_name: docs_{{ .version }} section_menu_id: guides --- -# Using Stash with TLS secured Minio Server +# Using KubeStash with TLS secured Minio Server -Minio is an open-source object storage server compatible with [AWS S3](https://aws.amazon.com/s3/) cloud storage service. This guide will show you how to use Stash to backup and restore volumes of a Kubernetes workload in [Minio](https://min.io/) backend. Here, we are going to backup a volume of a Deployment into a Minio bucket. Then, we are going to show how to restore this backed up data into a volume of another Deployment. +Minio is an open-source object storage server compatible with [AWS S3](https://aws.amazon.com/s3/) cloud storage service. This guide will show you how to use KubeStash to backup and restore volumes of a Kubernetes workload in [Minio](https://min.io/) backend. Here, we are going to backup a volume of a Deployment into a Minio bucket. Then, we are going to show how to restore this backed up data into a volume of another Deployment. ## Before You Begin - At first, you need to have a Kubernetes cluster. If you don't already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). -- Install `Stash` in your cluster following the steps [here](/docs/setup/README.md). +- Install `KubeStash` in your cluster following the steps [here](/docs/setup/README.md). -- You should be familiar with the following `Stash` concepts: +- You should be familiar with the following `KubeStash` concepts: - [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md) - [BackupSession](/docs/concepts/crds/backupsession/index.md) - [RestoreSession](/docs/concepts/crds/restoresession/index.md) - - [Repository](/docs/concepts/crds/repository/index.md) + - [BackupStorage](/docs/concepts/crds/backupstorage/index.md) + - You will need a TLS secured Minio server to store backed up data. If you already do not have a Minio server running, deploy one following the tutorial from [here](https://github.com/appscode/third-party-tools/blob/master/storage/minio/README.md). For this tutorial, we have deployed Minio server in `storage` namespace and it is accessible through `minio.storage.svc` dns. To keep everything isolated, we are going to use a separate namespace called `demo` throughout this tutorial. @@ -38,7 +39,7 @@ namespace/demo created **Choosing StorageClass:** -Stash works with any `StorageClass`. Check available `StorageClass` in your cluster using the following command: +KubeStash works with any `StorageClass`. Check available `StorageClass` in your cluster using the following command: ```bash $ kubectl get storageclass -n demo @@ -50,9 +51,9 @@ Here, we have `standard` StorageClass in our cluster. > **Note:** YAML files used in this tutorial are stored in [docs/guides/platforms/minio/examples](/docs/guides/platforms/minio/examples) directory of [kubestash/docs](https://github.com/kubestash/docs) repository. -## Backup the Volume of a Deployment +## Backup the Volumes of a Deployment -Here, we are going to deploy a Deployment with a PVC. This Deployment will automatically generate some sample data into the PVC. Then, we are going to backup this sample data using Stash. +Here, we are going to deploy a Deployment with a PVC. This Deployment will automatically generate some sample data into the PVC. Then, we are going to backup this sample data using KubeStash. ### Prepare Workload @@ -95,57 +96,57 @@ apiVersion: apps/v1 kind: Deployment metadata: labels: - app: stash-demo - name: stash-demo + app: kubestash-demo + name: kubestash-demo namespace: demo spec: replicas: 3 selector: matchLabels: - app: stash-demo + app: kubestash-demo template: metadata: labels: - app: stash-demo + app: kubestash-demo name: busybox spec: containers: - - args: ["echo sample_data > /source/data/data.txt && sleep 3000"] - command: ["/bin/sh", "-c"] - image: busybox - imagePullPolicy: IfNotPresent - name: busybox - volumeMounts: - - mountPath: /source/data - name: source-data + - args: ["echo sample_data > /source/data/data.txt && sleep 3000"] + command: ["/bin/sh", "-c"] + image: busybox + imagePullPolicy: IfNotPresent + name: busybox + volumeMounts: + - mountPath: /source/data + name: source-data restartPolicy: Always volumes: - - name: source-data - persistentVolumeClaim: - claimName: source-pvc + - name: source-data + persistentVolumeClaim: + claimName: source-pvc ``` Let's create the Deployment we have shown above. ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/platforms/minio/examples/deployment.yaml -deployment.apps/stash-demo created +deployment.apps/kubestash-demo created ``` Now, wait for the pods of the Deployment to go into the `Running` state. ```bash $ kubectl get pod -n demo -NAME READY STATUS RESTARTS AGE -stash-demo-69f9ffbbf7-6wwtr 1/1 Running 0 60s -stash-demo-69f9ffbbf7-88kgj 1/1 Running 0 60s -stash-demo-69f9ffbbf7-q8qld 1/1 Running 0 60s +NAME READY STATUS RESTARTS AGE +kubestash-demo-69f9ffbbf7-6wwtr 1/1 Running 0 60s +kubestash-demo-69f9ffbbf7-88kgj 1/1 Running 0 60s +kubestash-demo-69f9ffbbf7-q8qld 1/1 Running 0 60s ``` To verify that the sample data has been created in `/source/data` directory, use the following command: ```bash -$ kubectl exec -n demo stash-demo-69f9ffbbf7-6wwtr -- cat /source/data/data.txt +$ kubectl exec -n demo kubestash-demo-69f9ffbbf7-6wwtr -- cat /source/data/data.txt sample_data ``` @@ -158,12 +159,10 @@ We are going to store our backed up data into an [Minio Bucket](https://min.io/) Let's create a secret called `minio-secret` with access credentials to our desired [Minio bucket](https://min.io/), ```bash -$ echo -n 'changeit' > RESTIC_PASSWORD $ echo -n '' > AWS_ACCESS_KEY_ID $ echo -n '' > AWS_SECRET_ACCESS_KEY $ cat ./directory/of/root/certificate/ca.crt > CA_CERT_DATA $ kubectl create secret generic -n demo minio-secret \ - --from-file=./RESTIC_PASSWORD \ --from-file=./AWS_ACCESS_KEY_ID \ --from-file=./AWS_SECRET_ACCESS_KEY \ --from-file=./CA_CERT_DATA @@ -182,10 +181,9 @@ data: AWS_ACCESS_KEY_ID: YWRtaW4= AWS_SECRET_ACCESS_KEY: Y2hhbmdlaXQ= CA_CERT_DATA: dXNlIHlvdXIgb3duIGNyZWRlbnRpYWxz # - RESTIC_PASSWORD: Y2hhbmdlaXQ= kind: Secret metadata: - creationTimestamp: "2019-07-23T05:32:40Z" + creationTimestamp: "2024-02-23T05:32:40Z" name: minio-secret namespace: demo resourceVersion: "7824" @@ -194,284 +192,262 @@ metadata: type: Opaque ``` -**Create Repository:** +**Create BackupStorage:** -Now, let's create a `Repository` with the information of our desired Minio bucket. Below is the YAML of `Repository` crd we are going to create, +Now, create a `BackupStorage` using this secret. Below is the YAML of `BackupStorage` crd we are going to create, ```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: minio-repo + name: minio-storage namespace: demo spec: - backend: + storage: + provider: s3 s3: - endpoint: 'https://minio-nodeport-svc.storage.svc' + secretName: minio-secret + endpoint: http://minio.storage.svc.cluster.local:443 bucket: minio-bucket - prefix: /source/data - storageSecretName: minio-secret + region: us-east-1 + prefix: /demo + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut ``` +Here, +- If you didn't set `CA_CERT_DATA` into `minio-secret` secret file then you might want to skip TLS verification. In that case, within the `BackupStorage` configuration under the `S3` section, ensure the `insecureTLS` field is set to `true`. -Let's create the `Repository` we have shown above, + +Let's create the BackupStorage we have shown above, ```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/platforms/minio/examples/repository.yaml -repository.stash.appscode.com/minio-repo created +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/platforms/minio/examples/backupstorage.yaml +backupstorage.storage.kubestash.com/minio-storage created ``` Now, we are ready to backup our sample data into this backend. +**Create RetentionPolicy:** + +Now, let's create a `RetentionPolicy` to specify how the old Snapshots should be cleaned up. + +Below is the YAML of the `RetentionPolicy` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: All +``` +Notice the `spec.usagePolicy` that allows referencing the `RetentionPolicy` from all namespaces. To allow specific namespaces, we can configure it accordingly by following [RetentionPolicy usage policy](/docs/concepts/crds/retentionpolicy/index.md#retentionpolicy-spec). + +Let’s create the above `RetentionPolicy`, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/platforms/minio/examples/retentionpolicy.yaml +retentionpolicy.storage.kubestash.com/demo-retention created +``` + ### Backup -We have to create a `BackupConfiguration` crd targeting the `stash-demo` Deployment that we have deployed earlier. Stash will inject a sidecar container into the target. It will also create a `CronJob` to take a periodic backup of `/source/data` directory of the target. +We have to create a `BackupConfiguration` crd targeting the `kubestash-demo` Deployment that we have deployed earlier. Then, KubeStash will create a `CronJob` for each session to take periodic backup of `/source/data` directory of the target. + +At first, we need to create a secret with a Restic password for backup data encryption. + +**Create Secret:** + +Let's create a secret called `encry-secret` with the Restic password, + +```bash +$ echo -n 'changeit' > RESTIC_PASSWORD +$ kubectl create secret generic -n demo encryption-secret \ + --from-file=./RESTIC_PASSWORD \ +secret "encryption-secret" created +``` **Create BackupConfiguration:** Below is the YAML of the `BackupConfiguration` crd that we are going to create, ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: - name: deployment-backup + name: sample-backup-dep namespace: demo spec: - repository: - name: minio-repo - schedule: "*/5 * * * *" target: - ref: - apiVersion: apps/v1 - kind: Deployment - name: stash-demo - volumeMounts: - - name: source-data - mountPath: /source/data - paths: - - /source/data - retentionPolicy: - name: 'keep-last-5' - keepLast: 5 - prune: true + apiGroup: apps + kind: Deployment + name: kubestash-demo + namespace: demo + backends: + - name: minio-backend + storageRef: + name: minio-storage + namespace: demo + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: demo-session + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: minio-demo-repo + backend: minio-backend + directory: /dep + encryptionSecret: + name: encryption-secret + namespace: demo + addon: + name: workload-addon + tasks: + - name: logical-backup + targetVolumes: + volumeMounts: + - name: source-data + mountPath: /source/data + params: + paths: /source/data + exclude: /source/data/lost+found + retryConfig: + maxRetry: 2 + delay: 1m ``` -Here, - -- `spec.repository` refers to the `Repository` object `minio-repo` that holds backend [Minio bucket](https://min.io/) information. -- `spec.target.ref` refers to the `stash-demo` Deployment for backup target. -- `spec.target.volumeMounts` specifies a list of volumes and their mountPath that contain the target paths. -- `spec.target.paths` specifies list of file paths to backup. - Let's create the `BackupConfiguration` crd we have shown above, ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/platforms/minio/examples/backupconfiguration.yaml -backupconfiguration.stash.appscode.com/deployment-backup created +backupconfiguration.core.kubestash.com/sample-backup-dep created ``` -**Verify Sidecar:** +**Verify Backup Setup Successful** -If everything goes well, Stash will inject a sidecar container into the `stash-demo` Deployment to take backup of `/source/data` directory. Let’s check that the sidecar has been injected successfully, +If everything goes well, the phase of the `BackupConfiguration` should be `Ready`. The `Ready` phase indicates that the backup setup is successful. Let's verify the `Phase` of the BackupConfiguration, ```bash -$ kubectl get pod -n demo -NAME READY STATUS RESTARTS AGE -stash-demo-6548cf5cc-7qx9d 2/2 Running 0 42s -stash-demo-6548cf5cc-d26sx 2/2 Running 0 37s -stash-demo-6548cf5cc-f2xbl 2/2 Running 0 40s -``` - -Look at the pod. It now has 2 containers. If you view the resource definition of this pod, you will see that there is a container named `stash` which is running `run-backup` command. - -```yaml -$ kubectl get pod -n demo stash-demo-6548cf5cc-7qx9d -o yaml -apiVersion: v1 -kind: Pod -metadata: - annotations: - stash.appscode.com/last-applied-backupconfiguration-hash: "17185792379662025862" - creationTimestamp: "2019-07-23T08:52:42Z" - generateName: stash-demo-6548cf5cc- - labels: - app: stash-demo - pod-template-hash: 6548cf5cc - name: stash-demo-6548cf5cc-7qx9d - namespace: demo - ... -spec: - containers: - - args: - - echo sample_data > /source/data/data.txt && sleep 3000 - command: - - /bin/sh - - -c - image: busybox - imagePullPolicy: IfNotPresent - name: busybox - resources: {} - terminationMessagePath: /dev/termination-log - terminationMessagePolicy: File - volumeMounts: - - mountPath: /source/data - name: source-data - - mountPath: /var/run/secrets/kubernetes.io/serviceaccount - name: default-token-95xvp - readOnly: true - - args: - - run-backup - - --backup-configuration=deployment-backup - - --secret-dir=/etc/stash/repository/secret - - --enable-cache=true - - --max-connections=0 - - --metrics-enabled=true - - --pushgateway-url=http://stash-operator.kube-system.svc:56789 - - --enable-status-subresource=true - - --use-kubeapiserver-fqdn-for-aks=true - - --logtostderr=true - - --alsologtostderr=false - - --v=3 - - --stderrthreshold=0 - env: - - name: NODE_NAME - valueFrom: - fieldRef: - apiVersion: v1 - fieldPath: spec.nodeName - - name: POD_NAME - valueFrom: - fieldRef: - apiVersion: v1 - fieldPath: metadata.name - image: suaas21/stash:volumeTemp_linux_amd64 - imagePullPolicy: IfNotPresent - name: stash - resources: {} - terminationMessagePath: /dev/termination-log - terminationMessagePolicy: File - volumeMounts: - - mountPath: /etc/stash - name: stash-podinfo - - mountPath: /etc/stash/repository/secret - name: stash-secret-volume - - mountPath: /tmp - name: tmp-dir - - mountPath: /source/data - name: source-data - - mountPath: /var/run/secrets/kubernetes.io/serviceaccount - name: default-token-95xvp - readOnly: true - dnsPolicy: ClusterFirst - enableServiceLinks: true - nodeName: minikube - priority: 0 - restartPolicy: Always - schedulerName: default-scheduler - securityContext: {} - serviceAccount: default - serviceAccountName: default - terminationGracePeriodSeconds: 30 - volumes: - - name: source-data - persistentVolumeClaim: - claimName: source-pvc - - emptyDir: {} - name: tmp-dir - - downwardAPI: - defaultMode: 420 - items: - - fieldRef: - apiVersion: v1 - fieldPath: metadata.labels - path: labels - name: stash-podinfo - - name: stash-secret-volume - secret: - defaultMode: 420 - secretName: minio-secret - - name: default-token-95xvp - secret: - defaultMode: 420 - secretName: default-token-95xvp - ... -... +$ kubectl get backupconfiguration -n demo +NAME PHASE PAUSED AGE +sample-backup-dep Ready 2m50s ``` **Verify CronJob:** -It will also create a `CronJob` with the schedule specified in `spec.schedule` field of `BackupConfiguration` crd. +It will also create a `CronJob` with the schedule specified in `spec.sessions[*].scheduler.schedule` field of `BackupConfiguration` crd. Verify that the `CronJob` has been created using the following command, ```bash $ kubectl get cronjob -n demo -NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE -deployment-backup */1 * * * * False 0 13s 1m50s +NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE +trigger-sample-backup-dep-demo-session */5 * * * * 0 2m45s 3m25s ``` **Wait for BackupSession:** -The `deployment-backup` CronJob will trigger a backup on each schedule by creating a `BackupSession` crd. The sidecar container will watch for the `BackupSession` crd. When it finds one, it will take backup immediately. - Wait for the next schedule for backup. Run the following command to watch `BackupSession` crd, ```bash -$ watch -n 3 kubectl get backupsession -n demo -Every 3.0s: kubectl get backupsession -n demo suaas-appscode: Mon Jul 22 15:01:21 2019 +$ kubectl get backupsession -n demo -w -NAME INVOKER-TYPE INVOKER-NAME PHASE AGE -deployment-backup-1563786061 BackupConfiguration deployment-backup Succeeded 18s +NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE +sample-backup-dep-demo-session-1706015400 BackupConfiguration sample-backup-dep Succeeded 7m22s ``` We can see from the above output that the backup session has succeeded. Now, we are going to verify whether the backed up data has been stored in the backend. **Verify Backup:** -Once a backup is complete, Stash will update the respective `Repository` crd to reflect the backup. Check that the repository `minio-repo` has been updated by the following command, +Once a backup is complete, KubeStash will update the respective `Repository` crd to reflect the backup. Check that the repository `mino-demo-repo` has been updated by the following command, ```bash -$ kubectl get repository -n demo -NAME INTEGRITY SIZE SNAPSHOT-COUNT LAST-SUCCESSFUL-BACKUP AGE -minio-repo true 32 B 2 3m40s 5m18s +$ kubectl get repository -n demo minio-demo-repo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +minio-demo-repo true 1 806 B Ready 8m27s 9m18s ``` -Now, if we navigate to the Minio Bucket, we are going to see backed up data has been stored in `/source/data` directory as specified by `spec.backend.s3.prefix` field of `Repository` crd. +At this moment we have one `Snapshot`. Run the following command to check the respective `Snapshot` which represents the state of a backup run to a particular `Repository`. -
-  Backup data in Minio Bucket -
Fig: Backup data in Minio Bucket
-
+```bash +$ kubectl get snapshots -n demo -l=kubestash.com/repo-name=minio-demo-repo +NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE AGE +minio-demo-repo-sample-backup-dep-demo-session-1706015400 mino-demo-repo demo-session 2024-01-23T13:10:54Z Delete Succeeded 16h +``` -> **Note:** Stash keeps all the backed up data encrypted. So, data in the backend will not make any sense until they are decrypted. +> When a backup is triggered according to schedule, KubeStash will create a `Snapshot` with the following labels `kubestash.com/app-ref-kind: `, `kubestash.com/app-ref-name: `, `kubestash.com/app-ref-namespace: ` and `kubestash.com/repo-name: `. We can use these labels to watch only the `Snapshot` of our desired Workload or `Repository`. -## Restore the Backed up Data +If we check the YAML of the `Snapshot`, we can find the information about the backed up components of the Deployment. -This section will show you how to restore the backed up data from [Minio bucket](https://min.io/) we have taken in earlier section. +```bash +$ kubectl get snapshots -n demo minio-demo-repo-sample-backup-dep-demo-session-1706015400 -oyaml +``` -**Stop Taking Backup of the Old Deployment:** +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: Snapshot +metadata: + labels: + kubestash.com/app-ref-kind: Deployment + kubestash.com/app-ref-name: kubestash-demo + kubestash.com/app-ref-namespace: demo + kubestash.com/repo-name: minio-demo-repo + name: minio-demo-repo-sample-backup-dep-demo-session-1706015400 + namespace: demo +spec: + ... +status: + components: + dump: + driver: Restic + duration: 7.534461497s + integrity: true + path: repository/v1/demo-session/dump + phase: Succeeded + resticStats: + - hostPath: /source/data + id: f28441a36b2167d64597d66d1046573181cad81aa8ff5b0998b64b31ce16f077 + size: 11 B + uploaded: 1.049 KiB + size: 806 B + ... +``` -At first, let's stop taking any further backup of the old Deployment so that no backup is taken during the restore process. We are going to pause the `BackupConfiguration` that we created to backup the `stash-demo` Deployment. Then, Stash will stop taking any further backup for this Deployment. You can learn more how to pause a scheduled backup [here](/docs/guides/use-cases/pause-backup/index.md) +> For Deployment, KubeStash takes backup from only one pod of the Deployment. So, only one component has been taken backup. -Let's pause the `deployment-backup` BackupConfiguration, +Now, if we navigate to the Minio Bucket, we are going to see backed up data has been stored in `/demo/dep/repository/v1/demo-session/dump` directory as specified in the `repository` section within the `backupConfiguration` crd. -```bash -$ kubectl patch backupconfiguration -n demo deployment-backup --type="merge" --patch='{"spec": {"paused": true}}' -backupconfiguration.stash.appscode.com/deployment-backup patched -``` +
+  Backup data in Minio Bucket +
Fig: Backup data in Minio Bucket
+
-Now, wait for a moment. Stash will pause the BackupConfiguration. Verify that the BackupConfiguration has been paused, +> KubeStash keeps all backup data encrypted. So, the files in the bucket will not contain any meaningful data until they are decrypted. -```bash -$ kubectl get backupconfiguration -n demo -NAME TASK SCHEDULE PAUSED AGE -deployment-backup */1 * * * * true 26m -``` +## Restore the Backed up Data -Notice the `PAUSED` column. Value `true` for this field means that the BackupConfiguration has been paused. +This section will show you how to restore the backed up data from [Minio bucket](https://min.io/) we have taken in earlier section. **Deploy Deployment:** -We are going to create a new Deployment named `stash-recovered` with a new PVC and restore the backed up data inside it. +We are going to create a new Deployment named `kubestash-recovered` with a new PVC and restore the backed up data inside it. Below are the YAMLs of the Deployment and PVC that we are going to create, @@ -483,7 +459,7 @@ metadata: namespace: demo spec: accessModes: - - ReadWriteOnce + - ReadWriteOnce storageClassName: standard resources: requests: @@ -493,35 +469,35 @@ apiVersion: apps/v1 kind: Deployment metadata: labels: - app: stash-recovered - name: stash-recovered + app: kubestash-recovered + name: kubestash-recovered namespace: demo spec: replicas: 3 selector: matchLabels: - app: stash-recovered + app: kubestash-recovered template: metadata: labels: - app: stash-recovered + app: kubestash-recovered name: busybox spec: containers: - - args: - - sleep - - "3600" - image: busybox - imagePullPolicy: IfNotPresent - name: busybox - volumeMounts: - - mountPath: /restore/data - name: restore-data + - args: + - sleep + - "3600" + image: busybox + imagePullPolicy: IfNotPresent + name: busybox + volumeMounts: + - mountPath: /source/data + name: restore-data restartPolicy: Always volumes: - - name: restore-data - persistentVolumeClaim: - claimName: restore-pvc + - name: restore-data + persistentVolumeClaim: + claimName: restore-pvc ``` Let's create the Deployment and PVC we have shown above. @@ -529,161 +505,81 @@ Let's create the Deployment and PVC we have shown above. ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/platforms/minio/examples/recovered_deployment.yaml persistentvolumeclaim/restore-pvc created -deployment.apps/stash-recovered created +deployment.apps/kubestash-recovered created ``` **Create RestoreSession:** -Now, we need to create a `RestoreSession` crd targeting the `stash-recovered` Deployment. +Now, we need to create a `RestoreSession` crd targeting the `kubestash-recovered` Deployment. Below is the YAML of the `RestoreSesion` crd that we are going to create, ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: RestoreSession metadata: - name: deployment-restore + name: sample-restore namespace: demo spec: - repository: - name: minio-repo - target: # target indicates where the recovered data will be stored - ref: - apiVersion: apps/v1 - kind: Deployment - name: stash-recovered - volumeMounts: - - name: restore-data - mountPath: /source/data - rules: - - paths: - - /source/data/ + target: + apiGroup: apps + kind: Deployment + name: kubestash-recovered + namespace: demo + dataSource: + repository: minio-demo-repo + snapshot: latest + encryptionSecret: + name: encryption-secret + namespace: demo + addon: + name: workload-addon + tasks: + - name: logical-backup-restore ``` Here, -- `spec.repository.name` specifies the `Repository` crd that holds the backend information where our backed up data has been stored. -- `spec.target.ref` refers to the target workload where the recovered data will be stored. -- `spec.target.volumeMounts` specifies a list of volumes and their mountPath where the data will be restored. - - `mountPath` must be same `mountPath` as the original volume because Stash stores absolute path of the backed up files. If you use different `mountPath` for the restored volume the backed up files will not be restored into your desired volume. +- `spec.dataSource.snapshot` specifies to restore from latest `Snapshot`. Let's create the `RestoreSession` crd we have shown above, ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/platforms/minio/examples/restoresession.yaml -restoresession.stash.appscode.com/deployment-restore created +restoresession.core.kubestash.com/deployment-restore created ``` -Once, you have created the `RestoreSession` crd, Stash will inject `init-container` into `stash-recovered` Deployment. The Deployment will restart and the `init-container` will restore the desired data on start-up. - -**Verify Init-Container:** +Once, you have created the `RestoreSession` object, KubeStash will create restore Job(s). Run the following command to watch the phase of the `RestoreSession` object, -Wait until the `init-container` has been injected into the `stash-recovered` Deployment. Let’s describe the Deployment to verify that `init-container` has been injected successfully. - -```yaml -$ kubectl describe deployment -n demo stash-recovered -Name: stash-recovered -Namespace: demo -CreationTimestamp: Tue, 23 Jul 2019 15:05:11 +0600 -Labels: app=stash-recovered -Selector: app=stash-recovered -Replicas: 3 desired | 2 updated | 4 total | 3 available | 1 unavailable -StrategyType: RollingUpdate -... -Pod Template: - Labels: app=stash-recovered - Annotations: stash.appscode.com/last-applied-restoresession-hash: 9375001205249667775 - Init Containers: - stash-init: - Image: suaas21/stash:volumeTemp_linux_amd64 - Port: - Host Port: - Args: - restore - --restore-session=deployment-restore - --secret-dir=/etc/stash/repository/secret - --enable-cache=true - --max-connections=0 - --metrics-enabled=true - --pushgateway-url=http://stash-operator.kube-system.svc:56789 - --enable-status-subresource=true - --use-kubeapiserver-fqdn-for-aks=true - --logtostderr=true - --alsologtostderr=false - --v=3 - --stderrthreshold=0 - Environment: - NODE_NAME: (v1:spec.nodeName) - POD_NAME: (v1:metadata.name) - Mounts: - /etc/stash/repository/secret from stash-secret-volume (rw) - /source/data from restore-data (rw) - /tmp from tmp-dir (rw) - Containers: - busybox: - Image: busybox - Port: - Host Port: - Args: - sleep - 3600 - Environment: - Mounts: - /restore/data from restore-data (rw) - Volumes: - restore-data: - Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace) - ClaimName: restore-pvc - ReadOnly: false - tmp-dir: - Type: EmptyDir (a temporary directory that shares a pod's lifetime) - Medium: - SizeLimit: - stash-podinfo: - Type: DownwardAPI (a volume populated by information about the pod) - Items: - metadata.labels -> labels - stash-secret-volume: - Type: Secret (a volume populated by a Secret) - SecretName: minio-secret - Optional: false -... -``` - -Notice the `Init-Containers` section. We can see that the init-container `stash-init` has been injected which is running `restore` command. - -**Wait for RestoreSession to Succeeded:** - -Now, wait for the restore process to complete. You can watch the `RestoreSession` phase using the following command, ```bash -$ watch -n 2 kubectl get restoresession -n demo -Every 3.0s: kubectl get restoresession --all-namespaces suaas-appscode: Mon Jul 22 18:17:26 2019 +$ watch kubectl get restoresession -n demo +Every 2.0s: kubectl get restores... AppsCode-PC-03: Wed Jan 10 17:13:18 2024 -NAMESPACE NAME REPOSITORY-NAME PHASE AGE -demo deployment-restore minio-repo Succeeded 110s +NAME REPOSITORY FAILURE-POLICY PHASE DURATION AGE +deployment-restore minio-demo-repo Succeeded 3s 53s ``` -So, we can see from the output of the above command that the restore process has succeeded. +The `Succeeded` phase means that the restore process has been completed successfully. > **Note:** If you want to restore the backed up data inside the same Deployment whose volumes were backed up, you have to remove the corrupted data from the Deployment. Then, you have to create a RestoreSession targeting the Deployment. **Verify Restored Data:** -In this section, we are going to verify that the desired data has been restored successfully. At first, check if the `stash-recovered` pod of the Deployment has gone into `Running` state by the following command, +In this section, we are going to verify that the desired data has been restored successfully. At first, check if the `kubestash-recovered` pod of the Deployment has gone into `Running` state by the following command, ```bash $ kubectl get pod -n demo -NAME READY STATUS RESTARTS AGE -stash-recovered-6f5c46fdbf-lfthv 1/1 Running 0 2m39s -stash-recovered-6f5c46fdbf-s7rrq 1/1 Running 0 2m15s -stash-recovered-6f5c46fdbf-vn2z8 1/1 Running 0 2m35s +NAME READY STATUS RESTARTS AGE +kubestash-recovered-6f5c46fdbf-lfthv 1/1 Running 0 2m39s +kubestash-recovered-6f5c46fdbf-s7rrq 1/1 Running 0 2m15s +kubestash-recovered-6f5c46fdbf-vn2z8 1/1 Running 0 2m35s ``` -Verify that the sample data has been restored in `/restore/data` directory of the `stash-recovered` pod of the Deployment using the following command, +Verify that the sample data has been restored in `/source/data` directory of the `kubestash-recovered` pod of the Deployment using the following command, ```bash -$ kubectl exec -n demo stash-recovered-6f5c46fdbf-s7rrq -- cat /restore/data/data.txt +$ kubectl exec -n demo kubestash-recovered-6f5c46fdbf-s7rrq -- cat /source/data/data.txt sample_data ``` @@ -692,11 +588,13 @@ sample_data To clean up the Kubernetes resources created by this tutorial, run: ```bash -kubectl delete -n demo deployment stash-demo -kubectl delete -n demo deployment stash-recovered +kubectl delete -n demo deployment kubestash-demo +kubectl delete -n demo deployment kubestash-recovered kubectl delete -n demo backupconfiguration deployment-backup kubectl delete -n demo restoresession deployment-restore -kubectl delete -n demo repository minio-repo +kubectl delete -n demo backupstorage minio-storage +kubectl delete -n demo retentionpolicy demo-retention +kubectl delete -n demo secret encryption-secret kubectl delete -n demo secret minio-secret kubectl delete -n demo pvc --all ``` diff --git a/docs/guides/platforms/rook/examples/backupconfiguration.yaml b/docs/guides/platforms/rook/examples/backupconfiguration.yaml index d229311..3579629 100644 --- a/docs/guides/platforms/rook/examples/backupconfiguration.yaml +++ b/docs/guides/platforms/rook/examples/backupconfiguration.yaml @@ -1,23 +1,46 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: - name: deployment-backup + name: sample-backup-dep namespace: demo spec: - repository: - name: rook-repo - schedule: "*/5 * * * *" target: - ref: - apiVersion: apps/v1 - kind: Deployment - name: stash-demo - volumeMounts: - - name: source-data - mountPath: /source/data - paths: - - /source/data - retentionPolicy: - name: 'keep-last-5' - keepLast: 5 - prune: true + apiGroup: apps + kind: Deployment + name: kubestash-demo + namespace: demo + backends: + - name: rook-backend + storageRef: + name: rook-storage + namespace: demo + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: demo-session + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: rook-demo-repo + backend: rook-backend + directory: /dep + encryptionSecret: + name: encryption-secret + namespace: demo + addon: + name: workload-addon + tasks: + - name: logical-backup + targetVolumes: + volumeMounts: + - name: source-data + mountPath: /source/data + params: + paths: /source/data + exclude: /source/data/lost+found + retryConfig: + maxRetry: 2 + delay: 1m \ No newline at end of file diff --git a/docs/guides/platforms/rook/examples/backupstorage.yaml b/docs/guides/platforms/rook/examples/backupstorage.yaml new file mode 100644 index 0000000..76c0124 --- /dev/null +++ b/docs/guides/platforms/rook/examples/backupstorage.yaml @@ -0,0 +1,19 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: rook-storage + namespace: demo +spec: + storage: + provider: s3 + s3: + secretName: rook-secret + endpoint: 'http://rook-ceph-rgw-my-store-external.rook-ceph.svc' + bucket: rook-bucket + region: us-east-1 + prefix: /demo + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/platforms/rook/examples/deployment.yaml b/docs/guides/platforms/rook/examples/deployment.yaml index 93f0c38..282ea5e 100644 --- a/docs/guides/platforms/rook/examples/deployment.yaml +++ b/docs/guides/platforms/rook/examples/deployment.yaml @@ -2,35 +2,31 @@ apiVersion: apps/v1 kind: Deployment metadata: labels: - app: stash-demo - name: stash-demo + app: kubestash-demo + name: kubestash-demo namespace: demo spec: - replicas: 1 + replicas: 3 selector: matchLabels: - app: stash-demo + app: kubestash-demo template: metadata: labels: - app: stash-demo + app: kubestash-demo name: busybox spec: containers: - - args: ["echo sample_data > /source/data/data.txt && sleep 3000"] - command: ["/bin/sh", "-c"] - image: busybox - imagePullPolicy: IfNotPresent - name: busybox - volumeMounts: - - mountPath: /source/data - name: source-data + - args: ["echo sample_data > /source/data/data.txt && sleep 3000"] + command: ["/bin/sh", "-c"] + image: busybox + imagePullPolicy: IfNotPresent + name: busybox + volumeMounts: + - mountPath: /source/data + name: source-data restartPolicy: Always volumes: - - name: source-data - persistentVolumeClaim: - claimName: source-pvc - strategy: - rollingUpdate: - maxSurge: "0%" - maxUnavailable: "100%" + - name: source-data + persistentVolumeClaim: + claimName: source-pvc \ No newline at end of file diff --git a/docs/guides/platforms/rook/examples/recovered_deployment.yaml b/docs/guides/platforms/rook/examples/recovered_deployment.yaml index b9ff8b0..a90bdc1 100644 --- a/docs/guides/platforms/rook/examples/recovered_deployment.yaml +++ b/docs/guides/platforms/rook/examples/recovered_deployment.yaml @@ -5,7 +5,7 @@ metadata: namespace: demo spec: accessModes: - - ReadWriteOnce + - ReadWriteOnce storageClassName: "rook-ceph-block" resources: requests: @@ -15,36 +15,32 @@ apiVersion: apps/v1 kind: Deployment metadata: labels: - app: stash-recovered - name: stash-recovered + app: kubestash-recovered + name: kubestash-recovered namespace: demo spec: - replicas: 1 + replicas: 3 selector: matchLabels: - app: stash-recovered + app: kubestash-recovered template: metadata: labels: - app: stash-recovered + app: kubestash-recovered name: busybox spec: containers: - - args: - - sleep - - "3600" - image: busybox - imagePullPolicy: IfNotPresent - name: busybox - volumeMounts: - - mountPath: /restore/data - name: restore-data + - args: + - sleep + - "3600" + image: busybox + imagePullPolicy: IfNotPresent + name: busybox + volumeMounts: + - mountPath: /source/data + name: restore-data restartPolicy: Always volumes: - - name: restore-data - persistentVolumeClaim: - claimName: restore-pvc - strategy: - rollingUpdate: - maxSurge: "0%" - maxUnavailable: "100%" + - name: restore-data + persistentVolumeClaim: + claimName: restore-pvc \ No newline at end of file diff --git a/docs/guides/platforms/rook/examples/repository.yaml b/docs/guides/platforms/rook/examples/repository.yaml deleted file mode 100644 index 60f22f7..0000000 --- a/docs/guides/platforms/rook/examples/repository.yaml +++ /dev/null @@ -1,12 +0,0 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: rook-repo - namespace: demo -spec: - backend: - s3: - endpoint: 'http://rook-ceph-rgw-my-store-external.rook-ceph.svc' - bucket: rook-bucket - prefix: /source/data - storageSecretName: rook-secret diff --git a/docs/guides/platforms/rook/examples/restoresession.yaml b/docs/guides/platforms/rook/examples/restoresession.yaml index 3d1bcc5..63f99bd 100644 --- a/docs/guides/platforms/rook/examples/restoresession.yaml +++ b/docs/guides/platforms/rook/examples/restoresession.yaml @@ -1,19 +1,21 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: RestoreSession metadata: - name: deployment-restore + name: sample-restore namespace: demo spec: - repository: - name: rook-repo - target: # target indicates where the recovered data will be stored - ref: - apiVersion: apps/v1 - kind: Deployment - name: stash-recovered - volumeMounts: - - name: restore-data - mountPath: /source/data - rules: - - paths: - - /source/data/ + target: + apiGroup: apps + kind: Deployment + name: kubestash-recovered + namespace: demo + dataSource: + repository: rook-demo-repo + snapshot: latest + encryptionSecret: + name: encryption-secret + namespace: demo + addon: + name: workload-addon + tasks: + - name: logical-backup-restore \ No newline at end of file diff --git a/docs/guides/platforms/rook/examples/retentionpolicy.yaml b/docs/guides/platforms/rook/examples/retentionpolicy.yaml new file mode 100644 index 0000000..4591562 --- /dev/null +++ b/docs/guides/platforms/rook/examples/retentionpolicy.yaml @@ -0,0 +1,15 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: All \ No newline at end of file diff --git a/docs/guides/platforms/rook/index.md b/docs/guides/platforms/rook/index.md index 4f4fce6..edff2f4 100644 --- a/docs/guides/platforms/rook/index.md +++ b/docs/guides/platforms/rook/index.md @@ -1,6 +1,6 @@ --- -title: Rook | Stash -description: Using Stash with Rook Storage Service +title: Rook | KubeStash +description: Using KubeStash with Rook Storage Service menu: docs_{{ .version }}: identifier: platforms-rook @@ -12,21 +12,23 @@ menu_name: docs_{{ .version }} section_menu_id: guides --- -# Using Stash with Rook Managed Ceph Storage +# Using KubeStash with Rook Managed Ceph Storage -This guide will show you how to use Stash to backup and restore volumes of a Kubernetes workload in [Rook](https://rook.io/) managed Ceph storage. Here, we are going to backup a volume of a Deployment into [AWS S3](https://aws.amazon.com/s3/) compatible [Ceph Object Storage](https://rook.io/docs/rook/v1.0/ceph-object.html). Then, we are going to show how to restore this volume into a PersistentVolumeClaim of [Ceph Object Storage](https://rook.io/docs/rook/v1.0/ceph-object.html). We are going to also re-deploy deployment using this recovered volume. +This guide will show you how to use KubeStash to backup and restore volumes of a Kubernetes workload in [Rook](https://rook.io/) managed Ceph storage. Here, we are going to backup a volume of a Deployment into [AWS S3](https://aws.amazon.com/s3/) compatible [Ceph Object Storage](https://rook.io/docs/rook/v1.0/ceph-object.html). Then, we are going to show how to restore this volume into a PersistentVolumeClaim of [Ceph Object Storage](https://rook.io/docs/rook/v1.0/ceph-object.html). We are going to also re-deploy deployment using this recovered volume. ## Before You Begin + - At first, you need to have a Kubernetes cluster. If you don't already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). -- Install `Stash` in your cluster following the steps [here](/docs/setup/README.md). +- Install `KubeStash` in your cluster following the steps [here](/docs/setup/README.md). -- You should be familiar with the following `Stash` concepts: +- You should be familiar with the following `KubeStash` concepts: - [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md) - [BackupSession](/docs/concepts/crds/backupsession/index.md) - [RestoreSession](/docs/concepts/crds/restoresession/index.md) - - [Repository](/docs/concepts/crds/repository/index.md) + - [BackupStorage](/docs/concepts/crds/backupstorage/index.md) + - You will need a [Rook](https://rook.io/docs/rook/v1.7/quickstart.html) deployment with [Ceph Object Storage](https://rook.io/docs/rook/v1.7/ceph-object.html) and [Ceph Block Storage](https://rook.io/docs/rook/v1.7/ceph-block.html) configured. If you do not already have a Rook Storage Service configured, you can create one by following this [quickstart](https://rook.io/docs/rook/v1.7/quickstart.html) guide. To keep everything isolated, we are going to use a separate namespace called `demo` throughout this tutorial. @@ -49,9 +51,9 @@ rook-ceph-block ceph.rook.io/block 89m standard (default) k8s.io/minikube-hostpath 104m ``` -## Backup the Volume of a Deployment +## Backup the Volumes of a Deployment -Here, we are going to deploy a Deployment with a PVC. This Deployment will automatically generate some sample data into the PVC. Then, we are going to backup this sample data using Stash. +Here, we are going to deploy a Deployment with a PVC. This Deployment will automatically generate some sample data into the PVC. Then, we are going to backup this sample data using KubeStash. ### Prepare Workload @@ -69,7 +71,7 @@ metadata: namespace: demo spec: accessModes: - - ReadWriteOnce + - ReadWriteOnce storageClassName: "rook-ceph-block" resources: requests: @@ -94,18 +96,18 @@ apiVersion: apps/v1 kind: Deployment metadata: labels: - app: stash-demo - name: stash-demo + app: kubestash-demo + name: kubestash-demo namespace: demo spec: - replicas: 1 + replicas: 3 selector: matchLabels: - app: stash-demo + app: kubestash-demo template: metadata: labels: - app: stash-demo + app: kubestash-demo name: busybox spec: containers: @@ -122,31 +124,29 @@ spec: - name: source-data persistentVolumeClaim: claimName: source-pvc - strategy: - rollingUpdate: - maxSurge: "0%" - maxUnavailable: "100%" ``` Let's create the Deployment we have shown above. ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/platforms/rook/examples/deployment.yaml -deployment.apps/stash-demo created +deployment.apps/kubestash-demo created ``` Now, wait for the pods of the Deployment to go into the `Running` state. ```bash $ kubectl get pod -n demo -NAME READY STATUS RESTARTS AGE -stash-demo-69f9ffbbf7-98lth 1/1 Running 0 13s +NAME READY STATUS RESTARTS AGE +kubestash-demo-69f9ffbbf7-6wwtr 1/1 Running 0 60s +kubestash-demo-69f9ffbbf7-88kgj 1/1 Running 0 60s +kubestash-demo-69f9ffbbf7-q8qld 1/1 Running 0 60s ``` To verify that the sample data has been created in `/source/data` directory, use the following command: ```bash -$ kubectl exec -n demo stash-demo-69f9ffbbf7-98lth -- cat /source/data/data.txt +$ kubectl exec -n demo kubestash-demo-69f9ffbbf7-6wwtr -- cat /source/data/data.txt sample_data ``` @@ -159,11 +159,9 @@ We are going to store our backed up data into an [Ceph Storage Bucket](https://r Let's create a secret called `rook-secret` with access credentials to our desired [Ceph Storage Bucket](https://rook.io/docs/rook/v1.0/ceph-storage.html), ```bash -$ echo -n 'changeit' > RESTIC_PASSWORD $ echo -n '' > AWS_ACCESS_KEY_ID $ echo -n '' > AWS_SECRET_ACCESS_KEY $ kubectl create secret generic -n demo rook-secret \ - --from-file=./RESTIC_PASSWORD \ --from-file=./AWS_ACCESS_KEY_ID \ --from-file=./AWS_SECRET_ACCESS_KEY secret/rook-secret created @@ -180,10 +178,9 @@ apiVersion: v1 data: AWS_ACCESS_KEY_ID: Tk5HTkcwUVowS1lVOEhKMEFQVEQ= AWS_SECRET_ACCESS_KEY: alExVHJFNU9oa2QzUEk0QzlQbkYwTjVaV0hvd2Yycm9BS2U1MEVsdA== - RESTIC_PASSWORD: Y2hhbmdlaXQ= kind: Secret metadata: - creationTimestamp: "2019-07-25T10:34:25Z" + creationTimestamp: "2024-02-23T05:32:40Z" name: rook-secret namespace: demo resourceVersion: "43037" @@ -192,281 +189,249 @@ metadata: type: Opaque ``` -**Create Repository:** +**Create BackupStorage:** -Now, let's create a `Repository` with the information of our desired Ceph storage bucket. Below is the YAML of `Repository` crd we are going to create, +Now, create a `BackupStorage` using this secret. Below is the YAML of `BackupStorage` crd we are going to create, ```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: rook-repo + name: rook-storage namespace: demo spec: - backend: + storage: + provider: s3 s3: + secretName: rook-secret endpoint: 'http://rook-ceph-rgw-my-store-external.rook-ceph.svc' bucket: rook-bucket - prefix: /source/data - storageSecretName: rook-secret + region: us-east-1 + prefix: /demo + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut ``` - -Let's create the `Repository` we have shown above, +Let's create the BackupStorage we have shown above, ```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/platforms/rook/examples/repository.yaml -repository.stash.appscode.com/rook-repo created +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/platforms/rook/examples/backupstorage.yaml +backupstorage.storage.kubestash.com/rook-storage created ``` Now, we are ready to backup our sample data into this backend. +**Create RetentionPolicy:** + +Now, let's create a `RetentionPolicy` to specify how the old Snapshots should be cleaned up. + +Below is the YAML of the `RetentionPolicy` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: All +``` +Notice the `spec.usagePolicy` that allows referencing the `RetentionPolicy` from all namespaces. To allow specific namespaces, we can configure it accordingly by following [RetentionPolicy usage policy](/docs/concepts/crds/retentionpolicy/index.md#retentionpolicy-spec). + +Let’s create the above `RetentionPolicy`, +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/platforms/rook/examples/retentionpolicy.yaml +retentionpolicy.storage.kubestash.com/demo-retention created +``` ### Backup -We have to create a `BackupConfiguration` crd targeting the `stash-demo` Deployment that we have deployed earlier. Stash will inject a sidecar container into the target. It will also create a `CronJob` to take a periodic backup of `/source/data` directory of the target. +We have to create a `BackupConfiguration` crd targeting the `kubestash-demo` Deployment that we have deployed earlier. Then, KubeStash will create a `CronJob` for each session to take periodic backup of `/source/data` directory of the target. + +At first, we need to create a secret with a Restic password for backup data encryption. + +**Create Secret:** + +Let's create a secret called `encry-secret` with the Restic password, + +```bash +$ echo -n 'changeit' > RESTIC_PASSWORD +$ kubectl create secret generic -n demo encry-secret \ + --from-file=./RESTIC_PASSWORD \ +secret "encryption-secret" created +``` **Create BackupConfiguration:** Below is the YAML of the `BackupConfiguration` crd that we are going to create, + ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: - name: deployment-backup + name: sample-backup-dep namespace: demo spec: - repository: - name: rook-repo - schedule: "*/5 * * * *" target: - ref: - apiVersion: apps/v1 - kind: Deployment - name: stash-demo - volumeMounts: - - name: source-data - mountPath: /source/data - paths: - - /source/data - retentionPolicy: - name: 'keep-last-5' - keepLast: 5 - prune: true + apiGroup: apps + kind: Deployment + name: kubestash-demo + namespace: demo + backends: + - name: rook-backend + storageRef: + name: rook-storage + namespace: demo + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: demo-session + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: rook-demo-repo + backend: rook-backend + directory: /dep + encryptionSecret: + name: encryption-secret + namespace: demo + addon: + name: workload-addon + tasks: + - name: logical-backup + targetVolumes: + volumeMounts: + - name: source-data + mountPath: /source/data + params: + paths: /source/data + exclude: /source/data/lost+found + retryConfig: + maxRetry: 2 + delay: 1m ``` -Here, - -- `spec.repository` refers to the `Repository` object `rook-repo` that holds backend information. -- `spec.target.ref` refers to the `stash-demo` Deployment for backup target. -- `spec.target.volumeMounts` specifies a list of volumes and their mountPath that contain the target paths. -- `spec.target.paths` specifies list of file paths to backup. - Let's create the `BackupConfiguration` crd we have shown above, ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/platforms/rook/examples/backupconfiguration.yaml -backupconfiguration.stash.appscode.com/deployment-backup created +backupconfiguration.core.kubestash.com/sample-backup-dep created ``` -**Verify Sidecar:** -If everything goes well, Stash will inject a sidecar container into the `stash-demo` Deployment to take backup of `/source/data` directory. Let’s check that the sidecar has been injected successfully, +**Verify Backup Setup Successful** -```bash -$ kubectl get pod -n demo -NAME READY STATUS RESTARTS AGE -stash-demo-76d78d8966-nbkrl 2/2 Running 0 39s -``` +If everything goes well, the phase of the `BackupConfiguration` should be `Ready`. The `Ready` phase indicates that the backup setup is successful. Let's verify the `Phase` of the BackupConfiguration, -Look at the pod. It now has 2 containers. If you view the resource definition of this pod, you will see that there is a container named `stash` which is running `run-backup` command. - -```yaml -$ kubectl get pod -n demo stash-demo-76d78d8966-nbkrl -o yaml -apiVersion: v1 -kind: Pod -metadata: - generateName: stash-demo-76d78d8966- - labels: - app: stash-demo - pod-template-hash: 76d78d8966 - name: stash-demo-76d78d8966-nbkrl - namespace: demo - ... -spec: - containers: - - args: - - echo sample_data > /source/data/data.txt && sleep 3000 - command: - - /bin/sh - - -c - image: busybox - imagePullPolicy: IfNotPresent - name: busybox - resources: {} - terminationMessagePath: /dev/termination-log - terminationMessagePolicy: File - volumeMounts: - - mountPath: /source/data - name: source-data - - mountPath: /var/run/secrets/kubernetes.io/serviceaccount - name: default-token-2njxz - readOnly: true - - args: - - run-backup - - --backup-configuration=deployment-backup - - --secret-dir=/etc/stash/repository/secret - - --enable-cache=true - - --max-connections=0 - - --metrics-enabled=true - - --pushgateway-url=http://stash-operator.kube-system.svc:56789 - - --enable-status-subresource=true - - --use-kubeapiserver-fqdn-for-aks=true - - --logtostderr=true - - --alsologtostderr=false - - --v=3 - - --stderrthreshold=0 - env: - - name: NODE_NAME - valueFrom: - fieldRef: - apiVersion: v1 - fieldPath: spec.nodeName - - name: POD_NAME - valueFrom: - fieldRef: - apiVersion: v1 - fieldPath: metadata.name - image: suaas21/stash:volumeTemp_linux_amd64 - imagePullPolicy: IfNotPresent - name: stash - resources: {} - terminationMessagePath: /dev/termination-log - terminationMessagePolicy: File - volumeMounts: - - mountPath: /etc/stash - name: stash-podinfo - - mountPath: /etc/stash/repository/secret - name: stash-secret-volume - - mountPath: /tmp - name: tmp-dir - - mountPath: /source/data - name: source-data - - mountPath: /var/run/secrets/kubernetes.io/serviceaccount - name: default-token-2njxz - readOnly: true - dnsPolicy: ClusterFirst - enableServiceLinks: true - nodeName: minikube - priority: 0 - restartPolicy: Always - schedulerName: default-scheduler - securityContext: {} - serviceAccount: default - serviceAccountName: default - terminationGracePeriodSeconds: 30 - tolerations: - - effect: NoExecute - key: node.kubernetes.io/not-ready - operator: Exists - tolerationSeconds: 300 - - effect: NoExecute - key: node.kubernetes.io/unreachable - operator: Exists - tolerationSeconds: 300 - volumes: - - name: source-data - persistentVolumeClaim: - claimName: source-pvc - - emptyDir: {} - name: tmp-dir - - downwardAPI: - defaultMode: 420 - items: - - fieldRef: - apiVersion: v1 - fieldPath: metadata.labels - path: labels - name: stash-podinfo - - name: stash-secret-volume - secret: - defaultMode: 420 - secretName: rook-secret - - name: default-token-2njxz - secret: - defaultMode: 420 - secretName: default-token-2njxz - ... -... +```bash +$ kubectl get backupconfiguration -n demo +NAME PHASE PAUSED AGE +sample-backup-dep Ready 2m50s ``` **Verify CronJob:** -It will also create a `CronJob` with the schedule specified in `spec.schedule` field of `BackupConfiguration` crd. +It will also create a `CronJob` with the schedule specified in `spec.sessions[*].scheduler.schedule` field of `BackupConfiguration` crd. Verify that the `CronJob` has been created using the following command, ```bash $ kubectl get cronjob -n demo -NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE -deployment-backup */1 * * * * False 0 13s 1m50s +NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE +trigger-sample-backup-dep-demo-session */5 * * * * 0 2m45s 3m25s ``` **Wait for BackupSession:** -The `deployment-backup` CronJob will trigger a backup on each schedule by creating a `BackupSession` crd. The sidecar container will watch for the `BackupSession` crd. When it finds one, it will take backup immediately. - Wait for the next schedule for backup. Run the following command to watch `BackupSession` crd, ```bash -$ watch -n 3 kubectl get backupsession -n demo -Every 3.0s: kubectl get backupsession -n demo suaas-appscode: Mon Jul 22 15:01:21 2019 +$ kubectl get backupsession -n demo -w -NAME INVOKER-TYPE INVOKER-NAME PHASE AGE -deployment-backup-1663786062 BackupConfiguration deployment-backup Succeeded 21s +NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE +sample-backup-dep-demo-session-1706015400 BackupConfiguration sample-backup-dep Succeeded 7m22s ``` We can see from the above output that the backup session has succeeded. Now, we are going to verify whether the backed up data has been stored in the backend. **Verify Backup:** -Once a backup is complete, Stash will update the respective `Repository` crd to reflect the backup. Check that the repository `rook-repo` has been updated by the following command, +Once a backup is complete, KubeStash will update the respective `Repository` crd to reflect the backup. Check that the repository `rook-demo-repo` has been updated by the following command, ```bash -$ kubectl get repository -n demo -NAME INTEGRITY SIZE SNAPSHOT-COUNT LAST-SUCCESSFUL-BACKUP AGE -rook-repo true 30 B 2 3m10s 5m20s +$ kubectl get repository -n demo rook-demo-repo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +rook-demo-repo true 1 806 B Ready 8m27s 9m18s ``` -Here, `BACKUPCOUNT` field indicates the number of backup snapshots has taken in this repository. - -## Restore the Backed up Data - -This section will show you how to restore the backed up data from [Ceph Storage Bucket](https://rook.io/docs/rook/v1.0/ceph-storage.html) we have taken in the earlier section. +At this moment we have one `Snapshot`. Run the following command to check the respective `Snapshot` which represents the state of a backup run to a particular `Repository`. -**Stop Taking Backup of the Old Deployment:** +```bash +$ kubectl get snapshots -n demo -l=kubestash.com/repo-name=rook-demo-repo +NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE AGE +rook-demo-repo-sample-backup-dep-demo-session-1706015400 rook-demo-repo demo-session 2024-01-23T13:10:54Z Delete Succeeded 16h +``` -At first, let's stop taking any further backup of the old Deployment so that no backup is taken during the restore process. We are going to pause the `BackupConfiguration` that we created to backup the `stash-demo` Deployment. Then, Stash will stop taking any further backup for this Deployment. You can learn more how to pause a scheduled backup [here](/docs/guides/use-cases/pause-backup/index.md) +> When a backup is triggered according to schedule, KubeStash will create a `Snapshot` with the following labels `kubestash.com/app-ref-kind: `, `kubestash.com/app-ref-name: `, `kubestash.com/app-ref-namespace: ` and `kubestash.com/repo-name: `. We can use these labels to watch only the `Snapshot` of our desired Workload or `Repository`. -Let's pause the `deployment-backup` BackupConfiguration, +If we check the YAML of the `Snapshot`, we can find the information about the backed up components of the Deployment. ```bash -$ kubectl patch backupconfiguration -n demo deployment-backup --type="merge" --patch='{"spec": {"paused": true}}' -backupconfiguration.stash.appscode.com/deployment-backup patched +$ kubectl get snapshots -n demo rook-demo-repo-sample-backup-dep-demo-session-1706015400 -oyaml ``` -Now, wait for a moment. Stash will pause the BackupConfiguration. Verify that the BackupConfiguration has been paused, - -```bash -$ kubectl get backupconfiguration -n demo -NAME TASK SCHEDULE PAUSED AGE -deployment-backup */1 * * * * true 26m +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: Snapshot +metadata: + labels: + kubestash.com/app-ref-kind: Deployment + kubestash.com/app-ref-name: kubestash-demo + kubestash.com/app-ref-namespace: demo + kubestash.com/repo-name: rook-demo-repo + name: rook-demo-repo-sample-backup-dep-demo-session-1706015400 + namespace: demo +spec: + ... +status: + components: + dump: + driver: Restic + duration: 7.534461497s + integrity: true + path: repository/v1/demo-session/dump + phase: Succeeded + resticStats: + - hostPath: /source/data + id: f28441a36b2167d64597d66d1046573181cad81aa8ff5b0998b64b31ce16f077 + size: 11 B + uploaded: 1.049 KiB + size: 806 B + ... ``` -Notice the `PAUSED` column. Value `true` for this field means that the BackupConfiguration has been paused. +> For Deployment, KubeStash takes backup from only one pod of the Deployment. So, only one component has been taken backup. + +## Restore the Backed up Data + +This section will show you how to restore the backed up data from [Ceph Storage Bucket](https://rook.io/docs/rook/v1.0/ceph-storage.html) we have taken in the earlier section. **Deploy Deployment:** -We are going to create a new Deployment named `stash-recovered` with a new PVC and restore the backed up data inside it. +We are going to create a new Deployment named `kubestash-recovered` with a new PVC and restore the backed up data inside it. Below are the YAMLs of the Deployment and PVC that we are going to create, @@ -478,7 +443,7 @@ metadata: namespace: demo spec: accessModes: - - ReadWriteOnce + - ReadWriteOnce storageClassName: "rook-ceph-block" resources: requests: @@ -488,39 +453,35 @@ apiVersion: apps/v1 kind: Deployment metadata: labels: - app: stash-recovered - name: stash-recovered + app: kubestash-recovered + name: kubestash-recovered namespace: demo spec: - replicas: 1 + replicas: 3 selector: matchLabels: - app: stash-recovered + app: kubestash-recovered template: metadata: labels: - app: stash-recovered + app: kubestash-recovered name: busybox spec: containers: - - args: - - sleep - - "3600" - image: busybox - imagePullPolicy: IfNotPresent - name: busybox - volumeMounts: - - mountPath: /restore/data - name: restore-data + - args: + - sleep + - "3600" + image: busybox + imagePullPolicy: IfNotPresent + name: busybox + volumeMounts: + - mountPath: /source/data + name: restore-data restartPolicy: Always volumes: - - name: restore-data - persistentVolumeClaim: - claimName: restore-pvc - strategy: - rollingUpdate: - maxSurge: "0%" - maxUnavailable: "100%" + - name: restore-data + persistentVolumeClaim: + claimName: restore-pvc ``` Let's create the Deployment and PVC we have shown above. @@ -528,161 +489,79 @@ Let's create the Deployment and PVC we have shown above. ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/platforms/rook/examples/recovered_deployment.yaml persistentvolumeclaim/restore-pvc created -deployment.apps/stash-recovered created +deployment.apps/kubestash-recovered created ``` **Create RestoreSession:** -Now, we need to create a `RestoreSession` crd targeting the `stash-recovered` Deployment. +Now, we need to create a `RestoreSession` crd targeting the `kubestash-recovered` Deployment. Below is the YAML of the `RestoreSesion` crd that we are going to create, ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: RestoreSession metadata: - name: deployment-restore + name: sample-restore namespace: demo spec: - repository: - name: rook-repo - target: # target indicates where the recovered data will be stored - ref: - apiVersion: apps/v1 - kind: Deployment - name: stash-recovered - volumeMounts: - - name: restore-data - mountPath: /source/data - rules: - - paths: - - /source/data/ + target: + apiGroup: apps + kind: Deployment + name: kubestash-recovered + namespace: demo + dataSource: + repository: rook-demo-repo + snapshot: latest + encryptionSecret: + name: encryption-secret + namespace: demo + addon: + name: workload-addon + tasks: + - name: logical-backup-restore ``` Here, -- `spec.repository.name` specifies the `Repository` crd that holds the backend information where our backed up data has been stored. -- `spec.target.ref` refers to the target workload where the recovered data will be stored. -- `spec.target.volumeMounts` specifies a list of volumes and their mountPath where the data will be restored. - - `mountPath` must be same `mountPath` as the original volume because Stash stores absolute path of the backed up files. If you use different `mountPath` for the restored volume the backed up files will not be restored into your desired volume. +- `spec.dataSource.snapshot` specifies to restore from latest `Snapshot`. Let's create the `RestoreSession` crd we have shown above, ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/platforms/rook/examples/restoresession.yaml -restoresession.stash.appscode.com/deployment-restore created -``` - -Once, you have created the `RestoreSession` crd, Stash will inject `init-container` into `stash-recovered` Deployment. The Deployment will restart and the `init-container` will restore the desired data on start-up. - -**Verify Init-Container:** - -Wait until the `init-container` has been injected into the `stash-recovered` Deployment. Let’s describe the Deployment to verify that `init-container` has been injected successfully. - -```yaml -$ kubectl describe deployment -n demo stash-recovered -Name: stash-recovered -Namespace: demo -CreationTimestamp: Thu, 25 Jul 2019 19:14:18 +0600 -Labels: app=stash-recovered -Selector: app=stash-recovered -Replicas: 1 desired | 1 updated | 1 total | 1 available | 0 unavailable -StrategyType: RollingUpdate -MinReadySeconds: 0 -RollingUpdateStrategy: 100% max unavailable, 0% max surge -... -Pod Template: - Labels: app=stash-recovered - Annotations: stash.appscode.com/last-applied-restoresession-hash: 10604314699482840900 - Init Containers: - stash-init: - Image: suaas21/stash:volumeTemp_linux_amd64 - Port: - Host Port: - Args: - restore - --restore-session=deployment-restore - --secret-dir=/etc/stash/repository/secret - --enable-cache=true - --max-connections=0 - --metrics-enabled=true - --pushgateway-url=http://stash-operator.kube-system.svc:56789 - --enable-status-subresource=true - --use-kubeapiserver-fqdn-for-aks=true - --logtostderr=true - --alsologtostderr=false - --v=3 - --stderrthreshold=0 - Environment: - NODE_NAME: (v1:spec.nodeName) - POD_NAME: (v1:metadata.name) - Mounts: - /etc/stash/repository/secret from stash-secret-volume (rw) - /source/data from restore-data (rw) - /tmp from tmp-dir (rw) - Containers: - busybox: - Image: busybox - Port: - Host Port: - Args: - sleep - 3600 - Environment: - Mounts: - /restore/data from restore-data (rw) - Volumes: - restore-data: - Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace) - ClaimName: restore-pvc - ReadOnly: false - tmp-dir: - Type: EmptyDir (a temporary directory that shares a pod's lifetime) - Medium: - SizeLimit: - stash-podinfo: - Type: DownwardAPI (a volume populated by information about the pod) - Items: - metadata.labels -> labels - stash-secret-volume: - Type: Secret (a volume populated by a Secret) - SecretName: rook-secret - Optional: false -... -``` - -Notice the `Init-Containers` section. We can see that the init-container `stash-init` has been injected which is running `restore` command. - -**Wait for RestoreSession to Succeeded:** - -Now, wait for the restore process to complete. You can watch the `RestoreSession` phase using the following command, +restoresession.core.kubestash.com/deployment-restore created +```` +Once, you have created the `RestoreSession` object, KubeStash will create restore Job(s). Run the following command to watch the phase of the `RestoreSession` object, ```bash -$ watch -n 2 kubectl get restoresession -n demo -Every 3.0s: kubectl get restoresession --all-namespaces suaas-appscode: Mon Jul 28 18:17:22 2019 +$ watch kubectl get restoresession -n demo +Every 2.0s: kubectl get restores... AppsCode-PC-03: Wed Jan 10 17:13:18 2024 -NAMESPACE NAME REPOSITORY-NAME PHASE AGE -demo deployment-restore rook-repo Succeeded 100s +NAME REPOSITORY FAILURE-POLICY PHASE DURATION AGE +deployment-restore rook-demo-repo Succeeded 3s 53s ``` -So, we can see from the output of the above command that the restore process has succeeded. +The `Succeeded` phase means that the restore process has been completed successfully. > **Note:** If you want to restore the backed up data inside the same Deployment whose volumes were backed up, you have to remove the corrupted data from the Deployment. Then, you have to create a RestoreSession targeting the Deployment. **Verify Restored Data:** -In this section, we are going to verify that the desired data has been restored successfully. At first, check if the `stash-recovered` pod of the Deployment has gone into `Running` state by the following command, +In this section, we are going to verify that the desired data has been restored successfully. At first, check if the `kubestash-recovered` pod of the Deployment has gone into `Running` state by the following command, ```bash $ kubectl get pod -n demo -NAME READY STATUS RESTARTS AGE -stash-recovered-5c59587895-76tsx 1/1 Running 0 73s +NAME READY STATUS RESTARTS AGE +kubestash-recovered-6f5c46fdbf-lfthv 1/1 Running 0 2m39s +kubestash-recovered-6f5c46fdbf-s7rrq 1/1 Running 0 2m15s +kubestash-recovered-6f5c46fdbf-vn2z8 1/1 Running 0 2m35s ``` -Verify that the sample data has been restored in `/restore/data` directory of the `stash-recovered` pod of the Deployment using the following command, +Verify that the sample data has been restored in `/source/data` directory of the `kubestash-recovered` pod of the Deployment using the following command, ```bash -$ kubectl exec -n demo stash-recovered-5c59587895-76tsx -- cat /restore/data/data.txt +$ kubectl exec -n demo kubestash-recovered-6f5c46fdbf-s7rrq -- cat /source/data/data.txt sample_data ``` @@ -691,11 +570,13 @@ sample_data To clean up the Kubernetes resources created by this tutorial, run: ```bash -kubectl delete -n demo deployment stash-demo -kubectl delete -n demo deployment stash-recovered +kubectl delete -n demo deployment kubestash-demo +kubectl delete -n demo deployment kubestash-recovered kubectl delete -n demo backupconfiguration deployment-backup kubectl delete -n demo restoresession deployment-restore -kubectl delete -n demo repository rook-repo +kubectl delete -n demo backupstorage rook-storage +kubectl delete -n demo retentionpolicy demo-retention +kubectl delete -n demo secret encryption-secret kubectl delete -n demo secret rook-secret kubectl delete -n demo pvc --all ``` diff --git a/docs/guides/use-cases/ownership/index.md b/docs/guides/use-cases/ownership/index.md index a68c4c4..311674d 100644 --- a/docs/guides/use-cases/ownership/index.md +++ b/docs/guides/use-cases/ownership/index.md @@ -1,9 +1,9 @@ --- -title: File Ownership | Stash -description: Handling Restored File Ownership in Stash +title: File Ownership | KubeStash +description: Handling Restored File Ownership in KubeStash menu: docs_{{ .version }}: - identifier: file-ownership-stash + identifier: file-ownership-kubestash name: File Ownership parent: use-cases weight: 40 @@ -12,33 +12,32 @@ menu_name: docs_{{ .version }} section_menu_id: guides --- -# Handling Restored File Ownership in Stash +# Handling Restored File Ownership in KubeStash -Stash preserves permission bits of the restored files. However, it may change ownership (owner `uid` and `gid`) of the restored files in some cases. This tutorial will explain when and how ownership of the restored files can be changed. Then, we are going to explain how we can avoid or resolve this problem. +KubeStash preserves permission bits of the restored files. However, it may change ownership (owner `uid` and `gid`) of the restored files in some cases. This tutorial will explain when and how ownership of the restored files can be changed. Then, we are going to explain how we can avoid or resolve this problem. ## Understanding Backup and Restore Behaviour At first, let's understand how backup and restore behaves in different scenario. A table with some possible backup and restore scenario is given below. We have run the containers as different user in different scenario using [SecurityContext](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/). -| Case | Original File Owner | Backup `sidecar` User | Backup Succeed? | Restore `init-container` User | Restore Succeed? | Restored File Owner | Restored File Editable to Original Owner? | -| :---: | :-----------------: | :-------------------: | :-------------: | :---------------------------: | :-----------------------: | :-----------------: | :---------------------------------------: | -| 1 | root | stash (1005) | ✓ | stash(1005) | ✓ | 1005 | ✓ | -| 2 | 2000 | stash(1005) | ✓ | stash(1005) | ✓ | 1005 | ✗ | -| 3 | 2000 | root | ✓ | root | ✓ | 2000 | ✓ | -| 4 | 2000 | root | ✓ | stash(1005) | ✓ (remote backend) | 1005 | ✗ | -| 5 | 2000 | root | ✓ | stash(1005) | ✗ (local backend) | - | - | -| 6 | 2000 | 3000 | ✓ | stash(1005) | ✓ (remote backend) | 1005 | ✗ | -| 7 | 2000 | 3000 | ✓ | stash(1005) | ✗ (local backend) | - | - | -| 8 | 2000 | 3000 | ✓ | root | ✓ | 2000 | ✓ | -| 9 | 2000 | 3000 | ✓ | 3000 | ✓ | 3000 | ✗ | +| Case | Original File Owner | Backup `Job` User | Backup Succeed? | Restore `Job` User | Restore Succeed? | Restored File Owner | Restored File Editable to Original Owner? | +|:----:|:-------------------:|:-----------------:|:---------------:|:------------------:|:----------------:|:-------------------:|:-----------------------------------------:| +| 1 | root | kubestash(65534) | ✓ | kubstash(root) | ✓ | root | ✓ | +| 2 | root | kubestash(65534) | ✓ | 65534 | ✓ | root | ✗ | +| 4 | 2000 | kubestash(65534) | ✓ | kubestash(root) | ✓ | 2000 | ✓ | +| 4 | 2000 | kubestash(65534) | ✓ | 65534 | ✓ | 65534 | ✗ | +| 5 | 2000 | root | ✓ | 65534 | ✓ | 65534 | ✗ | +| 6 | 2000 | 3000 | ✓ | kubestash(root) | ✓ | 2000 | ✓ | +| 7 | 2000 | 3000 | ✓ | 65534 | ✓ | 65534 | ✗ | +| 8 | 2000 | 3000 | ✓ | 3000 | ✓ | 3000 | ✗ | If we look at the table carefully, we are going to notice the following behaviors: -1. The user of the backup `sidecar` does not have any effect on backup. It just needs read permission of the target files. -2. If the restore container runs as `root` user then original ownership of the restored files are preserved. -3. If the restore container runs as `non-root` user then the ownership of the restored files is changed to restore container's user and restored files become read-only to the original user unless it was `root` user. +1. The user of the backup `job` does not have any effect on backup. It just needs read permission of the target files. +2. If the restore job runs as `root` user then original ownership of the restored files are preserved. +3. If the restore job runs as `non-root` user then the ownership of the restored files is changed to restore jobs's user and restored files become read-only to the original user unless it was `root` user. -So, we can see when we run restore container as `non-root` user, it raises some serious concerns as the restored files become read-only to the original user. Next section will discuss how we can avoid or fix this problem. +So, we can see when we run restore job as `non-root` user, it raises some serious concerns as the restored files become read-only to the original user. Next section will discuss how we can avoid or fix this problem. ## Avoid or Fix Ownership Issue @@ -51,43 +50,43 @@ There could be two scenarios for the restored files user. ### Restored files user is the same as the original user -This is likely to be the most common scenario. Generally, the same application whose data was backed up will use the restored files after a restore. In this case, if your cluster supports running containers as `root` user, then it is fairly easy to avoid this issue. You just need to run the restore container as `root` user which Stash does by default. However, things get little more complicated when your cluster does not support running containers as `root` user. In this case, you can do followings: +This is likely to be the most common scenario. Generally, the same application whose data was backed up will use the restored files after a restore. In this case, if your cluster supports running containers as `root` user, then it is fairly easy to avoid this issue. You just need to run the restore job as `root` user which kubeStash does by default. However, things get little more complicated when your cluster does not support running containers as `root` user. In this case, you can do followings: -- Run restore container as the same user as the original container. +- Run restore job as the same user as the original job. - Change the ownership of the restored files using `chown` after the restore is completed. -For the first method, we can achieve this configuring SecurityContext under `RuntimeSetting` of `RestoreSession` object. A sample `RestoreSession` objects configured SecurityContext to run as the same user as the original user (let original user is 2000) is shown below. +For the first method, we can achieve this configuring SecurityContext under `SecurityContext` of `RestoreSession.spec.addon.jobTemplate.spec` object. A sample `RestoreSession` objects configured SecurityContext to run as the same user as the original user (let original user is 2000) is shown below. ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: RestoreSession metadata: name: deployment-restore namespace: demo spec: - repository: - name: local-repo - rules: - - paths: - - /source/data target: - ref: - apiVersion: apps/v1 - kind: Deployment - name: stash-demo - volumeMounts: - - name: source-data - mountPath: /source/data - runtimeSettings: - container: - securityContext: - runAsUser: 2000 - runAsGroup: 2000 + apiGroup: + kind: Deployment + name: kubestash-demo + namespace: demo + dataSource: + repository: deployment-demo-gcs + snapshot: latest + encryptionSecret: + name: encryption-secret # some addon may not support encryption + namespace: demo + addon: + name: workload-addon + tasks: + - name: logical-backup-restore + jobTemplate: + spec: + securityContext: + runAsUser: 2000 + runAsGroup: 2000 ``` ->If you are using [local](/docs/guides/backends/local/index.md) backend to store backed up data, then the original container, backup container and restore container all of them must be run as same user. By default, Stash runs backup container as `stash(1005)` user. You can change this to match with the user of your original container using `securityContext` field under `runtimeSettings` of `BackupConfiguration` object. - -The second method is necessary when the backup container was not run as the same user as the original container. This is similar to the process when the restored files user is different than the original user. In this case, you have to change the ownership of restored files using `chown` after restore. +The second method is necessary when the backup job was not run as the same user as the original job. This is similar to the process when the restored files user is different than the original user. In this case, you have to change the ownership of restored files using `chown` after restore. ### Restored file user is different than the original user diff --git a/docs/guides/volume-populator/_index.md b/docs/guides/volume-populator/_index.md new file mode 100644 index 0000000..6cf9679 --- /dev/null +++ b/docs/guides/volume-populator/_index.md @@ -0,0 +1,10 @@ +--- +title: Volume Populator | KubeStash +menu: + docs_{{ .version }}: + identifier: volume-populator + name: Volume Populator + parent: guides + weight: 70 +menu_name: docs_{{ .version }} +--- diff --git a/docs/guides/volume-populator/deployment/examples/backup-deployment.yaml b/docs/guides/volume-populator/deployment/examples/backup-deployment.yaml new file mode 100644 index 0000000..58f9736 --- /dev/null +++ b/docs/guides/volume-populator/deployment/examples/backup-deployment.yaml @@ -0,0 +1,37 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + app: kubestash-demo + name: kubestash-deployment + namespace: demo +spec: + replicas: 2 + selector: + matchLabels: + app: kubestash-demo + template: + metadata: + labels: + app: kubestash-demo + name: busybox + spec: + containers: + - args: ["echo sample_data > /source/data/data.txt; echo sample_config > /source/config/config.cfg && sleep 3000"] + command: ["/bin/sh", "-c"] + image: busybox + imagePullPolicy: IfNotPresent + name: busybox + volumeMounts: + - mountPath: /source/data + name: source-data + - mountPath: /source/config + name: source-config + restartPolicy: Always + volumes: + - name: source-data + persistentVolumeClaim: + claimName: source-data + - name: source-config + persistentVolumeClaim: + claimName: source-config \ No newline at end of file diff --git a/docs/guides/volume-populator/deployment/examples/backup-pvcs.yaml b/docs/guides/volume-populator/deployment/examples/backup-pvcs.yaml new file mode 100644 index 0000000..0137eb1 --- /dev/null +++ b/docs/guides/volume-populator/deployment/examples/backup-pvcs.yaml @@ -0,0 +1,23 @@ +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: source-data + namespace: demo +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi +--- +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: source-config + namespace: demo +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi \ No newline at end of file diff --git a/docs/guides/volume-populator/deployment/examples/backupconfiguration.yaml b/docs/guides/volume-populator/deployment/examples/backupconfiguration.yaml new file mode 100644 index 0000000..6665dae --- /dev/null +++ b/docs/guides/volume-populator/deployment/examples/backupconfiguration.yaml @@ -0,0 +1,41 @@ +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: sample-backup-dep + namespace: demo +spec: + target: + apiGroup: apps + kind: Deployment + name: kubestash-deployment + namespace: demo + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /dep + encryptionSecret: + name: encryption-secret # some addon may not support encryption + namespace: demo + deletionPolicy: WipeOut + addon: + name: workload-addon + tasks: + - name: logical-backup + params: + paths: /source/data,/source/config + exclude: /source/data/lost+found,/source/config/lost+found \ No newline at end of file diff --git a/docs/guides/volume-populator/deployment/examples/backupstorage.yaml b/docs/guides/volume-populator/deployment/examples/backupstorage.yaml new file mode 100644 index 0000000..5d53db5 --- /dev/null +++ b/docs/guides/volume-populator/deployment/examples/backupstorage.yaml @@ -0,0 +1,17 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut diff --git a/docs/guides/volume-populator/deployment/examples/populated-pvcs.yaml b/docs/guides/volume-populator/deployment/examples/populated-pvcs.yaml new file mode 100644 index 0000000..6b89b41 --- /dev/null +++ b/docs/guides/volume-populator/deployment/examples/populated-pvcs.yaml @@ -0,0 +1,35 @@ +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: restored-source-data + namespace: demo + annotations: + populator.kubestash.com/app-name: restored-kubestash-deployment +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + dataSourceRef: + apiGroup: storage.kubestash.com + kind: Snapshot + name: gcs-repository-sample-backup-dep-frequent-backup-1707826500 +--- +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: restored-source-config + namespace: demo + annotations: + populator.kubestash.com/app-name: restored-kubestash-deployment +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + dataSourceRef: + apiGroup: storage.kubestash.com + kind: Snapshot + name: gcs-repository-sample-backup-dep-frequent-backup-1707826500 \ No newline at end of file diff --git a/docs/guides/volume-populator/deployment/examples/restore-deployment.yaml b/docs/guides/volume-populator/deployment/examples/restore-deployment.yaml new file mode 100644 index 0000000..c691211 --- /dev/null +++ b/docs/guides/volume-populator/deployment/examples/restore-deployment.yaml @@ -0,0 +1,36 @@ +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + app: kubestash-populator-demo + name: restored-kubestash-deployment + namespace: demo +spec: + replicas: 1 + selector: + matchLabels: + app: kubestash-populator-demo + template: + metadata: + labels: + app: kubestash-populator-demo + name: busybox + spec: + containers: + - command: ["/bin/sh", "-c", "sleep 3000"] + image: busybox + imagePullPolicy: IfNotPresent + name: busybox + volumeMounts: + - mountPath: /source/data + name: source-data + - mountPath: /source/config + name: source-config + restartPolicy: Always + volumes: + - name: source-data + persistentVolumeClaim: + claimName: restored-source-data + - name: source-config + persistentVolumeClaim: + claimName: restored-source-config \ No newline at end of file diff --git a/docs/guides/volume-populator/deployment/examples/retentionpolicy.yaml b/docs/guides/volume-populator/deployment/examples/retentionpolicy.yaml new file mode 100644 index 0000000..3af9037 --- /dev/null +++ b/docs/guides/volume-populator/deployment/examples/retentionpolicy.yaml @@ -0,0 +1,15 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same \ No newline at end of file diff --git a/docs/guides/volume-populator/deployment/index.md b/docs/guides/volume-populator/deployment/index.md new file mode 100644 index 0000000..4946533 --- /dev/null +++ b/docs/guides/volume-populator/deployment/index.md @@ -0,0 +1,522 @@ +--- +title: Populate Volumes of a Deployment | KubeStash +description: An step by step guide to show how to populate volumes of a Deployment +menu: + docs_{{ .version }}: + identifier: volume-populate-deployment + name: Populate Volumes of a Deployment + parent: volume-populator + weight: 30 +product_name: kubestash +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +# Populating volumes of a Deployment + +This guide will show you how to use KubeStash to populate the volumes of a `Deployment`. Within this guide, we will demonstrate the process of backing up volumes of a `Deployment` and subsequently restoring it using Kubernetes native approach, facilitated by KubeStash. + +## Before You Begin +- At first, you need to have a Kubernetes cluster, and the `kubectl` command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). +- Install `KubeStash` in your cluster following the steps [here](/docs/setup/README.md). +- You should be familiar with the following `KubeStash` concepts: + - [BackupStorage](/docs/concepts/crds/backupstorage/index.md) + - [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md) + - [BackupSession](/docs/concepts/crds/backupsession/index.md) + - [SnapShot](/docs/concepts/crds/snapshot/index.md) + - [RestoreSession](/docs/concepts/crds/restoresession/index.md) + - [RetentionPolicy](/docs/concepts/crds/retentionpolicy/index.md) + +To keep everything isolated, we are going to use a separate namespace called `demo` throughout this tutorial. + +```bash +$ kubectl create ns demo +namespace/demo created +``` + +> **Note:** YAML files used in this tutorial are stored in [docs/guides/volumes/pvc/examples](/docs/guides/volumes/pvc/examples) directory of [kubestash/docs](https://github.com/kubestash/docs) repository. + +## Prepare Workload + +Here, we are going to deploy a Deployment with two PVCs and generate some sample data in it. Then, we are going to backup of these PVCs using KubeStash. + +**Create PersistentVolumeClaim :** + +At first, let's create two sample PVCs. We are going to mount these PVCs in our targeted Deployment. + +Below is the YAML of the sample PVCs, + +```yaml +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: source-data + namespace: demo +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi +--- +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: source-config + namespace: demo +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi +``` + +Let's create the PVCs we have shown above. + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/deployment/examples/backup-pvcs.yaml +persistentvolumeclaim/source-data created +persistentvolumeclaim/source-config created +``` + +**Deploy Deployment :** + +Now, we are going to deploy a Deployment that uses the above PVCs. This Deployment will automatically create `data.txt` and `config.cfg` file in `/source/data` and `/source/config` directory. + +Below is the YAML of the Deployment that we are going to create, + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + app: kubestash-demo + name: kubestash-deployment + namespace: demo +spec: + replicas: 2 + selector: + matchLabels: + app: kubestash-demo + template: + metadata: + labels: + app: kubestash-demo + name: busybox + spec: + containers: + - args: ["echo sample_data > /source/data/data.txt; echo sample_config > /source/config/config.cfg && sleep 3000"] + command: ["/bin/sh", "-c"] + image: busybox + imagePullPolicy: IfNotPresent + name: busybox + volumeMounts: + - mountPath: /source/data + name: source-data + - mountPath: /source/config + name: source-config + restartPolicy: Always + volumes: + - name: source-data + persistentVolumeClaim: + claimName: source-data + - name: source-config + persistentVolumeClaim: + claimName: source-config +``` + +Let's create the deployment we have shown above. + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volume-populator/deployment/examples/backup-deployment.yaml +deployment.apps/kubestash-deployment created +``` + +Now, wait for the pod of the Deployment to go into the `Running` state. + +```bash +$ kubectl get pods -n demo +NAME READY STATUS RESTARTS AGE +kubestash-deployment-745d49c4cd-vsj8m 1/1 Running 0 62s +kubestash-deployment-745d49c4cd-wwj8w 1/1 Running 0 62s +``` + +Verify that the sample data has been created in `/source/data` and `/source/config` directory using the following command, + +```bash +$ kubectl exec -n demo kubestash-deployment-745d49c4cd-vsj8m -- cat /source/data/data.txt +sample_data +$ kubectl exec -n demo kubestash-deployment-745d49c4cd-vsj8m -- cat /source/config/config.cfg +config_data +``` + +### Prepare Backend +We are going to store our backed up data into a GCS bucket. We have to create a Secret with necessary credentials and a BackupStorage crd to use this backend. If you want to use a different backend, please read the respective backend configuration doc from [here](/docs/guides/backends/overview/index.md). + +**Create Storage Secret:** + +Let's create a Secret named `gcs-secret` with access credentials of our desired GCS backend, + +```bash +$ echo -n '' > GOOGLE_PROJECT_ID +$ cat /path/to/downloaded/sa_key_file.json > GOOGLE_SERVICE_ACCOUNT_JSON_KEY +$ kubectl create secret generic -n demo gcs-secret \ + --from-file=./GOOGLE_PROJECT_ID \ + --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY +secret/gcs-secret created +``` + +**Create BackupStorage:** + +Now, create a `BackupStorage` custom resource specifying the desired bucket, and directory inside the bucket where the backed up data will be stored. + +Below is the YAML of `BackupStorage` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut +``` + +Let's create the `BackupStorage` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volume-populator/deployment/examples/backupstorage.yaml +backupstorage.storage.kubestash.com/gcs-storage created +``` + +Now, we have to create a `RetentionPolicy` custom resource which specifies how the old `Snapshots` should be cleaned up. + +**Create RetentionPolicy:** + +Below is the YAML of the `RetentionPolicy` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same +``` +Notice the `spec.usagePolicy` that allows referencing the `RetentionPolicy` from all namespaces. To allow specific namespaces, we can configure it accordingly by following [RetentionPolicy usage policy](/docs/concepts/crds/retentionpolicy/index.md#retentionpolicy-spec). + +Let's create the `RetentionPolicy` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volume-populator/deployment/examples/retentionpolicy.yaml +retentionpolicy.storage.kubestash.com/demo-retention created +``` + +### Backup +We have to create a `BackupConfiguration` crd targeting the `kubestash-demo` Deployment that we have deployed earlier. Then, KubeStash will create a `CronJob` for each session to take periodic backup of `source-data`, `source-config` volumes of the target deployment. + +At first, we need to create a secret with a `Restic` password for backup data encryption. + +**Create Secret:** +Let's create a secret called `encry-secret` with the Restic password, + +```bash +$ echo -n 'changeit' > RESTIC_PASSWORD +$ kubectl create secret generic -n demo encryption-secret \ + --from-file=./RESTIC_PASSWORD \ +secret "encryption-secret" created +``` + +**Create BackupConfiguration :** +Below is the YAML of the `BackupConfiguration` that we are going to create, + +```yaml +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: sample-backup-dep + namespace: demo +spec: + target: + apiGroup: apps + kind: Deployment + name: kubestash-deployment + namespace: demo + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /dep + encryptionSecret: + name: encryption-secret # some addon may not support encryption + namespace: demo + deletionPolicy: WipeOut + addon: + name: workload-addon + tasks: + - name: logical-backup + params: + paths: /source/data,/source/config + exclude: /source/data/lost+found,/source/config/lost+found +``` + +Let's create the `BackupConfiguration` object we have shown above. + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volume-populator/deployment/examples/backupconfiguration.yaml +backupconfiguration.core.kubestash.com/sample-backup-dep created +``` + +**Verify Backup Setup Successful**: + +If everything goes well, the phase of the `BackupConfiguration` should be in `Ready` state. The `Ready` phase indicates that the backup setup is successful. +Let's check the `Phase` of the BackupConfiguration, + +```bash +$ kubectl get backupconfiguration -n demo +NAME PHASE PAUSED AGE +sample-backup-dep Ready 2m50s +``` +**Verify CronJob:** + +It will also create a `CronJob` with the schedule specified in `spec.sessions[*].scheduler.schedule` field of `BackupConfiguration` crd. + +Check that the `CronJob` has been created using the following command, + +```bash +$ kubectl get cronjob -n demo +NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE +trigger-sample-backup-dep-frequent-backup */5 * * * * False 0 20s +``` + +**Wait for BackupSession:** + +Now, wait for the next backup schedule. You can watch for `BackupSession` object using the following command, + +```bash +$ watch -n 1 kubectl get backupsession -n demo -l=kubestash.com/invoker-name=sample-backup-dep + +Every 1.0s: kubectl get backupsession -n demo -l=kubestash.com/invoker-name=sample-backup-dep anisur: Wed Jan 17 15:20:09 2024 + +NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE +sample-backup-dep-frequent-backup-1705483201 BackupConfiguration sample-backup-dep Running 9s +``` + +We can see from the above output that the backup session has succeeded. Now, we are going to verify whether the backed up data has been stored in the backend. + +**Verify Backup:** + +Once a backup is complete, KubeStash will update the respective `Repository` crd to reflect the backup. Check that the repository `gcs-repository` has been updated by the following command, + +```bash +$ kubectl get repository -n demo gcs-demo-repo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository true 1 806 B Ready 8m27s 9m18s +``` + +At this moment we have one `Snapshot`. Run the following command to check the respective `Snapshot` which represents the state of a backup run to a particular `Repository`. + +```bash +$ kubectl get snapshots -n demo -l=kubestash.com/repo-name=gcs-repository +NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE AGE +gcs-repository-sample-backup-dep-frequent-backup-1706015400 gcs-demo-repo demo-session 2024-01-23T13:10:54Z Delete Succeeded 16h +``` + +## Populate Volumes + +This section will guide you through the process of populating volumes of a deployment by restoring data previously backed up using KubeStash. + +Now, we need to create new Persistent Volume Claims (PVCs) with the `spec.dataSourceRef` set to reference our `snapshot` object. These PVCs will contain the restored data. + +Below is the YAML of the restored PVCs, + +```yaml +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: restored-source-data + namespace: demo + annotations: + populator.kubestash.com/app-name: kubestash-restored-deployment +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + dataSourceRef: + apiGroup: storage.kubestash.com + kind: Snapshot + name: gcs-repository-sample-backup-dep-frequent-backup-1706015400 +--- +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: restored-source-data + namespace: demo + annotations: + populator.kubestash.com/app-name: kubestash-restored-deployment +spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi + dataSourceRef: + apiGroup: storage.kubestash.com + kind: Snapshot + name: gcs-repository-sample-backup-dep-frequent-backup-1706015400 +``` +Here, +- `spec.dataSourceRef` specifies that which `snapshot` we want to restore into our populate volume. we are referencing a `snapshot` object that we have backed up in the previous section. +- `metadata.annotations.populator.kubestash.com/app-name` field is mandatory for any volume population of a deployment through KubeStash. + - This field denotes the deployment that will attached those volumes via mount paths. The volume population will only be successful if the mount path of this volume matches the mount paths of the backup deployment. + - For example, you backed up a deployment with volumes named `source-data` and `source-config`, each with corresponding mount paths `/source/data` and `/source/config`. Now you wish to populate volumes named `restore-source-data` and `restore-source-config` attached to a deployment named `restored-kubestash-deployment`, then this deployment must have mount paths set to `/source/data` and `/source/config`, respectively. + +Let's create the PVCs YAMl that we have shown above. + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volume-populator/deployment/examples/populated-pvcs.yaml +persistentvolumeclaim/restored-source-data created +persistentvolumeclaim/restored-source-config created +``` + +**Deploy Deployment :** + +Now, we are going to deploy a Deployment that uses the above PVCs. This Deployment has mount paths `/source/data` and `/source/config` corresponding to volumes named `restored-source-data` and `restored-source-config` respectively. + +Below is the YAML of the Deployment that we are going to create, + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + labels: + app: kubestash-demo + name: restored-kubestash-deployment + namespace: demo +spec: + replicas: 1 + selector: + matchLabels: + app: kubestash-demo + template: + metadata: + labels: + app: kubestash-demo + name: busybox + spec: + containers: + - command: ["/bin/sh", "-c"] + image: busybox + imagePullPolicy: IfNotPresent + name: busybox + volumeMounts: + - mountPath: /source/data + name: source-data + - mountPath: /source/config + name: source-config + restartPolicy: Always + volumes: + - name: restored-source-data + persistentVolumeClaim: + claimName: source-data + - name: restored-source-config + persistentVolumeClaim: + claimName: source-config +``` + +Let's create the deployment we have shown above. + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volume-populator/deployment/examples/restore-deployment.yaml +deployment.apps/restored-kubestash-deployment created +``` + +**Wait for Populate Volume:** + +Now, wait for the volume populate process to complete. You can watch the `PVCs` status using the following command, + +```bash +$ watch kubectl get pvc -n demo + +Every 2.0s: kubectl get pvc -n demo anisur: Tue Feb 13 18:37:26 2024 + +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE +restored-source-config Bound pvc-27a19e86-0ed5-4b18-9263-c8a66459aeb0 1Gi RWO standard-rwo 2m50s +restored-source-data Bound pvc-d210787e-5137-40b7-89e7-73d982716b40 1Gi RWO standard-rwo 2m51s + +``` + +From the output of the above command, we can see that `PVCs` status is `Bound` that means populate volumes has been completed successfully. + +**Verify Restored Data :** + +We are going to exec a pod of `restored-dep` deployment to verify whether the volume population with the backed up data has been restored successfully. + +Now, wait for the deployment pod to go into the `Running` state. + +```bash +$ kubectl get pods -n demo +NAME READY STATUS RESTARTS AGE +restored-kubestash-deployment-84b974dfb5-4jzjd 1/1 Running 0 2m48s + +``` + +Verify that the backed up data has been restored into `/source/data` and `/source/config` directory of above pod using the following command, + +```bash +$ kubectl exec -it -n demo restored-kubestash-deployment-84b974dfb5-4jzjd -- cat /source/data/data.txt +sample_data + +$ kubectl exec -it -n demo restored-kubestash-deployment-84b974dfb5-4jzjd -- cat /source/config/config.cfg +sample_config +``` + +## Cleaning Up + +To clean up the Kubernetes resources created by this tutorial, run: + +```bash +kubectl delete backupconfiguration -n demo sample-backup-dep +kubectl delete backupstorage -n demo gcs-storage +kubectl delete retentionPolicy -n demo demo-retention +kubectl delete secret -n demo gcs-secret +kubectl delete secret -n demo encryption-secret +kubectl delete deploy -n demo kubestash-deployment restored-kubestash-deployment +kubectl delete pvc -n demo --all +``` \ No newline at end of file diff --git a/docs/guides/volume-populator/overview/images/volume-populator-overview.svg b/docs/guides/volume-populator/overview/images/volume-populator-overview.svg new file mode 100644 index 0000000..3deddec --- /dev/null +++ b/docs/guides/volume-populator/overview/images/volume-populator-overview.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/docs/guides/volume-populator/overview/index.md b/docs/guides/volume-populator/overview/index.md new file mode 100644 index 0000000..3a8c7fb --- /dev/null +++ b/docs/guides/volume-populator/overview/index.md @@ -0,0 +1,39 @@ +--- +title: Volume Populator Overview | KubeStash +description: An overview of how Volume Populator works in KubeStash +menu: + docs_{{ .version }}: + identifier: volume-populator-overview + name: How Volume Populator works? + parent: volume-populator + weight: 10 +product_name: kubestash +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +# Populate Volume Using KubeStash + +This guide will give you an overview of how volume populator process works in KubeStash. + +## How Volume Populator Process Works? + +The following diagram shows how KubeStash restores backup data into a specific volume and attaching it to user-provided Persistent Volume Claims (PVCs) for usage of various workload applications. + +
+  KubeStash Volume Populator Process Flow +
Fig: Volume Populating Process in KubeStash
+
+ +The `Volume Populator` process consists of the following steps: + +1. At first, a user creates a `workload` application that may mount a single or multiple `PVCs`. +2. KubeStash operator watches for `PVCs`. +3. When operator finds a `PVC` with `spec.dataSourceRef` set and refers our `snapshot` object, it starts populating this `PVC` volume by creating a volume populator `Job` that mounts a temporary `PVC`. +4. Then the populator `Job` reads restore data information from this `snapshot` object and restore data into this `PVC`. +5. Once the restoration process is complete, the KubeStash operator deletes this populator `Job` and temporary `PVC`, and binds the `PV` containing the restored data with our desired `PVC`. + +## Next Steps +1. See a step by step guide to populate the volume of a stand-alone PVC [here](/docs/guides/volume-populator/pvc/index.md). +2. See a step by step guide to populate the volumes of a Deployment [here](/docs/guides/volume-populator/deployment/index.md). +3. See a step by step guide to populate the volumes of a StatefulSet [here](/docs/guides/volume-populator/statefulset/index.md). diff --git a/docs/guides/volume-populator/pvc/examples/backupconfiguration.yaml b/docs/guides/volume-populator/pvc/examples/backupconfiguration.yaml new file mode 100644 index 0000000..3fa34e6 --- /dev/null +++ b/docs/guides/volume-populator/pvc/examples/backupconfiguration.yaml @@ -0,0 +1,38 @@ +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: pvc-backup + namespace: demo +spec: + target: + apiGroup: + kind: PersistentVolumeClaim + name: target-pvc + namespace: demo + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /pvc-backup-demo + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: pvc-addon + tasks: + - name: logical-backup \ No newline at end of file diff --git a/docs/guides/volume-populator/pvc/examples/backupstorage.yaml b/docs/guides/volume-populator/pvc/examples/backupstorage.yaml new file mode 100644 index 0000000..e358e3f --- /dev/null +++ b/docs/guides/volume-populator/pvc/examples/backupstorage.yaml @@ -0,0 +1,17 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secret: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut diff --git a/docs/guides/volume-populator/pvc/examples/demo-pod.yaml b/docs/guides/volume-populator/pvc/examples/demo-pod.yaml new file mode 100644 index 0000000..59647b2 --- /dev/null +++ b/docs/guides/volume-populator/pvc/examples/demo-pod.yaml @@ -0,0 +1,17 @@ +kind: Pod +apiVersion: v1 +metadata: + name: demo-pod + namespace: demo +spec: + containers: + - name: busybox + image: busybox + command: ["/bin/sh", "-c","echo 'hello from pod.' > /sample/data/hello.txt && sleep 3000"] + volumeMounts: + - name: my-volume + mountPath: /sample/data + volumes: + - name: my-volume + persistentVolumeClaim: + claimName: target-pvc diff --git a/docs/guides/volume-populator/pvc/examples/populate-pv.yaml b/docs/guides/volume-populator/pvc/examples/populate-pv.yaml new file mode 100644 index 0000000..44e6bbb --- /dev/null +++ b/docs/guides/volume-populator/pvc/examples/populate-pv.yaml @@ -0,0 +1,15 @@ +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: populate-pv + namespace: demo +spec: + accessModes: + - ReadWriteMany + resources: + requests: + storage: 1Gi + dataSourceRef: + apiGroup: storage.kubestash.com + kind: Snapshot + name: gcs-repository-nfs-pvc-backup-frequent-backup-1704281100 \ No newline at end of file diff --git a/docs/guides/volume-populator/pvc/examples/retentionpolicy.yaml b/docs/guides/volume-populator/pvc/examples/retentionpolicy.yaml new file mode 100644 index 0000000..951aad3 --- /dev/null +++ b/docs/guides/volume-populator/pvc/examples/retentionpolicy.yaml @@ -0,0 +1,15 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same diff --git a/docs/guides/volume-populator/pvc/examples/target-pvc.yaml b/docs/guides/volume-populator/pvc/examples/target-pvc.yaml new file mode 100644 index 0000000..3cd9749 --- /dev/null +++ b/docs/guides/volume-populator/pvc/examples/target-pvc.yaml @@ -0,0 +1,11 @@ +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: target-pvc + namespace: demo +spec: + accessModes: + - ReadWriteMany + resources: + requests: + storage: 1Gi diff --git a/docs/guides/volume-populator/pvc/index.md b/docs/guides/volume-populator/pvc/index.md new file mode 100644 index 0000000..6262386 --- /dev/null +++ b/docs/guides/volume-populator/pvc/index.md @@ -0,0 +1,456 @@ +--- +title: Populate Volume of a Stand-alone PVC | KubeStash +description: An step by step guide to show how to populate volume of a stand-alone PVC +menu: + docs_{{ .version }}: + identifier: volume-populate-pvc + name: Populate Volume of a Stand-alone PVC + parent: volume-populator + weight: 20 +product_name: kubestash +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +# Populating a volume of a Standalone PVC + +This guide will show you how to use KubeStash to populate the volume of standalone PersistentVolumeClaims (PVCs). Within this guide, we will demonstrate the process of backing up a standalone PVC and subsequently restoring it using Kubernetes native approach, facilitated by KubeStash. + +## Before You Begin + +- At first, you need to have a Kubernetes cluster, and the `kubectl` command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). +- Install `KubeStash` in your cluster following the steps [here](/docs/setup/README.md). +- You should be familiar with the following `KubeStash` concepts: + - [BackupStorage](/docs/concepts/crds/backupstorage/index.md) + - [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md) + - [BackupSession](/docs/concepts/crds/backupsession/index.md) + - [SnapShot](/docs/concepts/crds/snapshot/index.md) + - [RestoreSession](/docs/concepts/crds/restoresession/index.md) + - [RetentionPolicy](/docs/concepts/crds/retentionpolicy/index.md) + +To keep everything isolated, we are going to use a separate namespace called `demo` throughout this tutorial. + +```bash +$ kubectl create ns demo +namespace/demo created +``` + +> **Note:** YAML files used in this tutorial are stored in [docs/guides/volumes/pvc/examples](/docs/guides/volumes/pvc/examples) directory of [kubestash/docs](https://github.com/kubestash/docs) repository. + +## Prepare Volume + +At first, let's prepare our desired PVC. Here, we are going to create a PVC. Then, we are going to mount this PVC with a Pod. Pod will generate a sample file into the PVC. + +**Create PersistentVolumeClaim:** + +Below, is the YAML of the PVC that we are going to create, + +```yaml +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: target-pvc + namespace: demo +spec: + accessModes: + - ReadWriteMany + resources: + requests: + storage: 1Gi +``` + +Notice the `spec.accessModes` section. We are using `ReadWriteMany` access mode so that multiple pods can use this PVC simultaneously. Without this access mode, KubeStash will fail to backup the volume if any other pod mount it during backup. + +Let's create the PVC we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volume-populator/pvc/examples/target-pvc.yaml + +persistentvolumeclaim/target-pvc created +``` + +**Deploy Workload:** + +Now, we are going to deploy a sample pod `demo-pod` that will mount `/sample/data` path of the `target-pvc`. This pod will generate a sample file named `hello.txt` with some demo data. We are going to backup the entire PVC using KubeStash that contains the sample files. + +Below, is the YAML of the pod that we are going to deploy, + +```yaml +kind: Pod +apiVersion: v1 +metadata: + name: demo-pod + namespace: demo +spec: + containers: + - name: busybox + image: busybox + command: ["/bin/sh", "-c","echo 'hello from pod.' > /sample/data/hello.txt && sleep 3000"] + volumeMounts: + - name: my-volume + mountPath: /sample/data + volumes: + - name: my-volume + persistentVolumeClaim: + claimName: target-pvc +``` + +Let's deploy the pod we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volume-populator/pvc/examples/demo-pod.yaml +pod/demo-pod created +``` + +Verify that the sample data has been generated into `/sample/data/` directory, + +```bash +$ kubectl exec -n demo demo-pod -- cat /sample/data/hello.txt +hello from pod. +``` + +## Backup + +Now, we are going to backup the PVC `target-pvc` in a GCS bucket using KubeStash. We have to create a Secret and a `BackupStorage` object with access credentials and backend information respectively. + +> For GCS backend, if the bucket does not exist, KubeStash needs `Storage Object Admin` role permissions to create the bucket. For more details, please check the following [guide](/docs/guides/backends/gcs/index.md). + +**Create Storage Secret:** + +Let's create a Secret named `gcs-secret` with access credentials of our desired GCS backend, + +```bash +$ echo -n '' > GOOGLE_PROJECT_ID +$ cat /path/to/downloaded/sa_key_file.json > GOOGLE_SERVICE_ACCOUNT_JSON_KEY +$ kubectl create secret generic -n demo gcs-secret \ + --from-file=./GOOGLE_PROJECT_ID \ + --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY +secret/gcs-secret created +``` + +**Create BackupStorage:** + +Now, create a `BackupStorage` custom resource specifying the desired bucket, and directory inside the bucket where the backed up data will be stored. + +Below is the YAML of `BackupStorage` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secret: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut +``` + +Let's create the `BackupStorage` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volume-populator/pvc/examples/backupstorage.yaml +backupstorage.storage.kubestash.com/gcs-storage created +``` + +Now, we also have to create another secret with an encryption key `RESTIC_PASSWORD` for Restic. This secret will be used by Restic for both encrypting and decrypting the backup data during backup & restore. + +**Create Restic Repository Secret:** + +Let's create a Secret named `encry-secret` with an encryption key `RESTIC_PASSWORD` for Restic. + +```bash +$ echo -n 'changeit' > RESTIC_PASSWORD +$ kubectl create secret generic -n demo encry-secret \ + --from-file=./RESTIC_PASSWORD +secret/encryption-secret created +``` + +Now, we have to create a `RetentionPolicy` custom resource which specifies how the old `Snapshots` should be cleaned up. + +**Create RetentionPolicy:** + +Below is the YAML of the `RetentionPolicy` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same +``` + +Let's create the `RetentionPolicy` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volume-populator/pvc/examples/retentionpolicy.yaml +retentionpolicy.storage.kubestash.com/demo-retention created +``` + +We are ready to start taking backup. Now, we have to create a `BackupConfiguration` custom resource targeting `nfs-pvc`. + +**Create BackupConfiguration:** + +Below is the YAML of the `BackupConfiguration` object that we are going to create, + +```yaml +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: pvc-backup + namespace: demo +spec: + target: + apiGroup: + kind: PersistentVolumeClaim + name: target-pvc + namespace: demo + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /pvc-backup-demo + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: pvc-addon + tasks: + - name: logical-backup +``` + +Here, +- `spec.target` specifies the backup target. `apiGroup`, `kind`, `namespace` and `name` refers to the `apiGroup`, `kind`, `namespace` and `name` of the targeted workload respectively. +- `spec.sessions[*].scheduler.schedule` specifies a [cron expression](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) indicates that `BackupSession` will be created at 5 minute interval. +- `spec.sessions[*].repositories[*].name` specifies the name of the `Repository` object that holds the backend information. +- `spec.sessions[*].repositories[*].backend` specifies the name of the backend that holds the `BackupStorage` information. +- `spec.sessions[*].repositories[*].directory` specifies the path of the `Repository` where the backed up data will be stored. +- `spec.sessions[*].repositories[*].encryptionSecret` specifies the encryption secret for `Restic Repository` which will be used to encrypting the backup data. +- `spec.sessions[*].addon.name` specifies the name of the `Addon` object that specifies addon configuration that will be used to perform backup of a stand-alone PVC. +- `spec.sessions[*].addon.tasks[*].name` specifies the name of the `Task` that holds the `Function` and their order of execution to perform backup of a stand-alone PVC. + +Let's create the `BackupConfiguration` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumes/pvc/examples/backupconfiguration.yaml +backupconfiguration.core.kubestash.com/pvc-backup created +``` + +**Verify Backup Setup Successful** + +If everything goes well, the phase of the `BackupConfiguration` should be in `Ready` state. The `Ready` phase indicates that the backup setup is successful. Let's check the `Phase` of the BackupConfiguration, + +```bash +$ kubectl get backupconfiguration -n demo +NAME PHASE PAUSED AGE +pvc-backup Ready 19s +``` + +**Verify Repository:** + +Verify that KubeStash has created `Repositories` that holds the `BackupStorage` information by the following command, + +```bash +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository Ready 28s +``` + +**Verify CronJob:** + +Verify that KubeStash has created a CronJob to trigger a periodic backup of the targeted PVC by the following command, + +```bash +$ kubectl get cronjob -n demo +NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE +trigger-pvc-backup-frequent-backup */5 * * * * False 0 5s 40s +``` + +**Wait for BackupSession:** + +Now, wait for the next backup schedule. You can watch for `BackupSession` crd using the following command, + +```bash +$ watch -n 1 kubectl get backupsession -n demo -l=kubestash.com/invoker-name=pvc-backup + +Every 1.0s: kubectl get backupsession -n demo -l=kubestash.com/invoker-name=pvc-backup workstation: Wed Jan 3 17:26:00 2024 + +NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE +pvc-backup-frequent-backup-1704281100 BackupConfiguration nfs-pvc-backup Succeeded 60s + +``` + +**Verify Backup:** + +When backup session is created, kubestash operator creates `Snapshot` which represents the state of backup run for each `Repository` which are provided in `BackupConfiguration`. + +Run the following command to check `Snapshot` phase, + +```bash +$ kubectl get snapshots -n demo +NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE VERIFICATION-STATUS AGE +gcs-repository-pvc-backup-frequent-backup-1704281100 gcs-repository frequent-backup 2024-01-03T11:25:13Z Delete Succeeded 2m14s +``` + +When backup session is completed, KubeStash will update the respective `Repository` to reflect the latest state of backed up data. + +Run the following command to check if a backup snapshot has been stored in the backend, + +```bash +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository true 1 2.262 KiB Ready 103s 8m + +``` + +From the output above, we can see that `1` snapshot has been stored in the backend specified by Repository `gcs-repository`. + +## Populate Volume + +This section will show you through the process of populating a volume by restoring the previously backed-up data using KubeStash. + +**Create a Stand-alone PVC:** + +Now, we are going to create a stand-alone `PVC` with `spec.dataSourceRef` set and refers our `snapshot` object. + +```yaml +kind: PersistentVolumeClaim +apiVersion: v1 +metadata: + name: populate-pv + namespace: demo +spec: + accessModes: + - ReadWriteMany + resources: + requests: + storage: 1Gi + dataSourceRef: + apiGroup: storage.kubestash.com + kind: Snapshot + name: gcs-repository-nfs-pvc-backup-frequent-backup-1704281100 +``` +Here, + - `spec.dataSourceRef` specifies that which `snapshot` we want to restore into our populate volume. + - we are referencing a `snapshot` object that we have backed up in the previous section. + +Let's create the `PVC` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumes/pvc/examples/restoresession.yaml +restoresession.core.kubestash.com/populate-pv created +``` + +**Wait for Populate Volume:** + +Now, wait for the volume populate process to complete. You can watch the `PVC` status using the following command, + +```bash +$ watch -n 1 kubectl get pvc -n demo populate-pv + +Every 1.0s: kubectl get pvc -n demo populate-pv workstation: Wed Jan 3 17:30:20 2024 + +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE +populate-pv Bound pvc-8d509c47-c26e-4039-8c03-49b25d126429 2Gi RWO standard 5d1 +``` +From the output of the above command, we can see that `PVC` status is `Bound` that means populate volume has been completed successfully. + +**Verify Restored Data :** + +We are going to create a new pod with mounting our previously created PVC `populate-pv` to verify whether the volume population with the backed up data has been restored successfully. + +Below, the YAML for the Pod we are going to create. + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: restored-pod + namespace: demo +spec: + containers: + - name: busybox + image: busybox + args: + - sleep + - "3600" + volumeMounts: + - name: restore-data + mountPath: /restore/data + volumes: + - name: restore-data + persistentVolumeClaim: + claimName: populate-pv + readOnly: false +``` +Let's create the Pod we have shown above. + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/pvc/examples/restored-pod.yaml +pod/restored-pod created +``` + +Now, wait for the Pod to go into the `Running` state. + +```bash +$ kubectl get pod -n demo +NAME READY STATUS RESTARTS AGE +restored-pod 1/1 Running 0 25s +``` + +Verify that the backed up data has been restored in `/restore/data` directory for `restored-pod` pod using the following command, + +```bash +$ kubectl exec -n demo restored-pod -- cat /restore/data/hello.txt +hello from pod. +``` + + +## Cleaning Up + +To clean up the Kubernetes resources created by this tutorial, run: + +```bash +kubectl delete backupconfiguration -n demo pvc-backup +kubectl delete backupstorage -n demo gcs-storage +kubectl delete retentionPolicy -n demo demo-retention +kubectl delete secret -n demo gcs-secret +kubectl delete secret -n demo encry-secret +kubectl delete pod -n demo demo-pod +kubectl delete -n demo pod restored-pod +kubectl delete pvc -n demo target-pvc +kubectl delete pvc -n demo populate-pv +``` diff --git a/docs/guides/volume-populator/statefulset/examples/backup-statefulset.yaml b/docs/guides/volume-populator/statefulset/examples/backup-statefulset.yaml new file mode 100644 index 0000000..5548270 --- /dev/null +++ b/docs/guides/volume-populator/statefulset/examples/backup-statefulset.yaml @@ -0,0 +1,53 @@ +apiVersion: v1 +kind: Service +metadata: + name: busybox + namespace: demo +spec: + ports: + - name: http + port: 80 + targetPort: 0 + selector: + app: demo-busybox + clusterIP: None +--- +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: sample-sts + namespace: demo + labels: + app: demo-busybox +spec: + replicas: 3 + selector: + matchLabels: + app: demo-busybox + serviceName: busybox + template: + metadata: + labels: + app: demo-busybox + spec: + containers: + - name: busybox + image: busybox + command: ["/bin/sh", "-c","echo $(POD_NAME) > /source/data/data.txt && sleep 3000"] + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + volumeMounts: + - name: source-data + mountPath: "/source/data" + imagePullPolicy: IfNotPresent + volumeClaimTemplates: + - metadata: + name: source-data + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 256Mi \ No newline at end of file diff --git a/docs/guides/volume-populator/statefulset/examples/backupconfiguration.yaml b/docs/guides/volume-populator/statefulset/examples/backupconfiguration.yaml new file mode 100644 index 0000000..afaf6e7 --- /dev/null +++ b/docs/guides/volume-populator/statefulset/examples/backupconfiguration.yaml @@ -0,0 +1,42 @@ +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: sample-backup-sts + namespace: demo +spec: + target: + apiGroup: apps + kind: StatefulSet + name: sample-sts + namespace: demo + backends: + - name: gcs-backend + storageRef: + name: gcs-storage + namespace: demo + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: demo-session + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-demo-repo + backend: gcs-backend + directory: /demo/sample-sts + encryptionSecret: + name: encryption-secret + namespace: demo + addon: + name: workload-addon + tasks: + - name: logical-backup + params: + paths: /source/data + exclude: /source/data/lost+found + retryConfig: + maxRetry: 2 + delay: 1m \ No newline at end of file diff --git a/docs/guides/volume-populator/statefulset/examples/backupstorage.yaml b/docs/guides/volume-populator/statefulset/examples/backupstorage.yaml new file mode 100644 index 0000000..5d53db5 --- /dev/null +++ b/docs/guides/volume-populator/statefulset/examples/backupstorage.yaml @@ -0,0 +1,17 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut diff --git a/docs/guides/volume-populator/statefulset/examples/restore-statefulset.yaml b/docs/guides/volume-populator/statefulset/examples/restore-statefulset.yaml new file mode 100644 index 0000000..54c1087 --- /dev/null +++ b/docs/guides/volume-populator/statefulset/examples/restore-statefulset.yaml @@ -0,0 +1,40 @@ +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: sample-restored-sts + namespace: demo + labels: + app: demo-busybox +spec: + replicas: 3 + selector: + matchLabels: + app: demo-busybox + serviceName: busybox + template: + metadata: + labels: + app: demo-busybox + spec: + containers: + - name: busybox + image: busybox + command: ["/bin/sh", "-c","sleep 3000"] + volumeMounts: + - name: restored-source-data + mountPath: "/source/data" + imagePullPolicy: IfNotPresent + volumeClaimTemplates: + - metadata: + name: restored-source-data + annotations: + populator.kubestash.com/app-name: sample-restored-sts + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 256Mi + dataSourceRef: + apiGroup: storage.kubestash.com + kind: Snapshot + name: gcs-demo-repo-sample-backup-sts-demo-session-1707900900 \ No newline at end of file diff --git a/docs/guides/volume-populator/statefulset/examples/retentionpolicy.yaml b/docs/guides/volume-populator/statefulset/examples/retentionpolicy.yaml new file mode 100644 index 0000000..3af9037 --- /dev/null +++ b/docs/guides/volume-populator/statefulset/examples/retentionpolicy.yaml @@ -0,0 +1,15 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same \ No newline at end of file diff --git a/docs/guides/volume-populator/statefulset/index.md b/docs/guides/volume-populator/statefulset/index.md new file mode 100644 index 0000000..ef9fdc8 --- /dev/null +++ b/docs/guides/volume-populator/statefulset/index.md @@ -0,0 +1,452 @@ +--- +title: Populate Volumes of a StatefulSets | KubeStash +description: An step by step guide to show how to populate volumes of a StatefulSet +menu: + docs_{{ .version }}: + identifier: volume-populate-statefulset + name: Populate Volumes of a StatefulSet + parent: volume-populator + weight: 30 +product_name: kubestash +menu_name: docs_{{ .version }} +section_menu_id: guides +--- + +# Populating volumes of a StatefulSet + +This guide will show you how to use KubeStash to populate the volumes of a `StatefulSet`. Within this guide, we will demonstrate the process of backing up volumes of a `StatefulSet` and subsequently restoring it using Kubernetes native approach, facilitated by KubeStash. + +## Before You Begin +- At first, you need to have a Kubernetes cluster, and the `kubectl` command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). +- Install `KubeStash` in your cluster following the steps [here](/docs/setup/README.md). +- You should be familiar with the following `KubeStash` concepts: + - [BackupStorage](/docs/concepts/crds/backupstorage/index.md) + - [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md) + - [BackupSession](/docs/concepts/crds/backupsession/index.md) + - [SnapShot](/docs/concepts/crds/snapshot/index.md) + - [RestoreSession](/docs/concepts/crds/restoresession/index.md) + - [RetentionPolicy](/docs/concepts/crds/retentionpolicy/index.md) + +To keep everything isolated, we are going to use a separate namespace called `demo` throughout this tutorial. + +```bash +$ kubectl create ns demo +namespace/demo created +``` + +> **Note:** YAML files used in this tutorial are stored in [docs/guides/volume-populator/statefulset/examples](/docs/guides/volume-populator/statefulset/examples) directory of [kubestash/docs](https://github.com/kubestash/docs) repository. + +## Prepare Workload + +At first, We are going to deploy a StatefulSet with a PVC `source-data`. This StatefulSet will automatically generate sample data in `/source/data` directory. + +Below is the YAML of the StatefulSet that we are going to create, +```yaml +apiVersion: v1 +kind: Service +metadata: + name: busybox + namespace: demo +spec: + ports: + - name: http + port: 80 + targetPort: 0 + selector: + app: demo-busybox + clusterIP: None +--- +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: sample-sts + namespace: demo + labels: + app: demo-busybox +spec: + replicas: 3 + selector: + matchLabels: + app: demo-busybox + serviceName: busybox + template: + metadata: + labels: + app: demo-busybox + spec: + containers: + - name: busybox + image: busybox + command: ["/bin/sh", "-c","echo $(POD_NAME) > /source/data/data.txt && sleep 3000"] + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + volumeMounts: + - name: source-data + mountPath: "/source/data" + imagePullPolicy: IfNotPresent + volumeClaimTemplates: + - metadata: + name: source-data + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 256Mi +``` +Let's create the StatefulSet we have shown above. + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/workloads/statefulset/examples/statefulset.yaml +service/busybox created +statefulset.apps/sample-sts created +``` + +Now, wait for the pods of the StatefulSet to go into the `Running` state. + +```bash +$ kubectl get pod -n demo +NAME READY STATUS RESTARTS AGE +sample-sts-0 1/1 Running 0 42s +sample-sts-1 1/1 Running 0 40s +sample-sts-2 1/1 Running 0 36s +``` +Verify that the sample data has been generated in `/source/data` directory for `sample-sts-0` , `sample-sts-1` and `sample-sts-2` pod respectively using the following commands, + +```bash +$ kubectl exec -n demo sample-sts-0 -- cat /source/data/data.txt +sample-sts-0 +$ kubectl exec -n demo sample-sts-1 -- cat /source/data/data.txt +sample-sts-1 +$ kubectl exec -n demo sample-sts-2 -- cat /source/data/data.txt +sample-sts-2 +``` + +### Prepare Backend +We are going to store our backed up data into a GCS bucket. We have to create a Secret with necessary credentials and a BackupStorage crd to use this backend. If you want to use a different backend, please read the respective backend configuration doc from [here](/docs/guides/backends/overview/index.md). + +**Create Storage Secret:** + +Let's create a Secret named `gcs-secret` with access credentials of our desired GCS backend, + +```bash +$ echo -n '' > GOOGLE_PROJECT_ID +$ cat /path/to/downloaded/sa_key_file.json > GOOGLE_SERVICE_ACCOUNT_JSON_KEY +$ kubectl create secret generic -n demo gcs-secret \ + --from-file=./GOOGLE_PROJECT_ID \ + --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY +secret/gcs-secret created +``` + +**Create BackupStorage:** + +Now, create a `BackupStorage` custom resource specifying the desired bucket, and directory inside the bucket where the backed up data will be stored. + +Below is the YAML of `BackupStorage` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut +``` + +Let's create the `BackupStorage` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volume-populator/statefulset/examples/backupstorage.yaml +backupstorage.storage.kubestash.com/gcs-storage created +``` + +Now, we have to create a `RetentionPolicy` custom resource which specifies how the old `Snapshots` should be cleaned up. + +**Create RetentionPolicy:** + +Below is the YAML of the `RetentionPolicy` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same +``` +Notice the `spec.usagePolicy` that allows referencing the `RetentionPolicy` from all namespaces. To allow specific namespaces, we can configure it accordingly by following [RetentionPolicy usage policy](/docs/concepts/crds/retentionpolicy/index.md#retentionpolicy-spec). + +Let's create the `RetentionPolicy` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volume-populator/statefulset/examples/retentionpolicy.yaml +retentionpolicy.storage.kubestash.com/demo-retention created +``` + +### Backup + +We have to create a `BackupConfiguration` crd targeting the `sample-sts` StatefulSet that we have deployed earlier. KubeStash will create a `CronJob` for each session to take periodic backup of `/source/data` directory of the target. + +At first, we need to create a secret with a Restic password for backup data encryption. + +**Create Secret:** + +Let's create a secret called `encry-secret` with the Restic password, + +```bash +$ echo -n 'changeit' > RESTIC_PASSWORD +$ kubectl create secret generic -n demo encryption-secret \ + --from-file=./RESTIC_PASSWORD \ +secret "encryption-secret" created +``` + +**Create BackupConfiguration:** + +Below is the YAML of the `BackupConfiguration` object that we are going to create, + +```yaml +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration +metadata: + name: sample-backup-sts + namespace: demo +spec: + target: + apiGroup: apps + kind: StatefulSet + name: sample-sts + namespace: demo + backends: + - name: gcs-backend + storageRef: + name: gcs-storage + namespace: demo + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: demo-session + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-demo-repo + backend: gcs-backend + directory: /demo/sample-sts + encryptionSecret: + name: encryption-secret + namespace: demo + addon: + name: workload-addon + tasks: + - name: logical-backup + params: + paths: /source/data + exclude: /source/data/lost+found + retryConfig: + maxRetry: 2 + delay: 1m +``` + +Let's create the `BackupConfiguration` crd we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volume-populator/statefulset/examples/backupconfiguration.yaml +backupconfiguration.core.kubestash.com/sample-backup-sts created +``` + +**Verify Backup Setup Successful** + +If everything goes well, the phase of the `BackupConfiguration` should be `Ready`. The `Ready` phase indicates that the backup setup is successful. Let's verify the `Phase` of the BackupConfiguration, + +```bash +$ kubectl get backupconfiguration -n demo +NAME PHASE PAUSED AGE +sample-backup-sts Ready 2m50s +``` + +**Verify CronJob:** + +It will also create a `CronJob` with the schedule specified in `spec.sessions[*].scheduler.schedule` field of `BackupConfiguration` crd. + +Verify that the `CronJob` has been created using the following command, + +```bash +$ kubectl get cronjob -n demo +NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE +trigger-sample-backup-sts-demo-session */5 * * * * 0 2m45s 3m25s +``` + +**Wait for BackupSession:** + +Wait for the next schedule for backup. Run the following command to watch `BackupSession` crd, + +```bash +$ kubectl get backupsession -n demo -w +NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE +sample-backup-sts-demo-session-1706015400 BackupConfiguration sample-backup-sts Succeeded 7m22s +``` + +We can see from the above output that the backup session has succeeded. Now, we are going to verify whether the backed up data has been stored in the backend. + +**Verify Backup:** + +Once a backup is complete, KubeStash will update the respective `Repository` crd to reflect the backup. Check that the repository `gcs-repository` has been updated by the following command, + +```bash +$ kubectl get repository -n demo gcs-demo-repo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository true 1 806 B Ready 8m27s 9m18s +``` + +At this moment we have one `Snapshot`. Run the following command to check the respective `Snapshot` which represents the state of a backup run to a particular `Repository`. + +```bash +$ kubectl get snapshots -n demo -l=kubestash.com/repo-name=gcs-repository +NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE AGE +gcs-repository-sample-backup-sts-frequent-backup-1706015400 gcs-demo-repo demo-session 2024-01-23T13:10:54Z Delete Succeeded 16h +``` + +## Populate Volumes + +This section will guide you through the process of populating volumes of a `StatefulSet` by restoring data previously backed up using KubeStash. + +Now, we need to create a new StatefulSet along with a PersistentVolumeClaim (PVC). This PVC configure with `spec.dataSourceRef` pointing to our `snapshot` object. Kubestash will populate volume with the restored data from dataSourceRef pointing snapshot and attach it to this PVC. As a result, the PVC will contain the data that has been restored. + +Below is the YAML of the restored StatefulSet, + +```yaml +apiVersion: apps/v1 +kind: StatefulSet +metadata: + name: sample-restored-sts + namespace: demo + labels: + app: demo-busybox +spec: + replicas: 3 + selector: + matchLabels: + app: demo-busybox + serviceName: busybox + template: + metadata: + labels: + app: demo-busybox + spec: + containers: + - name: busybox + image: busybox + command: ["/bin/sh", "-c","sleep 3000"] + volumeMounts: + - name: restored-source-data + mountPath: "/source/data" + imagePullPolicy: IfNotPresent + volumeClaimTemplates: + - metadata: + name: restored-source-data + annotations: + populator.kubestash.com/app-name: sample-restored-sts + spec: + accessModes: [ "ReadWriteOnce" ] + resources: + requests: + storage: 256Mi + dataSourceRef: + apiGroup: storage.kubestash.com + kind: Snapshot + name: gcs-demo-repo-sample-backup-sts-demo-session-1707900900 +``` +Here, +- `spec.dataSourceRef` specifies that which `snapshot` we want to restore into our populate volume. we are referencing a `snapshot` object that we have backed up in the previous section. +- `metadata.annotations.populator.kubestash.com/app-name` field is mandatory for any volume population of a statefulset through KubeStash. + - This field denotes the statefulset that will attached those volumes via mount paths. The volume population will only be successful if the mount path of this volume matches the mount paths of the backup statefulset. + +Let's create the statefulset we have shown above. + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volume-populator/statefulset/examples/restore-statefulset.yaml +statefulset.apps/sample-restored-sts created +``` + +**Wait for Populate Volume:** + +Now, wait for the volume populate process to complete. You can watch the `PVCs` status using the following command, + +```bash +$ watch kubectl get pvc -n demo + +Every 2.0s: kubectl get pvc -n demo anisur: Tue Feb 13 18:37:26 2024 + +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE +restored-source-data-sample-restored-sts-0 Bound pvc-1fb1df97-6c36-4dd3-9b30-be5307abba60 1Gi RWO standard-rwo 2m46s +restored-source-data-sample-restored-sts-1 Bound pvc-4726726d-7f9d-4c67-8aa2-57615b88a989 1Gi RWO standard-rwo 2m25s +restored-source-data-sample-restored-sts-2 Bound pvc-df60d50f-9697-4d2f-a6e2-9e9c7ea89524 1Gi RWO standard-rwo 2m3s +``` + +From the output of the above command, we can see that `PVCs` status is `Bound` that means populate volumes has been completed successfully. + +**Verify Restored Data :** + +We are going to exec any pod of `sample-restored-sts` statefulset to verify whether the volume population with the backed up data has been restored successfully. + +Now, wait for the statefulset pod to go into the `Running` state. + +```bash +$ kubectl get pods -n demo +NAME READY STATUS RESTARTS AGE +sample-restored-sts-0 1/1 Running 0 95s +sample-restored-sts-1 1/1 Running 0 65s +sample-restored-sts-2 1/1 Running 0 41s +``` + +Verify that the backed up data has been restored into `/source/data` directory of above pod using the following command, + +```bash +$ kubectl exec -it -n demo sample-restored-sts-0 -- cat /source/data/data.txt +sample-sts-0 + +$ kubectl exec -it -n demo sample-restored-sts-1 -- cat /source/data/data.txt +sample-sts-1 + +$ kubectl exec -it -n demo sample-restored-sts-2 -- cat /source/data/data.txt +sample-sts-2 +``` + +## Cleaning Up + +To clean up the Kubernetes resources created by this tutorial, run: + +```bash +kubectl delete backupconfiguration -n demo sample-backup-sts +kubectl delete backupstorage -n demo gcs-storage +kubectl delete retentionPolicy -n demo demo-retention +kubectl delete secret -n demo gcs-secret +kubectl delete secret -n demo encryption-secret +kubectl delete statefulset -n demo sample-sts +kubectl delete statefulset -n demo sample-restored-sts +kubectl delete pvc -n demo --all +``` \ No newline at end of file diff --git a/docs/guides/volumes/_index.md b/docs/guides/volumes/_index.md index d5510a3..160f903 100644 --- a/docs/guides/volumes/_index.md +++ b/docs/guides/volumes/_index.md @@ -1,5 +1,5 @@ --- -title: Volume Backup | Stash +title: Volume Backup | KubeStash menu: docs_{{ .version }}: identifier: volume-backup diff --git a/docs/guides/volumes/overview/images/volume_backup_overview.svg b/docs/guides/volumes/overview/images/volume_backup_overview.svg index 8b13582..e6f3fbd 100644 --- a/docs/guides/volumes/overview/images/volume_backup_overview.svg +++ b/docs/guides/volumes/overview/images/volume_backup_overview.svg @@ -1,942 +1 @@ - - - - - - image/svg+xml - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + \ No newline at end of file diff --git a/docs/guides/volumes/overview/images/volume_restore_overview.svg b/docs/guides/volumes/overview/images/volume_restore_overview.svg index 65ce526..14dac0f 100644 --- a/docs/guides/volumes/overview/images/volume_restore_overview.svg +++ b/docs/guides/volumes/overview/images/volume_restore_overview.svg @@ -1,822 +1 @@ - - - - - - image/svg+xml - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + \ No newline at end of file diff --git a/docs/guides/volumes/overview/index.md b/docs/guides/volumes/overview/index.md index 3f4615a..3570b9e 100644 --- a/docs/guides/volumes/overview/index.md +++ b/docs/guides/volumes/overview/index.md @@ -1,6 +1,6 @@ --- -title: Stand-alone Volume Backup Overview | Stash -description: An overview on how stand-alone volume backup works in Stash. +title: Stand-alone Volume Backup Overview | KubeStash +description: An overview on how stand-alone volume backup works in KubeStash. menu: docs_{{ .version }}: identifier: volume-backup-overview @@ -14,11 +14,11 @@ section_menu_id: guides # Stand-alone Volume Backup Overview -If you are using a volume that can be mounted in multiple workloads, aka `ReadWriteMany`/`RWX`, you might want to backup the volume independent of the workloads. Stash supports backup of stand-alone volumes. This guide will give you an overview of how stand-alone volume backup and restore process works in Stash. +If you are using a volume that can be mounted in multiple workloads, aka `ReadWriteMany`/`RWX`, you might want to backup the volume independent of the workloads. KubeStash supports backup of stand-alone volumes. This guide will give you an overview of how stand-alone volume backup and restore process works in KubeStash. ## How Backup Works -The following diagram shows how Stash takes backup of a stand-alone volume. Open the image in a new tab to see the enlarged version. +The following diagram shows how KubeStash takes backup of a stand-alone volume. Open the image in a new tab to see the enlarged version.
  Stand-alone Volume Backup Overview @@ -27,27 +27,21 @@ The following diagram shows how Stash takes backup of a stand-alone volume. Open The backup process consists of the following steps: -1. At first, a user creates a secret with access credentials of the backend where the backed up data will be stored. - -2. Then, she creates a `Repository` crd that specifies the backend information along with the secret that holds the credentials to access the backend. - -3. Then, she creates a `BackupConfiguration` crd targeting the volume. The `BackupConfiguration` object also specifies the `Task` to use to backup the volume. - -4. Stash operator watches for `BackupConfiguration` crd. - -5. Once Stash operator finds a `BackupConfiguration` crd, it creates a CronJob with the schedule specified in `BackupConfiguration` object to trigger backup periodically. - -6. On next scheduled slot, the CronJob triggers a backup by creating a `BackupSession` crd. - -7. Stash operator also watches for `BackupSession` crd. - -8. When it finds a `BackupSession` object, it resolves the respective `Task` and `Function` and prepares a backup Job definition. - -9. Then, it mounts the targeted volume into the Job and creates it. - -10. The Job takes backup of the targeted volume. - -11. Finally, when backup is completed, the Job sends Prometheus metrics to the Pushgateway running inside Stash operator pod. It also updates the `BackupSession` and `Repository` status to reflect the backup procedure. +1. At first, a user creates a `Secret` with access credentials of the backend where the backed-up data will be stored. +2. Then, she creates a `BackupStorage` custom resource that specifies the backend information, along with the `Secret` containing the credentials needed to access the backend. +3. KubeStash operator watches for `BackupStorage` custom resources. When it finds a `BackupStorage` object, it initializes the `BackupStorage` by uploading the `metadata.yaml` file into the target storage. +4. Next, she creates a `BackupConfiguration` custom resource, targeting a standalone volume. The `BackupConfiguration` object specifies the `Repository` pointing to a `BackupStorage` that contains backend information, indicating where to upload backup data. It also defines the `Addon` information with a specified `Task` to be used for backing up the volume. +5. KubeStash operator watches for `BackupConfiguration` custom resources. +6. Once the KubeStash operator finds a `BackupConfiguration` object, it creates `Repository` with the information specified in the BackupConfiguration. +7. KubeStash operator watches for `Repository` custom resources. When it finds the `Repository` object, it Initializes `Repository` by uploading `repository.yaml` file into the `spec.sessions[*].repositories[*].directory` path specified in `BackupConfiguration`. +8. Then, it creates a `CronJob` with the schedule specified in `BackupConfiguration` to trigger backup periodically. +9. On the next scheduled slot, the `CronJob` triggers a backup by creating a `BackupSession` custom resource. +10. KubeStash operator watches for `BackupSession` custom resources. +11. When it finds a `BackupSession` object, it creates a `Snapshot` custom resource for each `Repository` specified in the `BackupConfiguration`. +12. Then it resolves the respective `Addon` and `Function` and prepares a backup `Job` definition. +13. Then, it mounts the targeted volume into the `Job` and creates it. +14. The Job takes backup of the targeted volume. +15. After the backup process is completed, the backup `Job` updates the `status.components[*]` field of the `Snapshot` resources with each target volume information. It also updates the `status.phase` field of the `BackupSession` to reflect backup completion. ## How Restore Works @@ -59,24 +53,12 @@ The following diagram shows how Stash restores backed up data into a stand-alone
The restore process consists of the following steps: - -1. At first, a user creates a `RestoreSession` crd which specifies the targeted volume where the backed up data will be restored and the `Repository` crd which holds the backend information where the backed up data has been stored. It also specifies the `Task` to use to restore the target. - -2. Stash operator watches for `RestoreSession` object. - -3. Once it finds a `RestoreSession` object, it resolves the respective `Task` and `Function` and prepares a restore Job definition. - -4. Then, it mounts the targeted volume into the Job and creates it. - -5. The Job restores the backed up data into the volume. - -6. Finally, when the restore process is completed, the Job sends Prometheus metrics to the Pushgateway and update the `RestoreSession` status to reflect restore completion. - -### Why use Function-Task model to backup or restore a volume - -You might be wondering why we have used [Function](/docs/concepts/crds/function/index.md) and [Task](/docs/concepts/crds/task/index.md) to backup or restore a volume. Well, let us explain. - -`Function-Task` model gives you the flexibility to customize the backup/restore process. For example, it enables you to execute some logic to prepare your apps before backup or execute logic to delete corrupted data before restore. For more details about what are the others benefits of `Function-Task` model, please visit [here](/docs/concepts/crds/task/index.md##why-function-and-task). +1. At first, a user creates a `RestoreSession` custom resource, specifying the target volume where the backed-up data will be restored, the `Repository` object that points to a `BackupStorage` that holds backend information, and the target `Snapshot`, which will be restored. It also specifies the `Addon` info with `Task` to use to restore the volume. +2. KubeStash operator watches for `RestoreSession` custom resources. +3. Once it finds a `RestoreSession` object, it resolves the respective `Addon` and `Function` and prepares a restore Job definition. +4. Then, it mounts the targeted volume and creates a Restore `Job` +5. The restore `Job` restores the backed-up data into the targeted volume. +6. Finally, when the restore process is completed, the job updates the `status.phase` field of the `Restoresession` to reflect restore completion. ## Next Steps diff --git a/docs/guides/volumes/pvc/examples/backupconfiguration.yaml b/docs/guides/volumes/pvc/examples/backupconfiguration.yaml index 9e26b09..1f3a78d 100644 --- a/docs/guides/volumes/pvc/examples/backupconfiguration.yaml +++ b/docs/guides/volumes/pvc/examples/backupconfiguration.yaml @@ -1,20 +1,38 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: name: nfs-pvc-backup namespace: demo spec: - task: - name: pvc-backup - repository: - name: gcs-repo - schedule: "*/5 * * * *" target: - ref: - apiVersion: v1 - kind: PersistentVolumeClaim - name: nfs-pvc - retentionPolicy: - name: keep-last-5 - keepLast: 5 - prune: true + apiGroup: + kind: PersistentVolumeClaim + name: nfs-pvc + namespace: demo + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /pvc-backup-demo + encryptionSecret: + name: encryption-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: pvc-addon + tasks: + - name: logical-backup \ No newline at end of file diff --git a/docs/guides/volumes/pvc/examples/backupstorage.yaml b/docs/guides/volumes/pvc/examples/backupstorage.yaml new file mode 100644 index 0000000..0461b26 --- /dev/null +++ b/docs/guides/volumes/pvc/examples/backupstorage.yaml @@ -0,0 +1,17 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/volumes/pvc/examples/pv.yaml b/docs/guides/volumes/pvc/examples/pv.yaml index 582bb70..e066fc0 100644 --- a/docs/guides/volumes/pvc/examples/pv.yaml +++ b/docs/guides/volumes/pvc/examples/pv.yaml @@ -2,13 +2,14 @@ apiVersion: v1 kind: PersistentVolume metadata: name: nfs-pv - labels: - app: nfs-demo spec: capacity: storage: 1Gi accessModes: - - ReadWriteMany - nfs: - server: "nfs-service.storage.svc.cluster.local" - path: "/" + - ReadWriteMany + csi: + driver: nfs.csi.k8s.io + volumeHandle: nfs-server.default.svc.cluster.local/share## + volumeAttributes: + server: nfs-server.default.svc.cluster.local + share: / \ No newline at end of file diff --git a/docs/guides/volumes/pvc/examples/pvc.yaml b/docs/guides/volumes/pvc/examples/pvc.yaml index 5203a80..9a41c54 100644 --- a/docs/guides/volumes/pvc/examples/pvc.yaml +++ b/docs/guides/volumes/pvc/examples/pvc.yaml @@ -1,5 +1,5 @@ -apiVersion: v1 kind: PersistentVolumeClaim +apiVersion: v1 metadata: name: nfs-pvc namespace: demo @@ -10,6 +10,4 @@ spec: resources: requests: storage: 1Gi - selector: - matchLabels: - app: nfs-demo + volumeName: nfs-pv \ No newline at end of file diff --git a/docs/guides/volumes/pvc/examples/repository.yaml b/docs/guides/volumes/pvc/examples/repository.yaml deleted file mode 100644 index 3c4c337..0000000 --- a/docs/guides/volumes/pvc/examples/repository.yaml +++ /dev/null @@ -1,11 +0,0 @@ -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository -metadata: - name: gcs-repo - namespace: demo -spec: - backend: - gcs: - bucket: appscode-qa - prefix: stash-backup/volumes/nfs-pvc - storageSecretName: gcs-secret diff --git a/docs/guides/volumes/pvc/examples/restoresession.yaml b/docs/guides/volumes/pvc/examples/restoresession.yaml index e7bd42e..bc04362 100644 --- a/docs/guides/volumes/pvc/examples/restoresession.yaml +++ b/docs/guides/volumes/pvc/examples/restoresession.yaml @@ -1,17 +1,21 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: RestoreSession metadata: name: nfs-pvc-restore namespace: demo spec: - task: - name: pvc-restore - repository: - name: gcs-repo target: - ref: - apiVersion: v1 - kind: PersistentVolumeClaim - name: nfs-pvc - rules: - - snapshots: ["latest"] + apiGroup: + kind: PersistentVolumeClaim + name: nfs-pvc + namespace: demo + dataSource: + repository: gcs-repository + snapshot: latest + encryptionSecret: + name: encryption-secret + namespace: demo + addon: + name: pvc-addon + tasks: + - name: logical-backup-restore \ No newline at end of file diff --git a/docs/guides/volumes/pvc/examples/retentionpolicy.yaml b/docs/guides/volumes/pvc/examples/retentionpolicy.yaml new file mode 100644 index 0000000..3af9037 --- /dev/null +++ b/docs/guides/volumes/pvc/examples/retentionpolicy.yaml @@ -0,0 +1,15 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same \ No newline at end of file diff --git a/docs/guides/volumes/pvc/images/pvc_repo.png b/docs/guides/volumes/pvc/images/pvc_repo.png index 908b51f..d1689b0 100644 Binary files a/docs/guides/volumes/pvc/images/pvc_repo.png and b/docs/guides/volumes/pvc/images/pvc_repo.png differ diff --git a/docs/guides/volumes/pvc/index.md b/docs/guides/volumes/pvc/index.md index c343b87..8153d43 100644 --- a/docs/guides/volumes/pvc/index.md +++ b/docs/guides/volumes/pvc/index.md @@ -1,6 +1,6 @@ --- -title: Backup Stand-alone PVC | Stash -description: A step by step guide on how to backup a stand-alone PVC using Stash. +title: Backup Stand-alone PVC | KubeStash +description: A step by step guide on how to backup a stand-alone PVC using KubeStash. menu: docs_{{ .version }}: identifier: volume-backup-pvc @@ -12,25 +12,24 @@ menu_name: docs_{{ .version }} section_menu_id: guides --- -# Backup Stand-alone PVC using Stash +# Backup Stand-alone PVC using KubeStash -This guide will show you how to backup a stand-alone PersistentVolumeClaim (PVC) using Stash. Here, we are going to backup a PVC provisioned using an NFS server into a GCS bucket. +This guide will show you how to backup a stand-alone PersistentVolumeClaim (PVC) using KubeStash. Here, we are going to backup a PVC provisioned using an NFS server into a GCS bucket. ## Before You Begin - At first, you need to have a Kubernetes cluster, and the `kubectl` command-line tool must be configured to communicate with your cluster. If you do not already have a cluster, you can create one by using [kind](https://kind.sigs.k8s.io/docs/user/quick-start/). -- Install `Stash` in your cluster following the steps [here](/docs/setup/README.md). +- Install `KubeStash` in your cluster following the steps [here](/docs/setup/README.md). -- You will need to have a PVC with `ReadWriteMany` access mode. Here, we are going to use an NFS server to provision a PVC with `ReadWriteMany` access mode. If you don't have an NFS server running, deploy one by following the guide [here](https://github.com/appscode/third-party-tools/blob/master/storage/nfs/README.md). +- You will need to have a PVC with `ReadWriteMany` access mode. Here, we are going to use an NFS server to provision a PVC with `ReadWriteMany` access mode. If you don't have an NFS server running, deploy one by following the guide [here](https://github.com/kubernetes-csi/csi-driver-nfs). -- You should be familiar with the following `Stash` concepts: +- You should be familiar with the following `KubeStash` concepts: + - [BackupStorage](/docs/concepts/crds/backupstorage/index.md) - [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md) - [BackupSession](/docs/concepts/crds/backupsession/index.md) - - [Repository](/docs/concepts/crds/repository/index.md) - [RestoreSession](/docs/concepts/crds/restoresession/index.md) - - [Function](/docs/concepts/crds/function/index.md) - - [Task](/docs/concepts/crds/task/index.md) + - [RetentionPolicy](/docs/concepts/crds/retentionpolicy/index.md) To keep everything isolated, we are going to use a separate namespace called `demo` throughout this tutorial. @@ -41,29 +40,45 @@ namespace/demo created > **Note:** YAML files used in this tutorial are stored in [docs/guides/volumes/pvc/examples](/docs/guides/volumes/pvc/examples) directory of [kubestash/docs](https://github.com/kubestash/docs) repository. -**Verify necessary Function and Task:** +**Verify necessary Addons and Functions:** -Stash uses a `Function-Task` model to backup stand-alone volume. When you install Stash, it automatically creates the necessary `Function` and `Task` to backup a stand-alone volume. +When you install KubeStash, it automatically creates the necessary `Addon` and `Function` to backup a stand-alone volume. -Let's verify that Stash has created the necessary `Function` to backup/restore PVC by the following command, +Let's verify that KubeStash has created the necessary `Function` to backup/restore PVC by the following command, ```bash -$ kubectl get function -NAME AGE -pvc-backup 117m -pvc-restore 117m -update-status 117m +$ kubectl get functions +NAME AGE +pvc-backup 3m25s +pvc-restore 3m25s +volumesnapshot-backup 3m25s +volumesnapshot-restore 3m25s ``` -Also, verify that the necessary `Task` has been created, +Also, verify that the necessary `Addon` has been created, ```bash -$ kubectl get task +$ kubectl get addons NAME AGE -pvc-backup 118m -pvc-restore 118m +pvc-addon 3m25s ``` +Now, you can view all `BackupTasks` and `RestoreTasks` of the `pvc-addon` addon using the following command + +```bash +$ kubectl get addon pvc-addon -o=jsonpath='{.spec.backupTasks[*].name}'| tr ' ' '\n' +logical-backup +volume-snapshot +``` + +```bash +$ kubectl get addon pvc-addon -o=jsonpath='{.spec.restoreTasks[*].name}'| tr ' ' '\n' +logical-backup-restore +volume-clone +volume-snapshot-restore +``` + + ## Prepare Volume At first, let's prepare our desired PVC. Here, we are going to create a [PersistentVolume](https://kubernetes.io/docs/concepts/storage/persistent-volumes/) (PV) that will use an NFS server as storage. Then, we are going to create a PVC that will bind with the PV. Then, we are going to mount this PVC in two different pods. Each pod will generate a sample file into the PVC. @@ -79,19 +94,20 @@ apiVersion: v1 kind: PersistentVolume metadata: name: nfs-pv - labels: - app: nfs-demo spec: capacity: storage: 1Gi accessModes: - - ReadWriteMany - nfs: - server: "nfs-service.storage.svc.cluster.local" - path: "/" + - ReadWriteMany + csi: + driver: nfs.csi.k8s.io + volumeHandle: nfs-server.storage.svc.cluster.local/share## + volumeAttributes: + server: nfs-server.storage.svc.cluster.local + share: / ``` -Notice the `metadata.labels` section. Here, we have added `app: nfs-demo` label. We are going to use this label as selector in PVC so that the PVC binds with this PV. +Notice the `spec.csi` section. Here, we have added `csi` driver information which represents that this storage is managed by an external CSI volume driver. Let's create the PV we have shown above, @@ -105,8 +121,8 @@ persistentvolume/nfs-pv created Now, create a PVC to bind with the PV we have just created. Below, is the YAML of the PVC that we are going to create, ```yaml -apiVersion: v1 kind: PersistentVolumeClaim +apiVersion: v1 metadata: name: nfs-pvc namespace: demo @@ -117,14 +133,12 @@ spec: resources: requests: storage: 1Gi - selector: - matchLabels: - app: nfs-demo + volumeName: nfs-pv ``` -Notice the `spec.accessModes` section. We are using `ReadWriteMany` access mode so that multiple pods can use this PVC simultaneously. Without this access mode, Stash will fail to backup the volume if any other pod mount it during backup. +Notice the `spec.accessModes` section. We are using `ReadWriteMany` access mode so that multiple pods can use this PVC simultaneously. Without this access mode, KubeStash will fail to backup the volume if any other pod mount it during backup. -Also, notice the `spec.selector` section. We have specified `app: nfs-demo` labels as a selector so that it binds with the PV that we have created earlier. +Also, notice the `spec.volumeName` section. We have specified `nfs-pv` as the PV that we have created earlier, which will be claimed by above PVC. Let's create the PVC we have shown above, @@ -145,7 +159,7 @@ Here, we can see that the PVC `nfs-pvc` has been bounded with PV `nfs-pv`. **Deploy Workload:** -Now, we are going to deploy two sample pods `demo-pod-1` and `demo-pod-2` that will mount `pod-1/data` and `pod-2/data` [subPath](https://kubernetes.io/docs/concepts/storage/volumes/#using-subpath) of the `nfs-pvc` respectively. Each of the pods will generate a sample file named `hello.txt` with some demo data. We are going to backup the entire PVC using Stash that contains the sample files. +Now, we are going to deploy two sample pods `demo-pod-1` and `demo-pod-2` that will mount `pod-1/data` and `pod-2/data` [subPath](https://kubernetes.io/docs/concepts/storage/volumes/#using-subpath) of the `nfs-pvc` respectively. Each of the pods will generate a sample file named `hello.txt` with some demo data. We are going to backup the entire PVC using KubeStash that contains the sample files. Below, is the YAML of the first pod that we are going to deploy, @@ -182,7 +196,7 @@ pod/demo-pod-1 created Verify that the sample data has been generated into `/sample/data/` directory, ```bash -$ kubectl exec -n demo demo-pod-1 cat /sample/data/hello.txt +$ kubectl exec -n demo demo-pod-1 -- cat /sample/data/hello.txt hello from pod 1. ``` @@ -221,98 +235,168 @@ pod/demo-pod-2 created Verify that the sample data has been generated into `/sample/data/` directory, ```bash -$ kubectl exec -n demo demo-pod-2 cat /sample/data/hello.txt +$ kubectl exec -n demo demo-pod-2 -- cat /sample/data/hello.txt hello from pod 2. ``` ## Backup -Now, we are going to backup the PVC `nfs-pvc` in a GCS bucket using Stash. We have to create a Secret and a `Repository` object with access credentials and backend information respectively. +Now, we are going to backup the PVC `nfs-pvc` in a GCS bucket using KubeStash. We have to create a Secret and a `BackupStorage` object with access credentials and backend information respectively. -> For GCS backend, if the bucket does not exist, Stash needs `Storage Object Admin` role permissions to create the bucket. For more details, please check the following [guide](/docs/guides/backends/gcs/index.md). +> For GCS backend, if the bucket does not exist, KubeStash needs `Storage Object Admin` role permissions to create the bucket. For more details, please check the following [guide](/docs/guides/backends/gcs/index.md). **Create Storage Secret:** Let's create a Secret named `gcs-secret` with access credentials of our desired GCS backend, ```bash -$ echo -n 'changeit' > RESTIC_PASSWORD $ echo -n '' > GOOGLE_PROJECT_ID $ cat /path/to/downloaded/sa_key_file.json > GOOGLE_SERVICE_ACCOUNT_JSON_KEY $ kubectl create secret generic -n demo gcs-secret \ - --from-file=./RESTIC_PASSWORD \ --from-file=./GOOGLE_PROJECT_ID \ --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY secret/gcs-secret created ``` -**Create Repository:** +**Create BackupStorage:** -Now, create a `Repository` crd specifying the desired bucket, and directory inside the bucket where the backed up data will be stored. +Now, create a `BackupStorage` custom resource specifying the desired bucket, and directory inside the bucket where the backed up data will be stored. -Below is the YAML of `Repository` crd that we are going to create, +Below is the YAML of `BackupStorage` object that we are going to create, ```yaml -apiVersion: stash.appscode.com/v1alpha1 -kind: Repository +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage metadata: - name: gcs-repo + name: gcs-storage namespace: demo spec: - backend: + storage: + provider: gcs gcs: - bucket: appscode-qa - prefix: stash-backup/volumes/nfs-pvc - storageSecretName: gcs-secret + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut +``` + +Let's create the `BackupStorage` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumes/pvc/examples/backupstorage.yaml +backupstorage.storage.kubestash.com/gcs-repo created +``` + +Now, we also have to create another secret with an encryption key `RESTIC_PASSWORD` for Restic. This secret will be used by Restic for both encrypting and decrypting the backup data during backup & restore. + +**Create Restic Repository Secret:** + +Let's create a Secret named `encry-secret` with an encryption key `RESTIC_PASSWORD` for Restic. + +```bash +$ echo -n 'changeit' > RESTIC_PASSWORD +$ kubectl create secret generic -n demo encryption-secret \ + --from-file=./RESTIC_PASSWORD +secret/encryption-secret created +``` + +Now, we have to create a `RetentionPolicy` custom resource which specifies how the old `Snapshots` should be cleaned up. + +**Create RetentionPolicy:** + +Below is the YAML of the `RetentionPolicy` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same ``` -Let's create the `Repository` object that we have shown above, +Let's create the `RetentionPolicy` object that we have shown above, ```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumes/pvc/examples/repository.yaml -repository.stash.appscode.com/gcs-repo created +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumes/pvc/examples/retentionpolicy.yaml +retentionpolicy.storage.kubestash.com/demo-retention created ``` -We are ready to start taking backup. Now, we have to create a `BackupConfiguration` object targeting `nfs-pvc`. +We are ready to start taking backup. Now, we have to create a `BackupConfiguration` custom resource targeting `nfs-pvc`. **Create BackupConfiguration:** Below is the YAML of the `BackupConfiguration` object that we are going to create, ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: name: nfs-pvc-backup namespace: demo spec: - task: - name: pvc-backup - repository: - name: gcs-repo - schedule: "*/5 * * * *" target: - ref: - apiVersion: v1 - kind: PersistentVolumeClaim - name: nfs-pvc - retentionPolicy: - name: keep-last-5 - keepLast: 5 - prune: true + apiGroup: + kind: PersistentVolumeClaim + name: nfs-pvc + namespace: demo + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /pvc-backup-demo + encryptionSecret: + name: encry-secret + namespace: demo + deletionPolicy: WipeOut + addon: + name: pvc-addon + tasks: + - name: logical-backup ``` Here, - -- `spec.task.name` specifies the name of the `Task` object that specifies the `Function` and their order of execution to perform a backup of a stand-alone PVC. -- `spec.repository.name` specifies the name of the `Repository` object that holds the backend information where the backed up data has been stored. -- `spec.target.ref` refers to the targeted PVC that will be backed up. +- `spec.target` specifies the backup target. `apiGroup`, `kind`, `namespace` and `name` refers to the `apiGroup`, `kind`, `namespace` and `name` of the targeted workload respectively. +- `spec.sessions[*].scheduler.schedule` specifies a [cron expression](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) indicates that `BackupSession` will be created at 5 minute interval. +- `spec.sessions[*].repositories[*].name` specifies the name of the `Repository` object that holds the backend information. +- `spec.sessions[*].repositories[*].backend` specifies the name of the backend that holds the `BackupStorage` information. +- `spec.sessions[*].repositories[*].directory` specifies the path of the `Repository` where the backed up data will be stored. +- `spec.sessions[*].repositories[*].encryptionSecret` specifies the encryption secret for `Restic Repository` which will be used to encrypting the backup data. +- `spec.sessions[*].addon.name` specifies the name of the `Addon` object that specifies addon configuration that will be used to perform backup of a stand-alone PVC. +- `spec.sessions[*].addon.tasks[*].name` specifies the name of the `Task` that holds the `Function` and their order of execution to perform backup of a stand-alone PVC. Let's create the `BackupConfiguration` object that we have shown above, ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumes/pvc/examples/backupconfiguration.yaml -backupconfiguration.stash.appscode.com/nfs-pvc-backup created +backupconfiguration.core.kubestash.com/nfs-pvc-backup created ``` **Verify Backup Setup Successful** @@ -321,18 +405,28 @@ If everything goes well, the phase of the `BackupConfiguration` should be in `Re ```bash $ kubectl get backupconfiguration -n demo -NAME TASK SCHEDULE PAUSED PHASE AGE -nfs-pvc-backup pvc-backup */5 * * * * Ready 11s +NAME PHASE PAUSED AGE +nfs-pvc-backup Ready 19s +``` + +**Verify Repository:** + +Verify that KubeStash has created `Repositories` that holds the `BackupStorage` information by the following command, + +```bash +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository Ready 28s ``` **Verify CronJob:** -Verify that Stash has created a CronJob to trigger a periodic backup of the targeted PVC by the following command, +Verify that KubeStash has created a CronJob to trigger a periodic backup of the targeted PVC by the following command, ```bash $ kubectl get cronjob -n demo -NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE -nfs-pvc-backup */5 * * * * False 0 28s +NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE +trigger-nfs-pvc-backup-frequent-backup */5 * * * * False 0 5s 40s ``` **Wait for BackupSession:** @@ -340,63 +434,52 @@ nfs-pvc-backup */5 * * * * False 0 28s Now, wait for the next backup schedule. You can watch for `BackupSession` crd using the following command, ```bash -$ watch -n 1 kubectl get backupsession -n demo -l=stash.appscode.com/backup-configuration=nfs-pvc-backup +$ watch -n 1 kubectl get backupsession -n demo -l=kubestash.com/invoker-name=nfs-pvc-backup -Every 1.0s: kubectl get backupsession -n demo -l=stash.appscode.com/backup-... workstation: Wed Jul 3 19:53:13 2019 +Every 1.0s: kubectl get backupsession -n demo -l=kubestash.com/invoker-name=nfs-pvc-backup workstation: Wed Jan 3 17:26:00 2024 -NAME INVOKER-TYPE INVOKER-NAME PHASE AGE -nfs-pvc-backup-1562161802 BackupConfiguration nfs-pvc-backup Succeeded 3m11s -``` +NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE +nfs-pvc-backup-frequent-backup-1704281100 BackupConfiguration nfs-pvc-backup Succeeded 60s -> Note: Respective CronJob creates `BackupSession` crd with the following label: `stash.appscode.com/backup-configuration=`. We can use this label to watch only the `BackupSession` of our desired `BackupConfiguration`. +``` **Verify Backup:** -When backup session is completed, Stash will update the respective `Repository` to reflect the latest state of backed up data. +When backup session is created, kubestash operator creates `Snapshot` which represents the state of backup run for each `Repository` which are provided in `BackupConfiguration`. + +Run the following command to check `Snapshot` phase, + +```bash +$ kubectl get snapshots -n demo +NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE VERIFICATION-STATUS AGE +gcs-repository-nfs-pvc-backup-frequent-backup-1704281100 gcs-repository frequent-backup 2024-01-03T11:25:13Z Delete Succeeded 2m14s +``` + +When backup session is completed, KubeStash will update the respective `Repository` to reflect the latest state of backed up data. Run the following command to check if a backup snapshot has been stored in the backend, ```bash -$ kubectl get repository -n demo gcs-repo -NAME INTEGRITY SIZE SNAPSHOT-COUNT LAST-SUCCESSFUL-BACKUP AGE -gcs-repo true 80 B 1 25s 49m +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository true 1 2.262 KiB Ready 103s 8m + ``` -From the output above, we can see that 1 snapshot has been stored in the backend specified by Repository `gcs-repo`. +From the output above, we can see that 1 snapshot has been stored in the backend specified by Repository `gcs-repository`. -If we navigate to `stash-backup/volumes/nfs-pvc` directory of our GCS bucket, we are going to see that the snapshot has been stored there. +If we navigate to `kubestash-qa/demo/pvc-backup-demo/repository/v1/frequent-backup/pvc` directory of our GCS bucket, we are going to see that the snapshot has been stored there.
  Backed up data of a stand-alone PVC in GCS backend
Fig: Backed up data of a stand-alone PVC in GCS backend
-> Stash keeps all backup data encrypted. So, snapshot files in the bucket will not contain any meaningful data until they are decrypted. +> KubeStash keeps all backup data encrypted. So, snapshot files in the bucket will not contain any meaningful data until they are decrypted. ## Restore -This section will show you how to restore the backed up data inside a stand-alone PVC using stash. Here, we are going to restore the data we have backed up in the previous section. - -**Stop Taking Backup of the PVC:** - -At first, let's stop taking any further backup of the PVC so that no backup is taken during the restore process. We are going to pause the `BackupConfiguration` that we created to backup the `nfs-pvc` PVC. Then, Stash will stop taking any further backup for this PVC. You can learn more how to pause a scheduled backup [here](/docs/guides/use-cases/pause-backup/index.md). - -Let's pause the `nfs-pvc-backup` BackupConfiguration, - -```bash -$ kubectl patch backupconfiguration -n demo nfs-pvc-backup --type="merge" --patch='{"spec": {"paused": true}}' -backupconfiguration.stash.appscode.com/nfs-pvc-backup patched -``` - -Now, wait for a moment. Stash will pause the BackupConfiguration. Verify that the BackupConfiguration has been paused, - -```bash -$ kubectl get backupconfiguration -n demo nfs-pvc-backup -NAME TASK SCHEDULE PAUSED AGE -nfs-pvc-backup */5 * * * * true 20m -``` - -Notice the `PAUSED` column. Value `true` for this field means that the BackupConfiguration has been paused. +This section will show you how to restore the backed up data inside a stand-alone PVC using KubeStash. Here, we are going to restore the data we have backed up in the previous section. **Simulate Disaster:** @@ -409,7 +492,7 @@ Delete the data of pod `demo-pod-1`: $ kubectl exec -n demo demo-pod-1 -- sh -c "rm /sample/data/*" # verify that data has been removed successfully -$ kubectl exec -n demo demo-pod-1 ls /sample/data/ +$ kubectl exec -n demo demo-pod-1 -- ls /sample/data/ # empty output which means all the files have been deleted ``` @@ -420,43 +503,51 @@ Delete the data of pod `demo-pod-2`: $ kubectl exec -n demo demo-pod-2 -- sh -c "rm /sample/data/*" # verify that data has been removed successfully -$ kubectl exec -n demo demo-pod-2 ls /sample/data/ +$ kubectl exec -n demo demo-pod-2 -- ls /sample/data/ # empty output which means all the files have been deleted ``` **Create RestoreSession:** -Now, we are going to create a `RestoreSession` object to restore the backed up data into the desired PVC. Below is the YAML of the `RestoreSession` object that we are going to create, +Now, we are going to create a `RestoreSession` object to restore the backed up data into the desired PVC. +Below is the YAML of the `RestoreSession` object that we are going to create, ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: RestoreSession metadata: name: nfs-pvc-restore namespace: demo spec: - task: - name: pvc-restore - repository: - name: gcs-repo target: - ref: - apiVersion: v1 - kind: PersistentVolumeClaim - name: nfs-pvc - rules: - - snapshots: ["latest"] -``` - -- `spec.task.name` specifies the name of the `Task` object that specifies the `Function` and their order of execution to restore data inside a stand-alone PVC. -- `spec.target.ref` refers to the targeted PVC where the data will be restored. -- `spec.target.rules[*].snapshots` specifies that we want to restore the latest snapshot of the `nfs-pvc`. + apiGroup: + kind: PersistentVolumeClaim + name: nfs-pvc + namespace: demo + dataSource: + repository: gcs-repository + snapshot: latest + encryptionSecret: + name: encryption-secret + namespace: demo + addon: + name: pvc-addon + tasks: + - name: logical-backup-restore +``` + +- `spec.target` refers to the targeted PVC where the data will be restored. +- `spec.dataSource.repository` specifies the name of the `Repository` from which the data will be restored. +- `spec.dataSource.snapshot` specifies that we want to restore the latest snapshot of the `gcs-repository`. +- `spec.dataSource.encryptionSecret` specifies the encryption secret for `Restic Repository` used during backup. It will be used to decrypting the backup data. +- `spec.addon.name` specifies the name of the `Addon` object that specifies addon configuration that will be used to perform restore of a stand-alone PVC. +- `spec.addon.tasks[*].name` specifies the name of the `Task` that holds the Function and their order of execution to perform restore of a stand-alone PVC. Let's create the `RestoreSession` object that we have shown above, ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumes/pvc/examples/restoresession.yaml -restoresession.stash.appscode.com/nfs-pvc-restore created +restoresession.core.kubestash.com/nfs-pvc-restore created ``` **Wait for RestoreSession to Succeed:** @@ -466,9 +557,10 @@ Now, wait for the restore process to complete. You can watch the `RestoreSession ```bash $ watch -n 1 kubectl get restoresession -n demo nfs-pvc-restore -Every 1.0s: kubectl get restoresession -n demo nfs-pvc-restore workstation: Wed Jul 3 20:10:52 2019 -NAME REPOSITORY-NAME PHASE AGE -nfs-pvc-restore gcs-repo Succeeded 32s +Every 1.0s: kubectl get restoresession -n demo nfs-pvc... workstation: Wed Jan 3 17:30:20 2024 + +NAME REPOSITORY FAILURE-POLICY PHASE DURATION AGE +nfs-pvc-restore gcs-repository Succeeded 10s 51s ``` From the output of the above command, we can see that restoration process has been completed successfully. @@ -480,14 +572,14 @@ Let's verify if the deleted files have been restored successfully into the PVC. Verify that the data of `demo-pod-1` has been restored: ```bash -$ kubectl exec -n demo demo-pod-1 cat /sample/data/hello.txt +$ kubectl exec -n demo demo-pod-1 -- cat /sample/data/hello.txt hello from pod 1. ``` Verify that the data of `demo-pod-2` has been restored: ```bash -$ kubectl exec -n demo demo-pod-2 cat /sample/data/hello.txt +$ kubectl exec -n demo demo-pod-2 -- cat /sample/data/hello.txt hello from pod 2. ``` @@ -498,11 +590,15 @@ So, we can see from the above output that the files we had deleted in **Simulate To cleanup the Kubernetes resources created by this tutorial, run: ```bash + kubectl delete backupconfiguration -n demo nfs-pvc-backup kubectl delete restoresession -n demo nfs-pvc-restore +kubectl delete backupstorage -n demo gcs-storage +kubectl delete retentionPolicy -n demo demo-retention + kubectl delete secret -n demo gcs-secret -kubectl delete repository -n demo gcs-repo +kubectl delete secret -n demo encryption-secret kubectl delete pod -n demo demo-pod-1 kubectl delete pod -n demo demo-pod-2 @@ -511,4 +607,4 @@ kubectl delete pvc -n demo nfs-pvc kubectl delete pv -n demo nfs-pv ``` -If you would like to uninstall Stash operator, please follow the steps [here](/docs/setup/README.md). +If you would like to uninstall KubeStash operator, please follow the steps [here](/docs/setup/README.md). diff --git a/docs/guides/volumesnapshot/deployment/examples/backupconfiguration.yaml b/docs/guides/volumesnapshot/deployment/examples/backupconfiguration.yaml index 743416c..da1827d 100644 --- a/docs/guides/volumesnapshot/deployment/examples/backupconfiguration.yaml +++ b/docs/guides/volumesnapshot/deployment/examples/backupconfiguration.yaml @@ -1,18 +1,37 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: - name: deployments-volume-snapshot + name: deployment-volume-snapshot namespace: demo spec: - schedule: "*/5 * * * *" - driver: VolumeSnapshotter target: - ref: - apiVersion: apps/v1 - kind: Deployment - name: stash-demo - snapshotClassName: csi-snapshot-class - retentionPolicy: - name: "keep-last-5" - keepLast: 5 - prune: true + apiGroup: apps + kind: Deployment + name: kubestash-demo + namespace: demo + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /deployment-volume-snapshot-repo + deletionPolicy: WipeOut + addon: + name: workload-addon + tasks: + - name: volume-snapshot + params: + volumeSnapshotClassName: csi-snapshot-class \ No newline at end of file diff --git a/docs/guides/volumesnapshot/deployment/examples/backupstorage.yaml b/docs/guides/volumesnapshot/deployment/examples/backupstorage.yaml new file mode 100644 index 0000000..0461b26 --- /dev/null +++ b/docs/guides/volumesnapshot/deployment/examples/backupstorage.yaml @@ -0,0 +1,17 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/volumesnapshot/deployment/examples/deployment.yaml b/docs/guides/volumesnapshot/deployment/examples/deployment.yaml index 98c57df..db694d2 100644 --- a/docs/guides/volumesnapshot/deployment/examples/deployment.yaml +++ b/docs/guides/volumesnapshot/deployment/examples/deployment.yaml @@ -2,36 +2,36 @@ apiVersion: apps/v1 kind: Deployment metadata: labels: - app: stash-demo - name: stash-demo + app: kubestash-demo + name: kubestash-demo namespace: demo spec: replicas: 1 selector: matchLabels: - app: stash-demo + app: kubestash-demo template: metadata: labels: - app: stash-demo + app: kubestash-demo name: busybox spec: containers: - - args: ["echo sample_data > /source/data/data.txt; echo sample_config > /source/config/config.cfg && sleep 3000"] - command: ["/bin/sh", "-c"] - image: busybox - imagePullPolicy: IfNotPresent - name: busybox - volumeMounts: - - mountPath: /source/data - name: source-data - - mountPath: /source/config - name: source-config + - args: ["echo sample_data > /source/data/data.txt; echo sample_config > /source/config/config.cfg && sleep 3000"] + command: ["/bin/sh", "-c"] + image: busybox + imagePullPolicy: IfNotPresent + name: busybox + volumeMounts: + - mountPath: /source/data + name: source-data + - mountPath: /source/config + name: source-config restartPolicy: Always volumes: - - name: source-data - persistentVolumeClaim: - claimName: source-data - - name: source-config - persistentVolumeClaim: - claimName: source-config + - name: source-data + persistentVolumeClaim: + claimName: source-data + - name: source-config + persistentVolumeClaim: + claimName: source-config \ No newline at end of file diff --git a/docs/guides/volumesnapshot/deployment/examples/restoresession.yaml b/docs/guides/volumesnapshot/deployment/examples/restoresession.yaml index b80e1e2..13f390d 100644 --- a/docs/guides/volumesnapshot/deployment/examples/restoresession.yaml +++ b/docs/guides/volumesnapshot/deployment/examples/restoresession.yaml @@ -1,33 +1,31 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: RestoreSession metadata: - name: restore-pvc + name: restore-deployment-pvc namespace: demo spec: - driver: VolumeSnapshotter - target: - volumeClaimTemplates: - - metadata: - name: restore-data - spec: - accessModes: ["ReadWriteOnce"] - storageClassName: "csi-standard" - resources: - requests: - storage: 1Gi - dataSource: - kind: VolumeSnapshot - name: source-data-fnbwz - apiGroup: snapshot.storage.k8s.io - - metadata: - name: restore-config - spec: - accessModes: ["ReadWriteOnce"] - storageClassName: "csi-standard" - resources: - requests: - storage: 1Gi - dataSource: - kind: VolumeSnapshot - name: source-config-fnbwz - apiGroup: snapshot.storage.k8s.io + dataSource: + repository: gcs-repository + snapshot: latest + addon: + name: pvc-addon + tasks: + - name: VolumeSnapshotRestore + targetVolumes: + volumeClaimTemplates: + - metadata: + name: restore-data + spec: + accessModes: [ "ReadWriteOnce" ] + storageClassName: "csi-standard" + resources: + requests: + storage: 1Gi + - metadata: + name: restore-config + spec: + accessModes: [ "ReadWriteOnce" ] + storageClassName: "csi-standard" + resources: + requests: + storage: 1Gi \ No newline at end of file diff --git a/docs/guides/volumesnapshot/deployment/examples/retentionpolicy.yaml b/docs/guides/volumesnapshot/deployment/examples/retentionpolicy.yaml new file mode 100644 index 0000000..3af9037 --- /dev/null +++ b/docs/guides/volumesnapshot/deployment/examples/retentionpolicy.yaml @@ -0,0 +1,15 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same \ No newline at end of file diff --git a/docs/guides/volumesnapshot/deployment/examples/snapshot.yaml b/docs/guides/volumesnapshot/deployment/examples/snapshot.yaml new file mode 100644 index 0000000..182d34c --- /dev/null +++ b/docs/guides/volumesnapshot/deployment/examples/snapshot.yaml @@ -0,0 +1,62 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: Snapshot +metadata: + creationTimestamp: "2024-01-17T09:20:01Z" + finalizers: + - kubestash.com/cleanup + generation: 1 + labels: + kubestash.com/app-ref-kind: Deployment + kubestash.com/app-ref-name: kubestash-demo + kubestash.com/app-ref-namespace: demo + kubestash.com/repo-name: gcs-repository + name: gcs-repository-deployment-volumeshot-frequent-backup-1705483201 + namespace: demo + ownerReferences: + - apiVersion: storage.kubestash.com/v1alpha1 + blockOwnerDeletion: true + controller: true + kind: Repository + name: gcs-repository + uid: 067f9690-8cf0-498b-bc0e-7d21a0a407d4 + resourceVersion: "8885" + uid: 4f4a5d2b-269b-4d44-ba91-f6188008ee92 +spec: + appRef: + apiGroup: apps + kind: Deployment + name: kubestash-demo + namespace: demo + backupSession: deployment-volume-snapshot-frequent-backup-1705483201 + deletionPolicy: Delete + repository: gcs-repository + session: frequent-backup + snapshotID: 01HMBBGTJDZ0F6XRD8BWSXB9C0 + type: FullBackup + version: v1 +status: + components: + volumesnapshot: + driver: VolumeSnapshotter + phase: Succeeded + volumeSnapshotterStats: + - hostPath: /source/data + pvcName: source-data + volumeSnapshotName: source-data-1705483201 + - hostPath: /source/config + pvcName: source-config + volumeSnapshotName: source-config-1705483201 + conditions: + - lastTransitionTime: "2024-01-17T09:20:01Z" + message: Recent snapshot list updated successfully + reason: SuccessfullyUpdatedRecentSnapshotList + status: "True" + type: RecentSnapshotListUpdated + - lastTransitionTime: "2024-01-17T09:20:26Z" + message: Metadata uploaded to backend successfully + reason: SuccessfullyUploadedSnapshotMetadata + status: "True" + type: SnapshotMetadataUploaded + phase: Succeeded + snapshotTime: "2024-01-17T09:20:04Z" + totalComponents: 1 \ No newline at end of file diff --git a/docs/guides/volumesnapshot/deployment/examples/volumesnapshot-source-data.yaml b/docs/guides/volumesnapshot/deployment/examples/volumesnapshot-source-data.yaml new file mode 100644 index 0000000..2b16457 --- /dev/null +++ b/docs/guides/volumesnapshot/deployment/examples/volumesnapshot-source-data.yaml @@ -0,0 +1,21 @@ +apiVersion: snapshot.storage.k8s.io/v1 +kind: VolumeSnapshot +metadata: + creationTimestamp: "2024-01-17T09:20:04Z" + finalizers: + - snapshot.storage.kubernetes.io/volumesnapshot-as-source-protection + - snapshot.storage.kubernetes.io/volumesnapshot-bound-protection + generation: 1 + name: source-data-1705483201 + namespace: demo + resourceVersion: "8875" + uid: b3b3118e-318d-4876-b69b-f124ce868d24 +spec: + source: + persistentVolumeClaimName: source-data + volumeSnapshotClassName: csi-snapshot-class +status: + boundVolumeSnapshotContentName: snapcontent-b3b3118e-318d-4876-b69b-f124ce868d24 + creationTime: "2024-01-17T09:20:06Z" + readyToUse: true + restoreSize: 1Gi diff --git a/docs/guides/volumesnapshot/deployment/images/gcp.png b/docs/guides/volumesnapshot/deployment/images/gcp.png index 363c002..1b474b2 100644 Binary files a/docs/guides/volumesnapshot/deployment/images/gcp.png and b/docs/guides/volumesnapshot/deployment/images/gcp.png differ diff --git a/docs/guides/volumesnapshot/deployment/index.md b/docs/guides/volumesnapshot/deployment/index.md index abfa540..2f68137 100644 --- a/docs/guides/volumesnapshot/deployment/index.md +++ b/docs/guides/volumesnapshot/deployment/index.md @@ -1,5 +1,5 @@ --- -title: Snapshot Deployment Volumes | Stash +title: Snapshot Deployment Volumes | KubeStash description: An step by step guide showing how to snapshot the volumes of a Deployment menu: docs_{{ .version }}: @@ -14,13 +14,13 @@ section_menu_id: guides # Snapshotting the volumes of a Deployment -This guide will show you how to use Stash to snapshot the volumes of a Deployment and restore them from the snapshots using Kubernetes [VolumeSnapshot](https://kubernetes.io/docs/concepts/storage/volume-snapshots/) API. In this guide, we are going to backup the volumes in Google Cloud Platform with the help of [GCE Persistent Disk CSI Driver](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver). +This guide will show you how to use KubeStash to snapshot the volumes of a Deployment and restore them from the snapshots using Kubernetes [VolumeSnapshot](https://kubernetes.io/docs/concepts/storage/volume-snapshots/) API. In this guide, we are going to backup the volumes in Google Cloud Platform with the help of [GCE Persistent Disk CSI Driver](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver). ## Before You Begin -- You need to be familiar with the [GCE Persistent Disk CSI Driver](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver). -- Install `Stash` in your cluster following the steps [here](/docs/setup/README.md). -- If you don't know how VolumeSnapshot works in Stash, please visit [here](/docs/guides/volumesnapshot/overview/index.md). +- At first, you need to be familiar with the [GCE Persistent Disk CSI Driver](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver). +- Install `KubeStash` in your cluster following the steps [here](/docs/setup/README.md). +- If you don't know how VolumeSnapshot works in KubeStash, please visit [here](/docs/guides/volumesnapshot/overview/index.md). ## Prepare for VolumeSnapshot @@ -75,8 +75,7 @@ To keep everything isolated, we are going to use a separate namespace called `de $ kubectl create ns demo namespace/demo created ``` - -> Note: YAML files used in this tutorial are stored in [/docs/guides/volumesnapshot/deployment/examples](/docs/guides/volumesnapshot/deployment/examples/) directory of [kubestash/docs](https://github.com/kubestash/docs) repository. +> Note: YAML files used in this tutorial are stored in [/docs/guides/volumesnapshot/pvc/examples](/docs/guides/volumesnapshot/statefulset/examples/) directory of [kubestash/docs](https://github.com/kubestash/docs) repository. ## Take Volume Snapshot @@ -116,7 +115,7 @@ spec: storage: 1Gi ``` -:et's create the PVCs we have shown above. +Let's create the PVCs we have shown above. ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/deployment/examples/pvcs.yaml @@ -135,18 +134,18 @@ apiVersion: apps/v1 kind: Deployment metadata: labels: - app: stash-demo - name: stash-demo + app: kubestash-demo + name: kubestash-demo namespace: demo spec: replicas: 1 selector: matchLabels: - app: stash-demo + app: kubestash-demo template: metadata: labels: - app: stash-demo + app: kubestash-demo name: busybox spec: containers: @@ -174,231 +173,370 @@ Let's create the deployment we have shown above. ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/deployment/examples/deployment.yaml -deployment.apps/stash-demo created +deployment.apps/kubestash-demo created ``` Now, wait for the pod of the Deployment to go into the `Running` state. ```bash $ kubectl get pod -n demo -NAME READY STATUS RESTARTS AGE -stash-demo-7fd48dd5b4-xqv5n 1/1 Running 0 2m10s +NAME READY STATUS RESTARTS AGE +kubestash-demo-5f7fcd67c8-4z766 1/1 Running 0 79s ``` Verify that the sample data has been created in `/source/data` and `/source/config` directory using the following command, ```bash -$ kubectl exec -n demo stash-demo-7fd48dd5b4-xqv5n -- cat /source/data/data.txt +$ kubectl exec -n demo kubestash-demo-5f7fcd67c8-4z766 -- cat /source/data/data.txt sample_data -$ kubectl exec -n demo stash-demo-7fd48dd5b4-xqv5n -- cat /source/config/config.cfg +$ kubectl exec -n demo kubestash-demo-5f7fcd67c8-4z766 -- cat /source/config/config.cfg config_data ``` +## Backup + +Now, we are going to taking `VolumeSnapshot` of the Deployment `kubestash-demo` PVCs using KubeStash. We have to create a `Secret` and a `BackupStorage` object with access credentials and backend information respectively. + +> For GCS backend, if the bucket does not exist, KubeStash needs `Storage Object Admin` role permissions to create the bucket. For more details, please check the following [guide](/docs/guides/backends/gcs/index.md). + +**Create Storage Secret:** + +Let's create a Secret named `gcs-secret` with access credentials of our desired GCS backend, + +```bash +$ echo -n '' > GOOGLE_PROJECT_ID +$ cat /path/to/downloaded/sa_key_file.json > GOOGLE_SERVICE_ACCOUNT_JSON_KEY +$ kubectl create secret generic -n demo gcs-secret \ + --from-file=./GOOGLE_PROJECT_ID \ + --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY +secret/gcs-secret created +``` + +**Create BackupStorage:** + +Now, create a `BackupStorage` custom resource specifying the desired bucket, and directory inside the bucket where the backed up data will be stored. + +Below is the YAML of `BackupStorage` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut +``` + +Let's create the `BackupStorage` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumes/statefulset/examples/backupstorage.yaml +backupstorage.storage.kubestash.com/gcs-storage created +``` + +Now, we have to create a `RetentionPolicy` custom resource which specifies how the old `Snapshots` should be cleaned up. + +**Create RetentionPolicy:** + +Below is the YAML of the `RetentionPolicy` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same +``` + +Let's create the `RetentionPolicy` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumes/statefulset/examples/retentionpolicy.yaml +retentionpolicy.storage.kubestash.com/demo-retention created +``` + +We are ready. Now, we have to create a `BackupConfiguration` custom resource targeting Deployment `kubestash-demo`. + **Create BackupConfiguration :** -Now, create a `BackupConfiguration` object to take snapshot of the PVCs of `stash-demo` Deployment. +Now, create a `BackupConfiguration` custom resource to take `volumeSnapshot` of the `kubestash-demo` Deployment. Below is the YAML of the `BackupConfiguration` that we are going to create, ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: name: deployment-volume-snapshot namespace: demo spec: - schedule: "*/5 * * * *" - driver: VolumeSnapshotter target: - ref: - apiVersion: apps/v1 - kind: Deployment - name: stash-demo - snapshotClassName: csi-snapshot-class - retentionPolicy: - name: 'keep-last-5' - keepLast: 5 - prune: true + apiGroup: apps + kind: Deployment + name: kubestash-demo + namespace: demo + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /deployment-volume-snapshot-repo + deletionPolicy: WipeOut + addon: + name: workload-addon + tasks: + - name: volume-snapshot + params: + volumeSnapshotClassName: csi-snapshot-class ``` - Here, - -- `spec.schedule` is a [cron expression](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) that indicates `BackupSession` will be created at 5 minutes interval. - -- `spec.driver` indicates the name of the agent to use to back up the target. Currently, Stash supports `Restic` and `VolumeSnapshotter` drivers. The `VolumeSnapshotter` is used to backup/restore PVC using `VolumeSnapshot` API. - -- `spec.target.ref` refers to the backup target. `apiVersion`, `kind` and `name` refers to the `apiVersion`, `kind` and `name` of the targeted workload respectively. Stash will use this information to create a Volume Snapshotter Job for creating VolumeSnapshot. - -- `spec.target.snapshotClassName` indicates the [VolumeSnapshotClass](https://kubernetes.io/docs/concepts/storage/volume-snapshot-classes/) to be used for volume snapshotting. +- `spec.target` specifies the backup target. `apiGroup`, `kind`, `namespace` and `name` refers to the `apiGroup`, `kind`, `namespace` and `name` of the targeted workload respectively. +- `spec.sessions[*].scheduler.schedule` specifies a [cron expression](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) indicates that `BackupSession` will be created at 5 minute interval. +- `spec.sessions[*].repositories[*].name` specifies the name of the `Repository` object that holds the backend information. +- `spec.sessions[*].repositories[*].backend` specifies the name of the backend that holds the `BackupStorage` information. +- `spec.sessions[*].repositories[*].directory` specifies the path of the `Repository` where the backed up data will be stored. +- `spec.sessions[*].addon.name` specifies the name of the `Addon` object that specifies addon configuration that will be used to perform backup of a stand-alone PVC. +- `spec.sessions[*].addon.tasks[*].name` specifies the name of the `Task` that holds the `Function` and their order of execution to perform backup of a stand-alone PVC. +- `spec.sessions[*].addon.tasks[*].params.volumeSnapshotClassName` indicates the [VolumeSnapshotClass](https://kubernetes.io/docs/concepts/storage/volume-snapshot-classes/) to be used for volume snapshotting. If we don't provide any then kubestash use default `volumeSnapshotClass` for volume snapshotting. Let's create the `BackupConfiguration` object we have shown above. ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/deployment/examples/backupconfiguration.yaml -backupconfiguration.stash.appscode.com/deployment-volume-snapshot created +backupconfiguration.core.kubestash.com/deployment-volume-snapshot created ``` **Verify Backup Setup Successful**: -If everything goes well, the phase of the `BackupConfiguration` should be `Ready`. The `Ready` phase indicates that the backup setup is successful. Let’s verify the Phase of the `BackupConfiguration`, +If everything goes well, the phase of the `BackupConfiguration` should be in `Ready` state. The `Ready` phase indicates that the backup setup is successful. Let's check the `Phase` of the BackupConfiguration, ```bash $ kubectl get backupconfiguration -n demo -NAME TASK SCHEDULE PAUSED PHASE AGE -deployment-volume-snapshot */5 * * * * Ready 11s +NAME PHASE PAUSED AGE +deployment-volume-snapshot Ready 12s ``` +**Verify Repository:** -**Verify CronJob :** +Verify that KubeStash has created `Repositories` that holds the `BackupStorage` information by the following command, -Stash will create a CronJob with the schedule specified in `spec.schedule` field of `BackupConfiguration` CRD. Verify that the CronJob has been created using the following command, +```bash +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository Ready 28s +``` + +**Verify CronJob:** + +If everything goes well, KubeStash will create a `CronJob` to take periodic `VolumeSnapshot` of `source-data`, `source-config` volumes of the Deployment with the schedule specified in `spec.schedule` field of `BackupConfiguration` object. + +Check that the `CronJob` has been created using the following command, ```bash $ kubectl get cronjob -n demo -NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE -deployment-volume-snapshot */1 * * * * False 0 39s 2m41s +NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE +trigger-deployment-volume-snapshot-frequent-backup */5 * * * * False 0 20s ``` -**Wait for BackupSession :** +**Wait for BackupSession:** + +Now, wait for the next backup schedule. You can watch for `BackupSession` object using the following command, -The `deployment-volume-snapshot` CronJob will trigger a backup on each schedule by creating a `BackupSession` crd. +```bash +$ watch -n 1 kubectl get backupsession -n demo -l=kubestash.com/invoker-name=deployment-volume-snapshot -Wait for the next schedule for backup. Run the following command to watch `BackupSession` crd, +Every 1.0s: kubectl get backupsession -n demo -l=kubestash.com/invoker-name=deployment-volume-snapshot anisur: Wed Jan 17 15:20:09 2024 + +NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE +deployment-volume-snapshot-frequent-backup-1705483201 BackupConfiguration deployment-volume-snapshot Running 9s +``` + +Once a `BackupSession` object is created, it creates a `Snapshot` custom resource for each `Repository`. This snapshot `spec.status.components` section represents the backup information of the targeted backup components. + +Verify created `Snapshot` object by the following command, ```bash -$ watch -n 1 kubectl get backupsession -n demo -Every 1.0s: kubectl get backupsession -n demo +$ watch -n 1 kubectl get snapshot -n demo -l=kubestash.com/repo-name=gcs-repository + +Every 1.0s: kubectl get snapshot -n demo -l=kubestash.com/repo-name=gcs-repository anisur: Wed Jan 17 15:20:56 2024 -NAME INVOKER-TYPE INVOKER-NAME PHASE AGE -deployment-volume-snapshot-fnbwz BackupConfiguration deployment-volume-snapshot Succeeded 50s +NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE VERIFICATION-STATUS AGE +gcs-repository-deployment-volumeshot-frequent-backup-1705483201 gcs-repository frequent-backup 2024-01-17T09:20:04Z Delete Succeeded 56s ``` +Now, retrieve the YAML representation of the above created `Snapshot`, and inspect the `spec.status` section to see the backup information of the targeted backup components. -We can see above that the BackupSession has been succeeded. Now, we are going to verify that the `VolumeSnapshot` has been created and the snapshots has been stored in the respective backend. +```bash +$ kubectl get snapshots -n demo gcs-repository-deployment-volumeshot-frequent-backup-1705483201 -o yaml +``` +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: Snapshot +metadata: + name: gcs-repository-deployment-volumeshot-frequent-backup-1705483201 + namespace: demo +spec: + --- +status: + components: + volumesnapshot: + driver: VolumeSnapshotter + phase: Succeeded + volumeSnapshotterStats: + - hostPath: /source/data + pvcName: source-data + volumeSnapshotName: source-data-1705483201 + - hostPath: /source/config + pvcName: source-config + volumeSnapshotName: source-config-1705483201 + conditions: + --- + phase: Succeeded + totalComponents: 1 +``` + +Here, +- `spec.status.components` specifies the backup information of the backup target components. +- `spec.status.components.[].volumeSnapshotterStats` specifies the information about the `VolumeSnapshotter` driver. In the case of a deployment `PVC`, the component name is designated as `volumesnapshot`. + - `volumeSnapshotterStats.hostPath` specifies the corresponding path of `PVC` volume for which `volumeSnapshot` has created. + - `volumeSnapshotterStats.pvcName` specifies the name of the backed-up target `PVC` volume. + - `volumeSnapshotterStats.volumeSnapshotName` specifies the created `VolumeSnapshot` object name corresponding to the target `PVC` volume. **Verify Volume Snapshot :** -Once a `BackupSession` crd is created, it creates volume snapshotter `Job`. Then the `Job` creates `VolumeSnapshot` crd for the targeted PVCs. +Once a `BackupSession` object is created, it creates volume snapshotter `Job`. Then the `Job` creates `VolumeSnapshot` object for each the targeted StatefulSet volumes. We can see above that the backup session has succeeded. Now, we are going to verify that the `VolumeSnapshot` has been created and the snapshot have been stored in the respective backend. -Check that the `VolumeSnapshot` has been created Successfully. +Check that all the `VolumeSnapshot` has been created Successfully. ```bash $ kubectl get volumesnapshot -n demo -NAME AGE -source-config-fnbwz 1m46s -source-data-fnbwz 1m46s +NAME READYTOUSE SOURCEPVC SOURCESNAPSHOTCONTENT RESTORESIZE SNAPSHOTCLASS SNAPSHOTCONTENT CREATIONTIME AGE +source-config-1705483201 true source-config 1Gi csi-snapshot-class snapcontent-3fa30373-e2f7-4288-8bee-4ff2c7720117 3m1s 3m3s +source-data-1705483201 true source-data 1Gi csi-snapshot-class snapcontent-b3b3118e-318d-4876-b69b-f124ce868d24 3m1s 3m3s ``` -Let's find the name of the snapshot that has been saved in the Google Cloud by the following command, +Let's find out the actual `VolumeSnapshotContent` that has been saved in the Google Cloud by the following command, ```bash -kubectl get volumesnapshot source-data-fnbwz -n demo -o yaml +kubectl get volumesnapshot source-data-1705483201 -n demo -o yaml ``` ```yaml apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshot metadata: - creationTimestamp: "2019-07-15T06:14:09Z" + creationTimestamp: "2024-01-17T09:20:04Z" finalizers: - - snapshot.storage.kubernetes.io/volumesnapshot-protection - generation: 4 - name: source-data-fnbwz + - snapshot.storage.kubernetes.io/volumesnapshot-as-source-protection + - snapshot.storage.kubernetes.io/volumesnapshot-bound-protection + generation: 1 + name: source-data-1705483201 namespace: demo - resourceVersion: "9220" - selfLink: /apis/snapshot.storage.k8s.io/v1/namespaces/demo/volumesnapshots/source-data-fnbwz - uid: c1bc3390-a6c7-11e9-9f3a-42010a800050 + resourceVersion: "8875" + uid: b3b3118e-318d-4876-b69b-f124ce868d24 spec: source: persistentVolumeClaimName: source-data volumeSnapshotClassName: csi-snapshot-class status: - boundVolumeSnapshotContentName: snapcontent-c1bc3390-a6c7-11e9-9f3a-42010a800050 - creationTime: "2019-07-15T06:14:10Z" + boundVolumeSnapshotContentName: snapcontent-b3b3118e-318d-4876-b69b-f124ce868d24 + creationTime: "2024-01-17T09:20:06Z" readyToUse: true restoreSize: 1Gi + ``` -Here, `status.snapshotContentName` field specifies the name of the `VolumeSnapshotContent` crd. It also represents the actual snapshot name that has been saved in Google Cloud. If we navigate to the `Snapshots` tab in the GCP console, we are going to see the snapshot `snapcontent-c1bc3390-a6c7-11e9-9f3a-42010a800050` has been stored successfully. +Here, `spec.status.boundVolumeSnapshotContentName` field specifies the name of the `VolumeSnapshotContent` object. It also represents the actual snapshot name that has been saved in Google Cloud. If we navigate to the `Snapshots` tab in the GCP console, we are going to see snapshot `snapcontent-f4a199c2-eed5-4438-aa09-e9c9683556ef` has been stored successfully.
-  Stash Backup Flow +  Snapshots in GCP
Fig: Snapshots in GCP
-## Restore PVC from VolumeSnapshot - -This section will show you how to restore the PVCs from the snapshots we have taken in the previous section. - -**Stop Taking Backup of the Old Deployment:** - -At first, let's stop taking any further backup of the old Deployment so that no backup is taken during the restore process. We are going to pause the `BackupConfiguration` that we created to backup the `stash-demo` Deployment. Then, Stash will stop taking any further backup for this Deployment. You can learn more how to pause a scheduled backup [here](/docs/guides/use-cases/pause-backup/index.md) - -Let's pause the `deployment-volume-snapshot` BackupConfiguration, - -```bash -$ kubectl patch backupconfiguration -n demo deployment-volume-snapshot --type="merge" --patch='{"spec": {"paused": true}}' -backupconfiguration.stash.appscode.com/deployment-volume-snapshot patched -``` - -Now, wait for a moment. Stash will pause the BackupConfiguration. Verify that the BackupConfiguration has been paused, -```bash -$ kubectl get backupconfiguration -n demo -NAME TASK SCHEDULE PAUSED AGE -deployment-volume-snapshot */1 * * * * true 18m -``` +## Restore PVCs from VolumeSnapshots -Notice the `PAUSED` column. Value `true` for this field means that the BackupConfiguration has been paused. +This section will show you how to restore the PVCs from the snapshots we have taken in the previous section. **Create RestoreSession :** -At first, we have to create a `RestoreSession` crd to restore the PVCs from the respective snapshot. - -Below is the YAML of the `RestoreSesion` crd that we are going to create, +Now, we are going to create a `RestoreSession` custom resource to restore all backed-up `PVC` volumes individually from their respective `VolumeSnapshot`. Below is the YAML of the `RestoreSession` object that we are going to create, ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: RestoreSession metadata: - name: restore-pvc + name: restore-deployment-pvc namespace: demo spec: - driver: VolumeSnapshotter - target: - volumeClaimTemplates: - - metadata: - name: restore-data - spec: - accessModes: [ "ReadWriteOnce" ] - storageClassName: "csi-standard" - resources: - requests: - storage: 1Gi - dataSource: - kind: VolumeSnapshot - name: source-data-fnbwz - apiGroup: snapshot.storage.k8s.io - - metadata: - name: restore-config - spec: - accessModes: [ "ReadWriteOnce" ] - storageClassName: "csi-standard" - resources: - requests: - storage: 1Gi - dataSource: - kind: VolumeSnapshot - name: source-config-fnbwz - apiGroup: snapshot.storage.k8s.io + dataSource: + repository: gcs-repository + snapshot: latest + addon: + name: pvc-addon + tasks: + - name: VolumeSnapshotRestore + targetVolumes: + volumeClaimTemplates: + - metadata: + name: restore-data + spec: + accessModes: [ "ReadWriteOnce" ] + storageClassName: "csi-standard" + resources: + requests: + storage: 1Gi + - metadata: + name: restore-config + spec: + accessModes: [ "ReadWriteOnce" ] + storageClassName: "csi-standard" + resources: + requests: + storage: 1Gi ``` Here, -- `spec.target.volumeClaimTemplates`: - - `metadata.name` is a template for the name of the restored PVC that will be created by Stash. You have to provide this named template to match with the desired PVC of your Deployment. - - `spec.dataSource`: `spec.dataSource` specifies the source of the data from where the newly created PVC will be initialized. It requires the following fields to be set: - - `apiGroup` is the group for resource being referenced. Now, Kubernetes supports only `snapshot.storage.k8s.io`. - - `kind` is resource of the kind being referenced. Now, Kubernetes supports only `VolumeSnapshot`. - - `name` is the `VolumeSnapshot` resource name. In `RestoreSession` crd, You must set the VolumeSnapshot name directly. +- `spec.dataSource.repository` specifies the name of the `Repository` from which the data will be restored. +- `spec.dataSource.snapshot` specifies that we want to restore the latest snapshot of the `gcs-repository`. KubeStash retrieves `VolumeSnapshot` information from the `status.components[*]` field of this snapshot. +- `spec.addon.name` specifies the name of the `Addon` object that specifies addon configuration that will be used to perform restore of a stand-alone PVC. +- `spec.addon.tasks[*].name` specifies the name of the `Task` that holds the `Function` and their order of execution to perform restore of a stand-alone PVC. +- `spec.addon.targetVolumes.volumeClaimTemplates[*]`: + - `metadata.name` is a template for the name of the restored `PVC`. KubeStash will create `PVC` with the specified name. + Let's create the `RestoreSession` crd we have shown above. @@ -412,12 +550,13 @@ Once, you have created the `RestoreSession` crd, Stash will create a job to rest Run the following command to watch RestoreSession phase, ```bash -$ watch -n 1 kubectl get -n demo restoresession -n -Every 1.0s: kubectl get restore -n demo +$ watch -n 1 kubectl get -n demo restoresession + +Every 1.0s: kubectl get -n demo restoresession anisur: Wed Jan 17 15:28:39 2024 -NAME REPOSITORY-NAME PHASE AGE -restore-pvc Running 10s -restore-pvc Succeeded 1m +NAME REPOSITORY FAILURE-POLICY PHASE DURATION AGE +restore-deployment-pvc gcs-repository Running 27s 62s +restore-deployment-pvc gcs-repository Succeeded 27s 62s ``` So, we can see from the output of the above command that the restore process succeeded. @@ -431,8 +570,8 @@ Verify that the PVCs have been created by the following command, ```bash $ kubectl get pvc -n demo NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE -restore-config Bound pvc-26758eda-a6ca-11e9-9f3a-42010a800050 1Gi RWO standard 30s -restore-data Bound pvc-267335ff-a6ca-11e9-9f3a-42010a800050 1Gi RWO standard 30s +restore-config Bound pvc-f52515f0-f8a2-4939-8d38-fd625725ce7c 1Gi RWO csi-standard 104s +restore-data Bound pvc-b45ef4e3-bed3-4305-aabf-580300fc3db8 1Gi RWO csi-standard 104s ``` Notice the `STATUS` field. It indicates that the respective PV has been provisioned and initialized from the respective VolumeSnapshot by CSI driver and the PVC has been bound with the PV. @@ -499,16 +638,16 @@ Now, wait for the pod of the Deployment to go into the `Running` state. ```bash $ kubectl get pod -n demo -NAME READY STATUS RESTARTS AGE -restore-demo-544db78b8b-tnzb2 1/1 Running 0 34s +NAME READY STATUS RESTARTS AGE +restore-demo-8bf47f8cb-4flpf 1/1 Running 0 30s ``` Verify that the backed up data has been restored in `/restore/data` and `/restore/config` directory using the following command, ```bash -$ kubectl exec -n demo restore-demo-544db78b8b-tnzb2 ls /restore/config/config.cfg +$ kubectl exec -n demo restore-demo-8bf47f8cb-4flpf -- cat /restore/data/config.txt config_data -$ kubectl exec -n demo restore-demo-544db78b8b-tnzb2 ls /restore/data/data.txt +$ kubectl exec -n demo restore-demo-8bf47f8cb-4flpf -- cat /restore/data/data.txt sample_data ``` @@ -517,10 +656,14 @@ sample_data To clean up the Kubernetes resources created by this tutorial, run: ```bash -kubectl delete -n demo deployment stash-demo +kubectl delete -n demo deployment kubestash-demo kubectl delete -n demo deployment restore-demo +kubectl delete -n demo restoresession restore-deployment-pvc kubectl delete -n demo backupconfiguration deployment-volume-snapshot -kubectl delete -n demo restoresession restore-pvc +kubectl delete -n demo retentionPolicy demo-retention +kubectl delete -n demo backupstorage gcs-storage +kubectl delete -n demo pvc --all +kubectl delete -n demo volumesnapshot --all kubectl delete -n demo storageclass csi-standard kubectl delete -n demo volumesnapshotclass csi-snapshot-class ``` diff --git a/docs/guides/volumesnapshot/overview/images/restore-overview.svg b/docs/guides/volumesnapshot/overview/images/restore-overview.svg index 28b2f09..05b90e5 100644 --- a/docs/guides/volumesnapshot/overview/images/restore-overview.svg +++ b/docs/guides/volumesnapshot/overview/images/restore-overview.svg @@ -1,106 +1 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + \ No newline at end of file diff --git a/docs/guides/volumesnapshot/overview/images/volumesnapshot-overview.svg b/docs/guides/volumesnapshot/overview/images/volumesnapshot-overview.svg index a859c04..b1a9a73 100644 --- a/docs/guides/volumesnapshot/overview/images/volumesnapshot-overview.svg +++ b/docs/guides/volumesnapshot/overview/images/volumesnapshot-overview.svg @@ -1,111 +1 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - + \ No newline at end of file diff --git a/docs/guides/volumesnapshot/overview/index.md b/docs/guides/volumesnapshot/overview/index.md index 40aef95..a604f79 100644 --- a/docs/guides/volumesnapshot/overview/index.md +++ b/docs/guides/volumesnapshot/overview/index.md @@ -1,6 +1,6 @@ --- -title: VolumeSnapshot Overview | Stash -description: An overview of how VolumeSnapshot works in Stash +title: VolumeSnapshot Overview | KubeStash +description: An overview of how VolumeSnapshot works in KubeStash menu: docs_{{ .version }}: identifier: volume-snapshot-overview @@ -12,73 +12,63 @@ menu_name: docs_{{ .version }} section_menu_id: guides --- -# VolumeSnapshot Using Stash +# VolumeSnapshot Using KubeStash -This guide will give you an overview of how VolumeSnapshot process works in Stash. +This guide will give you an overview of how VolumeSnapshot process works in KubeStash. ## How Backup Process Works? -The following diagram shows how Stash creates VolumeSnapshot via Kubernetes native API. Open the image in a new tab to see the enlarged version. +The following diagram shows how KubeStash creates VolumeSnapshot via Kubernetes native API. Open the image in a new tab to see the enlarged version.
-  Stash Backup Flow -
Fig: Volume Snapshotting Process in Stash
+  KubeStash Backup Flow +
Fig: Volume Snapshotting Process in KubeStash
The `VolumeSnapshot` process consists of the following steps: - -1. At first, a user creates a `BackupConfiguration` crd which specifies the targeted workload or targeted PVC. - -2. Stash operator watches for `BackupConfiguration` crd. - -3. When it finds a `BackupConfiguration` crd, it creates a `CronJob` to take a periodic backup of the target volumes. - -4. The `CronJob` triggers backup on each scheduled time slot by creating a `BackupSession` crd. - -5. Stash operator watches for `BackupSession` crd. - -6. When it finds a `BackupSession` crd, it creates a volume snapshotter `Job` to take snapshot of the targeted volumes. - -7. The volume snapshotter `Job` creates `VolumeSnapshot` crd for each PVC of the target and waits for the CSI driver to complete snapshotting. These `VolumeSnasphot` crd names follow the following format: +1. At first, a user creates a `Secret` with access credentials of the backend where the backed-up data will be stored. +2. Then, she creates a `BackupStorage` custom resource that specifies the backend information, along with the `Secret` containing the credentials needed to access the backend. +3. KubeStash operator watches for `BackupStorage` custom resources. When it finds a `BackupStorage` object, it initializes the `BackupStorage` by uploading the `metadata.yaml` file into the target storage. +4. Next, she creates a `BackupConfiguration` custom resource, targeting either a workload application or a standalone volume. The `BackupConfiguration` object specifies the `Repository` pointing to a `BackupStorage` that contains backend information, indicating where to upload backup data. It also defines the `Addon` information with a specified `Task` to be used for backing up the volume. +5. KubeStash operator watches for `BackupConfiguration` custom resources. +6. Once the KubeStash operator finds a `BackupConfiguration` object, it creates `Repository` with the information specified in the BackupConfiguration. +7. KubeStash operator watches for `Repository` custom resources. When it finds the `Repository` object, it Initializes `Repository` by uploading `repository.yaml` file into the `spec.sessions[*].repositories[*].directory` path specified in `BackupConfiguration`. +8. Then, it creates a `CronJob` with the schedule specified in `BackupConfiguration` to trigger backup periodically. +9. On the next scheduled slot, the `CronJob` triggers a backup by creating a `BackupSession` custom resource. +10. KubeStash operator watches for `BackupSession` custom resources. +11. When it finds a `BackupSession` object, it creates a `Snapshot` custom resource for each `Repository` specified in the `BackupConfiguration`. +12. Then it resolves the respective `Addon` and `Function` and prepares a volume snapshotter `Job`. +13. It then creates a volume snapshotter `Job` to capture snapshots of the targeted volumes. +14. The volume snapshotter `Job` creates a `VolumeSnapshot` object for each Persistent Volume Claim (PVC) of the target and waits for the Container Storage Interface (CSI) driver to complete snapshotting. + +These `VolumeSnasphot` custom resources names follow the following format: ```bash - ``` - -8. CSI `external-snapshotter` controller watches for `VolumeSnapshot`. - -9. When it finds a `VolumeSnapshot` object, it backups `VolumeSnapshot` in the respective cloud storage. - -10. Once the snapsotting is completed, Stash Operator updates the `status.phase` field of the `BackupSession` crd. +15. Container Storage Interface (CSI) `external-snapshotter` controller watches for `VolumeSnapshot` resources. +16. When it finds a `VolumeSnapshot` object, it backups `VolumeSnapshot` into the respective cloud storage. +17. After the snapshotting process is completed, the volume snapshotter `Job` updates the `status.components[*]` field of the `Snapshot` resources with each target `PVC` and the corresponding created `VolumeSnapshot` information. It also updates the `status.phase` field of the `BackupSession` to reflect backup completion. ## How Restore Process Works? - -The following diagram shows how Stash restores PersistentVolumeClaims from snapshot using Kubernetes VolumeSnapshot API. Open the image in a new tab to see the enlarged version. +The following diagram shows how KubeStash restores PersistentVolumeClaims from snapshot using Kubernetes VolumeSnapshot API. Open the image in a new tab to see the enlarged version.
-  Stash Backup Flow -
Fig: Restore process from snapshot in Stash
+  KubeStash Restore Flow +
Fig: Restore process from snapshot in KubeStash
The restore process consists of the following steps: - -1. At first, a user creates a `RestoreSession` crd which specifies the `volumeClaimTemplates`. `VolumeClaimTemplates` hold the information about `VolumeSnapshot`. - -2. Stash operator watches for `RestoreSession` crd. - -3. When it finds a `RestoreSession` crd, it creates a Restore `Job`to restore PVC from the snapshot. - -4. The restore `Job` creates PVC with `spec.dataSource` field set to the respective VolumeSnapshot name. - -5. CSI `external-snapshotter` controller watches for PVC. - -6. When it finds a new PVC with `spec.dataSource` field set, it reads the information about the `VolumeSnapshot`. - -7. The controller downloads the respective data from the cloud and populate the PVC with it. - -8. Once restore process is completed, the Stash operator updates the `status.phase` field of the `BackupSession` crd. +1. At first, a user creates a `RestoreSession` custom resource, specifying the `volumeClaimTemplates`, the `Repository` object that points to a `BackupStorage` that holds backend information, and the target `Snapshot`, which holds information about the `VolumeSnapshots` created during backup. It also specifies the `Addon` info with `Task` to use to restore the volume. +2. KubeStash operator watches for `RestoreSession` custom resources. +3. Once it finds a `RestoreSession` object, it resolves the respective `Addon` and `Function` and prepares a restore Job definition. +4. Then, it creates a Restore `Job` to restore `PVC` from the `volumeSnapshot`. +5. The restore `Job` gets VolumeSnapshot's information from Snapshot `status.phase---` and creates `PVCs` with `spec.dataSourceRef` field set to the respective VolumeSnapshot name. +6. Container Storage Interface (CSI) `external-snapshotter` controller watches for `PVCs`. +7. When it finds a new `PVC` with `spec.dataSource` field set, it reads the information about the `VolumeSnapshot`. +8. The controller downloads the respective data from the cloud and populates the `PVC`. +9. The restore `Job` waits until the `Status.Phase` of created `PVCs` goes to `Bound`. Once completed, the job updates the `status.phase` field of the `Restoresession` to reflect restore completion. ## Next Steps - 1. See a step by step guide to snapshot the volumes of a Deployment [here](/docs/guides/volumesnapshot/deployment/index.md). 2. See a step by step guide to snapshot the volumes of a StatefulSet [here](/docs/guides/volumesnapshot/statefulset/index.md). 3. See a step by step guide to snapshot a stand-alone PVC [here](/docs/guides/volumesnapshot/pvc/index.md). diff --git a/docs/guides/volumesnapshot/pvc/examples/backupconfiguration.yaml b/docs/guides/volumesnapshot/pvc/examples/backupconfiguration.yaml index 27eca4a..fdd6fd6 100644 --- a/docs/guides/volumesnapshot/pvc/examples/backupconfiguration.yaml +++ b/docs/guides/volumesnapshot/pvc/examples/backupconfiguration.yaml @@ -1,18 +1,37 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: name: pvc-volume-snapshot namespace: demo spec: - schedule: "*/5 * * * *" - driver: VolumeSnapshotter target: - ref: - apiVersion: v1 - kind: PersistentVolumeClaim - name: source-data - snapshotClassName: csi-snapshot-class - retentionPolicy: - name: "keep-last-5" - keepLast: 5 - prune: true + apiGroup: + kind: PersistentVolumeClaim + name: source-pvc + namespace: demo + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /pvc-volume-snapshot-repo + deletionPolicy: WipeOut + addon: + name: pvc-addon + tasks: + - name: volume-snapshot + params: + volumeSnapshotClassName: csi-snapshot-class \ No newline at end of file diff --git a/docs/guides/volumesnapshot/pvc/examples/backupstorage.yaml b/docs/guides/volumesnapshot/pvc/examples/backupstorage.yaml new file mode 100644 index 0000000..0461b26 --- /dev/null +++ b/docs/guides/volumesnapshot/pvc/examples/backupstorage.yaml @@ -0,0 +1,17 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/volumesnapshot/pvc/examples/restored-pod.yaml b/docs/guides/volumesnapshot/pvc/examples/restored-pod.yaml index 4fee01c..09e043e 100644 --- a/docs/guides/volumesnapshot/pvc/examples/restored-pod.yaml +++ b/docs/guides/volumesnapshot/pvc/examples/restored-pod.yaml @@ -16,5 +16,5 @@ spec: volumes: - name: restore-data persistentVolumeClaim: - claimName: restore-data + claimName: restore-source-pvc readOnly: false diff --git a/docs/guides/volumesnapshot/pvc/examples/restoresession.yaml b/docs/guides/volumesnapshot/pvc/examples/restoresession.yaml index 7bab6ff..4de837c 100644 --- a/docs/guides/volumesnapshot/pvc/examples/restoresession.yaml +++ b/docs/guides/volumesnapshot/pvc/examples/restoresession.yaml @@ -1,21 +1,23 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: RestoreSession metadata: name: restore-pvc namespace: demo spec: - driver: VolumeSnapshotter - target: - volumeClaimTemplates: - - metadata: - name: restore-data - spec: - accessModes: ["ReadWriteOnce"] - storageClassName: "csi-standard" - resources: - requests: - storage: 1Gi - dataSource: - kind: VolumeSnapshot - name: source-data-fnbwz - apiGroup: snapshot.storage.k8s.io + dataSource: + repository: gcs-repository + snapshot: latest + addon: + name: pvc-addon + tasks: + - name: VolumeSnapshotRestore + targetVolumes: + volumeClaimTemplates: + - metadata: + name: restore-source-pvc + spec: + accessModes: [ "ReadWriteOnce" ] + storageClassName: "csi-standard" + resources: + requests: + storage: 1Gi \ No newline at end of file diff --git a/docs/guides/volumesnapshot/pvc/examples/retentionpolicy.yaml b/docs/guides/volumesnapshot/pvc/examples/retentionpolicy.yaml new file mode 100644 index 0000000..3af9037 --- /dev/null +++ b/docs/guides/volumesnapshot/pvc/examples/retentionpolicy.yaml @@ -0,0 +1,15 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same \ No newline at end of file diff --git a/docs/guides/volumesnapshot/pvc/examples/source-pod.yaml b/docs/guides/volumesnapshot/pvc/examples/source-pod.yaml index fcf7aa7..6b0d77e 100644 --- a/docs/guides/volumesnapshot/pvc/examples/source-pod.yaml +++ b/docs/guides/volumesnapshot/pvc/examples/source-pod.yaml @@ -5,15 +5,15 @@ metadata: namespace: demo spec: containers: - - name: busybox - image: busybox - command: ["/bin/sh", "-c"] - args: ["echo sample_data > /source/data/data.txt && sleep 3000"] - volumeMounts: - - name: source-data - mountPath: /source/data + - name: busybox + image: busybox + command: ["/bin/sh", "-c"] + args: ["echo sample_data > /source/data/data.txt && sleep 3000"] + volumeMounts: + - name: source-data + mountPath: /source/data volumes: - - name: source-data - persistentVolumeClaim: - claimName: source-data - readOnly: false + - name: source-data + persistentVolumeClaim: + claimName: source-pvc + readOnly: false \ No newline at end of file diff --git a/docs/guides/volumesnapshot/pvc/examples/source-pvc.yaml b/docs/guides/volumesnapshot/pvc/examples/source-pvc.yaml index 7af855c..1e78132 100644 --- a/docs/guides/volumesnapshot/pvc/examples/source-pvc.yaml +++ b/docs/guides/volumesnapshot/pvc/examples/source-pvc.yaml @@ -1,12 +1,12 @@ kind: PersistentVolumeClaim apiVersion: v1 metadata: - name: source-data + name: source-pvc namespace: demo spec: accessModes: - - ReadWriteOnce - storageClassName: standard + - ReadWriteOnce + storageClassName: csi-standard resources: requests: - storage: 1Gi + storage: 1Gi \ No newline at end of file diff --git a/docs/guides/volumesnapshot/pvc/examples/storageclass.yaml b/docs/guides/volumesnapshot/pvc/examples/storageclass.yaml index e1fa0f0..62655df 100644 --- a/docs/guides/volumesnapshot/pvc/examples/storageclass.yaml +++ b/docs/guides/volumesnapshot/pvc/examples/storageclass.yaml @@ -6,4 +6,4 @@ parameters: type: pd-standard provisioner: pd.csi.storage.gke.io reclaimPolicy: Delete -volumeBindingMode: Immediate +volumeBindingMode: Immediate \ No newline at end of file diff --git a/docs/guides/volumesnapshot/pvc/examples/volumesnapshot.yaml b/docs/guides/volumesnapshot/pvc/examples/volumesnapshot.yaml new file mode 100644 index 0000000..e2d4b23 --- /dev/null +++ b/docs/guides/volumesnapshot/pvc/examples/volumesnapshot.yaml @@ -0,0 +1,21 @@ +apiVersion: snapshot.storage.k8s.io/v1 +kind: VolumeSnapshot +metadata: + creationTimestamp: "2024-01-16T11:00:08Z" + finalizers: + - snapshot.storage.kubernetes.io/volumesnapshot-as-source-protection + - snapshot.storage.kubernetes.io/volumesnapshot-bound-protection + generation: 1 + name: source-pvc-1705402801 + namespace: demo + resourceVersion: "11593" + uid: b5692a5b-8834-48dc-9185-56ed9a2fa124 +spec: + source: + persistentVolumeClaimName: source-pvc + volumeSnapshotClassName: csi-snapshot-class +status: + boundVolumeSnapshotContentName: snapcontent-b5692a5b-8834-48dc-9185-56ed9a2fa124 + creationTime: "2024-01-16T11:00:10Z" + readyToUse: true + restoreSize: 1Gi \ No newline at end of file diff --git a/docs/guides/volumesnapshot/pvc/examples/volumesnapshotclass.yaml b/docs/guides/volumesnapshot/pvc/examples/volumesnapshotclass.yaml index 382ba7d..3bf5d3b 100644 --- a/docs/guides/volumesnapshot/pvc/examples/volumesnapshotclass.yaml +++ b/docs/guides/volumesnapshot/pvc/examples/volumesnapshotclass.yaml @@ -3,4 +3,4 @@ kind: VolumeSnapshotClass metadata: name: csi-snapshot-class driver: pd.csi.storage.gke.io -deletionPolicy: Delete +deletionPolicy: Delete \ No newline at end of file diff --git a/docs/guides/volumesnapshot/pvc/images/gcp.png b/docs/guides/volumesnapshot/pvc/images/gcp.png index 2b65e54..182b12a 100644 Binary files a/docs/guides/volumesnapshot/pvc/images/gcp.png and b/docs/guides/volumesnapshot/pvc/images/gcp.png differ diff --git a/docs/guides/volumesnapshot/pvc/index.md b/docs/guides/volumesnapshot/pvc/index.md index e2cddf2..f53d5cd 100644 --- a/docs/guides/volumesnapshot/pvc/index.md +++ b/docs/guides/volumesnapshot/pvc/index.md @@ -1,5 +1,5 @@ --- -title: Snapshot Stand-alone PVC | Stash +title: Snapshot Stand-alone PVC | KubeStash description: An step by step guide showing how to snapshot a stand-alone PVC menu: docs_{{ .version }}: @@ -14,13 +14,13 @@ section_menu_id: guides # Snapshotting a Standalone PVC -This guide will show you how to use Stash to snapshot standalone PersistentVolumeClaims and restore it from the snapshot using Kubernetes [VolumeSnapshot](https://kubernetes.io/docs/concepts/storage/volume-snapshots/) API. In this guide, we are going to backup the volumes in Google Cloud Platform with the help of [GCE Persistent Disk CSI Driver](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver). +This guide will show you how to use KubeStash to take volumeSnapshot of a standalone PersistentVolumeClaims and restore it from the volumeSnapshot using Kubernetes [VolumeSnapshot](https://kubernetes.io/docs/concepts/storage/volume-snapshots/) API. In this guide, we are going to backup the volumes in Google Cloud Platform with the help of [GCE Persistent Disk CSI Driver](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver). ## Before You Begin - At first, you need to be familiar with the [GCE Persistent Disk CSI Driver](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver). -- Install `Stash` in your cluster following the steps [here](/docs/setup/README.md). -- If you don't know how VolumeSnapshot works in Stash, please visit [here](/docs/guides/volumesnapshot/overview/index.md). +- Install `KubeStash` in your cluster following the steps [here](/docs/setup/README.md). +- If you don't know how VolumeSnapshot works in KubeStash, please visit [here](/docs/guides/volumesnapshot/overview/index.md). ## Prepare for VolumeSnapshot @@ -81,9 +81,9 @@ namespace/demo created ## Take Volume Snapshot -Here, we are going to create a PVC and mount it with a pod and we are going to also generate some sample data on it. Then, we are going to take snapshot of this PVC using Stash. +Here, we are going to create a PVC and mount it with a pod and we are going to also generate some sample data on it. Then, we are going to take volumeSnapshot of this PVC using KubeStash. -**Create PersistentVolumeClaim :** +**Create PersistentVolumeClaim:** At first, let's create a PVC. We are going to mount this PVC in a pod. @@ -93,12 +93,12 @@ Below is the YAML of the sample PVC, kind: PersistentVolumeClaim apiVersion: v1 metadata: - name: source-data + name: source-pvc namespace: demo spec: accessModes: - ReadWriteOnce - storageClassName: standard + storageClassName: csi-standard resources: requests: storage: 1Gi @@ -108,7 +108,7 @@ Let's create the PVC we have shown above. ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/pvc/examples/source-pvc.yaml -persistentvolumeclaim/source-data created +persistentvolumeclaim/source-pvc created ``` **Create Pod :** @@ -135,7 +135,7 @@ spec: volumes: - name: source-data persistentVolumeClaim: - claimName: source-data + claimName: source-pvc readOnly: false ``` @@ -151,7 +151,7 @@ Now, wait for the Pod to go into the `Running` state. ```bash $ kubectl get pod -n demo NAME READY STATUS RESTARTS AGE -source-pod 1/1 Running 0 25s +source-pod 1/1 Running 0 11s ``` Verify that the sample data has been created in `/source/data` directory for `source-pod` pod @@ -162,234 +162,379 @@ $ kubectl exec -n demo source-pod -- cat /source/data/data.txt sample_data ``` +## Backup + +Now, we are going to taking `VolumeSnapshot` of the PVC `source-pvc` using KubeStash. We have to create a `Secret` and a `BackupStorage` object with access credentials and backend information respectively. + +> For GCS backend, if the bucket does not exist, KubeStash needs `Storage Object Admin` role permissions to create the bucket. For more details, please check the following [guide](/docs/guides/backends/gcs/index.md). + +**Create Storage Secret:** + +Let's create a Secret named `gcs-secret` with access credentials of our desired GCS backend, + +```bash +$ echo -n '' > GOOGLE_PROJECT_ID +$ cat /path/to/downloaded/sa_key_file.json > GOOGLE_SERVICE_ACCOUNT_JSON_KEY +$ kubectl create secret generic -n demo gcs-secret \ + --from-file=./GOOGLE_PROJECT_ID \ + --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY +secret/gcs-secret created +``` + +**Create BackupStorage:** + +Now, create a `BackupStorage` custom resource specifying the desired bucket, and directory inside the bucket where the backed up data will be stored. + +Below is the YAML of `BackupStorage` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut +``` + +Let's create the `BackupStorage` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumes/pvc/examples/backupstorage.yaml +backupstorage.storage.kubestash.com/gcs-storage created +``` + +Now, we have to create a `RetentionPolicy` custom resource which specifies how the old `Snapshots` should be cleaned up. + +**Create RetentionPolicy:** + +Below is the YAML of the `RetentionPolicy` object that we are going to create, + +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same +``` + +Let's create the `RetentionPolicy` object that we have shown above, + +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumes/pvc/examples/retentionpolicy.yaml +retentionpolicy.storage.kubestash.com/demo-retention created +``` + +We are ready. Now, we have to create a `BackupConfiguration` custom resource targeting PVC `source-pvc`. + **Create BackupConfiguration :** -Now, create a `BackupConfiguration` crd to take snapshot of the `source-data` PVC. +Now, create a `BackupConfiguration` custom resource to take `volumeSnapshot` of the `source-pvc` PVC. -Below is the YAML of the `BackupConfiguration` crd that we are going to create, +Below is the YAML of the `BackupConfiguration` object that we are going to create, ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: name: pvc-volume-snapshot namespace: demo spec: - schedule: "*/5 * * * *" - driver: VolumeSnapshotter target: - ref: - apiVersion: v1 - kind: PersistentVolumeClaim - name: source-data - snapshotClassName: csi-snapshot-class - retentionPolicy: - name: 'keep-last-5' - keepLast: 5 - prune: true + apiGroup: + kind: PersistentVolumeClaim + name: source-pvc + namespace: demo + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /pvc-volume-snapshot-repo + deletionPolicy: WipeOut + addon: + name: pvc-addon + tasks: + - name: volume-snapshot + params: + volumeSnapshotClassName: csi-snapshot-class ``` - Here, +- `spec.target` specifies the backup target. `apiGroup`, `kind`, `namespace` and `name` refers to the `apiGroup`, `kind`, `namespace` and `name` of the targeted workload respectively. +- `spec.sessions[*].scheduler.schedule` specifies a [cron expression](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) indicates that `BackupSession` will be created at 5 minute interval. +- `spec.sessions[*].repositories[*].name` specifies the name of the `Repository` object that holds the backend information. +- `spec.sessions[*].repositories[*].backend` specifies the name of the backend that holds the `BackupStorage` information. +- `spec.sessions[*].repositories[*].directory` specifies the path of the `Repository` where the backed up data will be stored. +- `spec.sessions[*].addon.name` specifies the name of the `Addon` object that specifies addon configuration that will be used to perform backup of a stand-alone PVC. +- `spec.sessions[*].addon.tasks[*].name` specifies the name of the `Task` that holds the `Function` and their order of execution to perform backup of a stand-alone PVC. +- `spec.sessions[*].addon.tasks[*].params.volumeSnapshotClassName` indicates the [VolumeSnapshotClass](https://kubernetes.io/docs/concepts/storage/volume-snapshot-classes/) to be used for volume snapshotting. If we don't provide any then kubestash use default `volumeSnapshotClass` for volume snapshotting. -- `spec.schedule` is a [cron expression](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) indicates that `BackupSession` will be created at 5 minute interval. +Let's create the `BackupConfiguration` crd we have shown above. -- `spec.driver` indicates the name of the agent to use to back up the target. Currently, Stash supports `Restic`, `VolumeSnapshotter` drivers. The `VolumeSnapshotter` is used to backup/restore PVC using `VolumeSnapshot` API. +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/pvc/examples/backupconfiguration.yaml +backupconfiguration.core.kubestash.com/pvc-volume-snapshot created +``` -- `spec.target.ref` refers to the backup target. `apiVersion`, `kind` and `name` refers to the `apiVersion`, `kind` and `name` of the targeted workload respectively. Stash will use this information to create a Volume Snapshotter Job for creating VolumeSnapshot. +**Verify Backup Setup Successful** -- `spec.target.snapshotClassName` indicates the [VolumeSnapshotClass](https://kubernetes.io/docs/concepts/storage/volume-snapshot-classes/) to be used for volume snapshotting. +If everything goes well, the phase of the `BackupConfiguration` should be in `Ready` state. The `Ready` phase indicates that the backup setup is successful. Let's check the `Phase` of the BackupConfiguration, -Let's create the `BackupConfiguration` crd we have shown above. +```bash +$ kubectl get backupconfiguration -n demo +NAME PHASE PAUSED AGE +pvc-volume-snapshot Ready 17s +``` +**Verify Repository:** + +Verify that KubeStash has created `Repositories` that holds the `BackupStorage` information by the following command, ```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/pvc/examples/backupconfiguration.yaml -backupconfiguration.stash.appscode.com/pvc-volume-snapshot created +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository Ready 35s ``` -**Verify CronJob :** +**Verify CronJob:** -If everything goes well, Stash will create a `CronJob` to take periodic snapshot of the PVC with the schedule specified in `spec.schedule` field of `BackupConfiguration` crd. +Verify that KubeStash has created a CronJob to trigger a periodic backup of the targeted PVC by the following command, - Check that the `CronJob` has been created using the following command, +```bash +$ kubectl get cronjob -n demo +NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE +trigger-pvc-volume-snapshot-frequent-backup */5 * * * * False 0 50s +``` - ```bash - $ kubectl get cronjob -n demo -NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE -pvc-volume-snapshot */1 * * * * False 0 39s 2m41s - ``` +**Wait for BackupSession:** -**Wait for BackupSession :** +Now, wait for the next backup schedule. You can watch for `BackupSession` object using the following command, + +```bash +$ watch -n 1 kubectl get backupsession -n demo -l=kubestash.com/invoker-name=pvc-volume-snapshot + +Every 1.0s: kubectl get backupsession -n demo -l=kubestash.com/invoker-name=pvc-volume-snapshot anisur: Tue Jan 16 17:01:06 2024 + +NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE +pvc-volume-snapshot-frequent-backup-1705402801 BackupConfiguration pvc-volume-snapshot Succeeded 66s +``` -The `pvc-volume-snapshot` CronJob will trigger a backup on each scheduled time slot by creating a `BackupSession` crd. +**Verify Snapshot:** -Wait for the next schedule of backup. Run the following command to watch `BackupSession` crd, +Once a `BackupSession` object is created, it creates a `Snapshot` custom resource for each `Repository`. This snapshot `spec.status.components` section represents the backup information of the targeted backup components. + +Verify created `Snapshot` object by the following command, ```bash -$ watch -n 1 kubectl get backupsession -n demo -Every 1.0s: kubectl get backupsession -n demo +$ watch kubectl get snapshots.storage.kubestash.com -n demo -l=kubestash.com/repo-name=gcs-repository + +Every 2.0s: kubectl get snapshots.storage.kubestash.com -n demo -l=kubestash.com/repo-name=gcs-repository anisur: Tue Jan 16 17:03:06 2024 -NAME INVOKER-TYPE INVOKER-NAME PHASE AGE -pvc-volume-snapshot-fnbwz BackupConfiguration pvc-volume-snapshot Succeeded 1m32s +NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE VERIFICATION-STATUS AGE +gcs-repository-pvc-volume-snapshot-frequent-backup-1705402801 gcs-repository frequent-backup 2024-01-16T11:00:08Z Delete Succeeded 3m6s ``` -We can see above that the backup session has succeeded. Now, we are going to verify that the `VolumeSnapshot` has been created and the snapshots have been stored in the respective backend. +Now, retrieve the YAML representation of the above created `Snapshot`, and inspect the `spec.status` section to see the backup information of the targeted backup components. + +```bash +$ kubectl get snapshots -n demo gcs-repository-pvc-volume-snapshot-frequent-backup-1705402801 -o yaml +``` +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: Snapshot +metadata: + name: gcs-repository-pvc-volume-snapshot-frequent-backup-1705402801 + namespace: demo +spec: + --- +status: + components: + volumesnapshot: + driver: VolumeSnapshotter + phase: Succeeded + volumeSnapshotterStats: + - pvcName: source-pvc + volumeSnapshotName: source-pvc-1705402801 + conditions: + --- + phase: Succeeded + snapshotTime: "2024-01-16T11:00:08Z" + totalComponents: 1 +``` +Here, +- `spec.status.components` specifies the backup information of the backup target components. +- `spec.status.components.[].volumeSnapshotterStats` specifies the information about the `VolumeSnapshotter` driver. In the case of a standalone `PVC`, the component name is designated as `volumesnapshot`. + - `volumeSnapshotterStats.hostPath` specifies the corresponding path of `PVC` volume for which `volumeSnapshot` has created. + - `volumeSnapshotterStats.pvcName` specifies the name of the backed-up target `PVC` volume. + - `volumeSnapshotterStats.volumeSnapshotName` specifies the created `VolumeSnapshot` object name corresponding to the target `PVC` volume. **Verify Volume Snapshot :** -Once a `BackupSession` crd is created, it creates volume snapshotter `Job`. Then the `Job` creates `VolumeSnapshot` crd for the targeted PVC. +Once a `BackupSession` object is created, it creates volume snapshotter `Job`. Then the `Job` creates `VolumeSnapshot` object for the targeted PVC. We can see above that the backup session has succeeded. Now, we are going to verify that the `VolumeSnapshot` has been created and the snapshot have been stored in the respective backend. Check that the `VolumeSnapshot` has been created Successfully. ```bash $ kubectl get volumesnapshot -n demo -NAME AGE -source-data-fnbwz 1m30s +NAME READYTOUSE SOURCEPVC SOURCESNAPSHOTCONTENT RESTORESIZE SNAPSHOTCLASS SNAPSHOTCONTENT CREATIONTIME AGE +source-pvc-1705402801 true source-pvc 1Gi csi-snapshot-class snapcontent-b5692a5b-8834-48dc-9185-56ed9a2fa124 7m20s 7m22s ``` -Let's find out the actual snapshot name that will be saved in the Google Cloud by the following command, +Let's find out the `VolumeSnapshotContent` that has been saved in the Google Cloud by the following command, ```bash -kubectl get volumesnapshot source-data-fnbwz -n demo -o yaml +kubectl get volumesnapshot source-pvc-fnbwz -n demo -o yaml ``` ```yaml apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshot metadata: - creationTimestamp: "2019-07-15T10:31:09Z" + creationTimestamp: "2024-01-16T11:00:08Z" finalizers: - - snapshot.storage.kubernetes.io/volumesnapshot-protection - generation: 4 - name: source-data-fnbwz + - snapshot.storage.kubernetes.io/volumesnapshot-as-source-protection + - snapshot.storage.kubernetes.io/volumesnapshot-bound-protection + generation: 1 + name: source-pvc-1705402801 namespace: demo - resourceVersion: "32098" - selfLink: /apis/snapshot.storage.k8s.io/v1/namespaces/demo/volumesnapshots/source-data-fnbwz - uid: a8e8faeb-a6eb-11e9-9f3a-42010a800050 + resourceVersion: "11593" + uid: b5692a5b-8834-48dc-9185-56ed9a2fa124 spec: source: - persistentVolumeClaimName: source-data + persistentVolumeClaimName: source-pvc volumeSnapshotClassName: csi-snapshot-class status: - boundVolumeSnapshotContentName: snapcontent-a8e8faeb-a6eb-11e9-9f3a-42010a800050 - creationTime: "2019-07-15T10:31:10Z" + boundVolumeSnapshotContentName: snapcontent-b5692a5b-8834-48dc-9185-56ed9a2fa124 + creationTime: "2024-01-16T11:00:10Z" readyToUse: true restoreSize: 1Gi ``` -Here, `spec.snapshotContentName` field specifies the name of the `VolumeSnapshotContent` crd. It also represents the actual snapshot name that has been saved in Google Cloud. If we navigate to the `Snapshots` tab in the GCP console, we are going to see snapshot `snapcontent-a8e8faeb-a6eb-11e9-9f3a-42010a800050` has been stored successfully. +Here, `spec.status.boundVolumeSnapshotContentName` field specifies the name of the `VolumeSnapshotContent` object. It also represents the actual snapshot name that has been saved in Google Cloud. If we navigate to the `Snapshots` tab in the GCP console, we are going to see snapshot `snapcontent-1fa0a06c-80bb-4e7c-b584-579ddceb649d` has been stored successfully.
-  Stash Backup Flow +  Snapshots in GCP console
Fig: Snapshots in GCP
## Restore PVC from VolumeSnapshot -This section will show you how to restore the PVC from the snapshot we have taken in the earlier section. - -**Stop Taking Backup of the Old PVC:** - -At first, let's stop taking any further backup of the old PVC so that no backup is taken during the restore process. We are going to pause the `BackupConfiguration` that we created to backup the `source-data` PVC. Then, Stash will stop taking any further backup for this PVC. You can learn more how to pause a scheduled backup [here](/docs/guides/use-cases/pause-backup/index.md) - -Let's pause the `pvc-volume-snapshot` BackupConfiguration, - -```bash -$ kubectl patch backupconfiguration -n demo pvc-volume-snapshot --type="merge" --patch='{"spec": {"paused": true}}' -backupconfiguration.stash.appscode.com/pvc-volume-snapshot patched -``` - -Now, wait for a moment. Stash will pause the BackupConfiguration. Verify that the BackupConfiguration has been paused, - -```bash -$ kubectl get backupconfiguration -n demo -NAME TASK SCHEDULE PAUSED AGE -pvc-volume-snapshot */1 * * * * true 22m -``` - -Notice the `PAUSED` column. Value `true` for this field means that the BackupConfiguration has been paused. +This section will show you how to restore the PVC from the `VolumeSnapshot` we have taken in the earlier section. **Create RestoreSession :** -At first, we have to create a `RestoreSession` crd to restore the PVC from respective snapshot. - -Below is the YAML of the RestoreSesion crd that we are going to create, +Now, we are going to create a `RestoreSession` object to restore the `PVC` from respective `VolumeSnapshot`. Below is the YAML of the `RestoreSession` object that we are going to create, ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: RestoreSession metadata: name: restore-pvc namespace: demo spec: - driver: VolumeSnapshotter - target: - volumeClaimTemplates: - - metadata: - name: restore-data - spec: - accessModes: [ "ReadWriteOnce" ] - storageClassName: "csi-standard" - resources: - requests: - storage: 1Gi - dataSource: - kind: VolumeSnapshot - name: source-data-fnbwz - apiGroup: snapshot.storage.k8s.io + dataSource: + repository: gcs-repository + snapshot: latest + addon: + name: pvc-addon + tasks: + - name: VolumeSnapshotRestore + targetVolumes: + volumeClaimTemplates: + - metadata: + name: restore-source-pvc + spec: + accessModes: [ "ReadWriteOnce" ] + storageClassName: "csi-standard" + resources: + requests: + storage: 1Gi ``` Here, -- `spec.target.volumeClaimTemplates`: - - `metadata.name` is the name of the restored `PVC` or prefix of the `VolumeSnapshot` name. - - `spec.dataSource`: `spec.dataSource` specifies the source of the data from where the newly created PVC will be initialized. It requires following fields to be set: - - `apiGroup` is the group for resource being referenced. Now, Kubernetes supports only `snapshot.storage.k8s.io`. - - `kind` is resource of the kind being referenced. Now, Kubernetes supports only `VolumeSnapshot`. - - `name` is the `VolumeSnapshot` resource name. In `RestoreSession` crd, You must set the VolumeSnapshot name directly. +- `spec.dataSource.repository` specifies the name of the `Repository` from which the data will be restored. +- `spec.dataSource.snapshot` specifies that we want to restore the latest snapshot of the `gcs-repository`. KubeStash retrieves `VolumeSnapshot` information from the `status.components[*]` field of this snapshot. +- `spec.addon.name` specifies the name of the `Addon` object that specifies addon configuration that will be used to perform restore of a stand-alone PVC. +- `spec.addon.tasks[*].name` specifies the name of the `Task` that holds the `Function` and their order of execution to perform restore of a stand-alone PVC. +- `spec.addon.targetVolumes.volumeClaimTemplates[*]`: + - `metadata.name` is a template for the name of the restored `PVC`. KubeStash will create `PVC` with the specified name. -Let's create the `RestoreSession` crd we have shown above. +Let's create the `RestoreSession` object we have shown above. ```bash $ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/pvc/examples/restoresession.yaml -restoresession.stash.appscode.com/restore-pvc created +restoresession.core.kubestash.com/restore-pvc created ``` -Once, you have created the `RestoreSession` crd, Stash will create a job to restore. We can watch the `RestoreSession` phase to check if the restore process has succeeded or not. +Once, you have created the `RestoreSession` object, KubeStash will create a job to restore. We can watch the `RestoreSession` phase to check if the restore process has succeeded or not. Run the following command to watch RestoreSession phase, ```bash $ watch -n 1 kubectl get restore -n demo -Every 1.0s: kubectl get restore -n demo -NAME REPOSITORY-NAME PHASE AGE -restore-pvc Running 10s -restore-pvc Succeeded 1m +Every 1.0s: kubectl get restore -n demo anisur: Tue Jan 16 17:22:34 2024 + +NAME REPOSITORY FAILURE-POLICY PHASE DURATION AGE +restore-pvc gcs-repository Succeeded 25s 28s ``` So, we can see from the output of the above command that the restore process succeeded. **Verify Restored PVC :** -Once the restore process is complete, we are going to see that new PVC with the name `restore-data` has been created. - -To verify that the PVC has been created, run by the following command, +Once the restore process is complete, we are going to see that new `PVC` with the name `restore-source-pvc` has been created. +To verify that the `PVC` has been created, run by the following command, + ```bash $ kubectl get pvc -n demo -NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE -restore-data Bound pvc-c5f0e7f5-a6ec-11e9-9f3a-42010a800050 1Gi RWO standard 52s +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE +restore-source-pvc Bound pvc-60890df8-ff1c-448c-9310-14d8ed9c06f8 1Gi RWO csi-standard 69s ``` -Notice the `STATUS` field. It indicates that the respective PV has been provisioned and initialized from the respective VolumeSnapshot by CSI driver and the PVC has been bound with the PV. +Notice the `STATUS` field. It indicates that the respective `PV` has been provisioned and initialized from the respective `VolumeSnapshot` by CSI driver and the `PVC` has been bound with the `PV`. >The [volumeBindingMode](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode) field controls when volume binding and dynamic provisioning should occur. Kubernetes allows `Immediate` and `WaitForFirstConsumer` modes for binding volumes. The `Immediate` mode indicates that volume binding and dynamic provisioning occurs once the PVC is created and `WaitForFirstConsumer` mode indicates that volume binding and provisioning does not occur until a pod is created that uses this PVC. By default `volumeBindingMode` is `Immediate`. ->If you use `volumeBindingMode: WaitForFirstConsumer`, respective PVC will be initialized from respective VolumeSnapshot after you create a workload with that PVC. In this case, Stash will mark the restore session as completed with phase `Unknown`. +>If you use `volumeBindingMode: WaitForFirstConsumer`, respective PVC will be initialized from respective VolumeSnapshot after you create a workload with that PVC. In this case, KubeStash will mark the restore session as completed with phase `Unknown`. **Verify Restored Data :** -We are going to create a new pod with the restored PVC to verify whether the backed up data has been restored. +We are going to create a new pod with the restored `PVC` to verify whether the backed up data has been restored. Below, the YAML for the Pod we are going to create. @@ -412,10 +557,9 @@ spec: volumes: - name: restore-data persistentVolumeClaim: - claimName: restore-data + claimName: restore-source-pvc readOnly: false ``` - Let's create the Pod we have shown above. ```bash @@ -428,7 +572,7 @@ Now, wait for the Pod to go into the `Running` state. ```bash $ kubectl get pod -n demo NAME READY STATUS RESTARTS AGE -restored-pod 1/1 Running 0 34s +restored-pod 1/1 Running 0 25s ``` Verify that the backed up data has been restored in `/restore/data` directory for `restored-pod` pod using the following command, @@ -447,7 +591,10 @@ kubectl delete -n demo pod source-pod kubectl delete -n demo pod restored-pod kubectl delete -n demo backupconfiguration pvc-volume-snapshot kubectl delete -n demo restoresession restore-pvc +kubectl delete -n demo retentionpolicy demo-retention +kubectl delete -n demo backupstorage gcs-storage +kubectl delete -n demo pvc --all +kubectl delete -n demo volumesnapshots --all kubectl delete -n demo storageclass csi-standard kubectl delete -n demo volumesnapshotclass csi-snapshot-class -kubectl delete -n demo pvc --all ``` diff --git a/docs/guides/volumesnapshot/statefulset/examples/backupconfiguration-single-replica.yaml b/docs/guides/volumesnapshot/statefulset/examples/backupconfiguration-single-replica.yaml deleted file mode 100644 index ecc9e46..0000000 --- a/docs/guides/volumesnapshot/statefulset/examples/backupconfiguration-single-replica.yaml +++ /dev/null @@ -1,19 +0,0 @@ -apiVersion: stash.appscode.com/v1beta1 -kind: BackupConfiguration -metadata: - name: statefulset-volume-snapshot - namespace: demo -spec: - schedule: "*/5 * * * *" - driver: VolumeSnapshotter - target: - ref: - apiVersion: apps/v1 - kind: StatefulSet - name: stash-demo - replicas: 1 - snapshotClassName: csi-snapshot-class - retentionPolicy: - name: "keep-last-5" - keepLast: 5 - prune: true diff --git a/docs/guides/volumesnapshot/statefulset/examples/backupconfiguration.yaml b/docs/guides/volumesnapshot/statefulset/examples/backupconfiguration.yaml index 4e07cee..2f84623 100644 --- a/docs/guides/volumesnapshot/statefulset/examples/backupconfiguration.yaml +++ b/docs/guides/volumesnapshot/statefulset/examples/backupconfiguration.yaml @@ -1,19 +1,37 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: BackupConfiguration metadata: name: statefulset-volume-snapshot namespace: demo spec: - schedule: "*/5 * * * *" - driver: VolumeSnapshotter target: - ref: - apiVersion: apps/v1 - kind: StatefulSet - name: stash-demo - replicas: 3 - snapshotClassName: csi-snapshot-class - retentionPolicy: - name: "keep-last-5" - keepLast: 5 - prune: true + apiGroup: apps + kind: StatefulSet + name: kubestash-demo + namespace: demo + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /sts-volume-snapshot-repo + deletionPolicy: WipeOut + addon: + name: workload-addon + tasks: + - name: volume-snapshot + params: + volumeSnapshotClassName: csi-snapshot-class \ No newline at end of file diff --git a/docs/guides/volumesnapshot/statefulset/examples/backupstorage.yaml b/docs/guides/volumesnapshot/statefulset/examples/backupstorage.yaml new file mode 100644 index 0000000..0461b26 --- /dev/null +++ b/docs/guides/volumesnapshot/statefulset/examples/backupstorage.yaml @@ -0,0 +1,17 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut \ No newline at end of file diff --git a/docs/guides/volumesnapshot/statefulset/examples/restored-statefulset.yaml b/docs/guides/volumesnapshot/statefulset/examples/restored-statefulset.yaml index 132b248..292e147 100644 --- a/docs/guides/volumesnapshot/statefulset/examples/restored-statefulset.yaml +++ b/docs/guides/volumesnapshot/statefulset/examples/restored-statefulset.yaml @@ -7,8 +7,8 @@ metadata: namespace: demo spec: ports: - - port: 80 - name: web + - port: 80 + name: web clusterIP: None selector: app: restore-demo @@ -30,25 +30,25 @@ spec: app: restore-demo spec: containers: - - args: - - sleep - - "3600" - name: nginx - image: nginx - ports: - - containerPort: 80 - name: web - volumeMounts: - - name: restore-data - mountPath: /restore/data + - args: + - sleep + - "3600" + name: nginx + image: nginx + ports: + - containerPort: 80 + name: web + volumeMounts: + - name: restore-data + mountPath: /restore/data volumeClaimTemplates: - metadata: name: restore-data namespace: demo spec: accessModes: - - ReadWriteOnce + - ReadWriteOnce storageClassName: csi-standard resources: requests: - storage: 1Gi + storage: 1Gi \ No newline at end of file diff --git a/docs/guides/volumesnapshot/statefulset/examples/restoresession-single-replica.yaml b/docs/guides/volumesnapshot/statefulset/examples/restoresession-single-replica.yaml deleted file mode 100644 index 0c5f799..0000000 --- a/docs/guides/volumesnapshot/statefulset/examples/restoresession-single-replica.yaml +++ /dev/null @@ -1,22 +0,0 @@ -apiVersion: stash.appscode.com/v1beta1 -kind: RestoreSession -metadata: - name: restore-pvc - namespace: demo -spec: - driver: VolumeSnapshotter - target: - replicas: 3 - volumeClaimTemplates: - - metadata: - name: restore-data-restore-demo-${POD_ORDINAL} - spec: - accessModes: ["ReadWriteOnce"] - storageClassName: "csi-standard" - resources: - requests: - storage: 1Gi - dataSource: - kind: VolumeSnapshot - name: source-data-stash-demo-${POD_ORDINAL}-fnbwz - apiGroup: snapshot.storage.k8s.io diff --git a/docs/guides/volumesnapshot/statefulset/examples/restoresession.yaml b/docs/guides/volumesnapshot/statefulset/examples/restoresession.yaml index 0c5f799..623f0b3 100644 --- a/docs/guides/volumesnapshot/statefulset/examples/restoresession.yaml +++ b/docs/guides/volumesnapshot/statefulset/examples/restoresession.yaml @@ -1,22 +1,23 @@ -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: RestoreSession metadata: - name: restore-pvc + name: restore-statefulset-pvc namespace: demo spec: - driver: VolumeSnapshotter - target: - replicas: 3 - volumeClaimTemplates: - - metadata: - name: restore-data-restore-demo-${POD_ORDINAL} - spec: - accessModes: ["ReadWriteOnce"] - storageClassName: "csi-standard" - resources: - requests: - storage: 1Gi - dataSource: - kind: VolumeSnapshot - name: source-data-stash-demo-${POD_ORDINAL}-fnbwz - apiGroup: snapshot.storage.k8s.io + dataSource: + repository: gcs-repository + snapshot: latest + addon: + name: pvc-addon + tasks: + - name: VolumeSnapshotRestore + targetVolumes: + volumeClaimTemplates: + - metadata: + name: restore-data-restore-demo-${POD_ORDINAL} + spec: + accessModes: [ "ReadWriteOnce" ] + storageClassName: "csi-standard" + resources: + requests: + storage: 1Gi \ No newline at end of file diff --git a/docs/guides/volumesnapshot/statefulset/examples/retentionpolicy.yaml b/docs/guides/volumesnapshot/statefulset/examples/retentionpolicy.yaml new file mode 100644 index 0000000..3af9037 --- /dev/null +++ b/docs/guides/volumesnapshot/statefulset/examples/retentionpolicy.yaml @@ -0,0 +1,15 @@ +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy +metadata: + name: demo-retention + namespace: demo +spec: + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same \ No newline at end of file diff --git a/docs/guides/volumesnapshot/statefulset/examples/statefulset.yaml b/docs/guides/volumesnapshot/statefulset/examples/statefulset.yaml index ac66f2c..f160c4d 100644 --- a/docs/guides/volumesnapshot/statefulset/examples/statefulset.yaml +++ b/docs/guides/volumesnapshot/statefulset/examples/statefulset.yaml @@ -7,52 +7,52 @@ metadata: namespace: demo spec: ports: - - port: 80 - name: web + - port: 80 + name: web clusterIP: None selector: - app: stash + app: kubestash --- apiVersion: apps/v1 kind: StatefulSet metadata: - name: stash-demo + name: kubestash-demo namespace: demo spec: selector: matchLabels: - app: stash + app: kubestash serviceName: svc replicas: 3 template: metadata: labels: - app: stash + app: kubestash spec: containers: - - args: ["echo $(POD_NAME) > /source/data/data.txt && sleep 3000"] - command: ["/bin/sh", "-c"] - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - name: nginx - image: nginx - ports: - - containerPort: 80 - name: web - volumeMounts: - - name: source-data - mountPath: /source/data + - args: ["echo $(POD_NAME) > /source/data/data.txt && sleep 3000"] + command: ["/bin/sh", "-c"] + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + name: nginx + image: nginx + ports: + - containerPort: 80 + name: web + volumeMounts: + - name: source-data + mountPath: /source/data volumeClaimTemplates: - - metadata: - name: source-data - namespace: demo - spec: - accessModes: - - ReadWriteOnce - storageClassName: csi-standard - resources: - requests: - storage: 1Gi + - metadata: + name: source-data + namespace: demo + spec: + accessModes: + - ReadWriteOnce + storageClassName: csi-standard + resources: + requests: + storage: 1Gi \ No newline at end of file diff --git a/docs/guides/volumesnapshot/statefulset/examples/storageclass.yaml b/docs/guides/volumesnapshot/statefulset/examples/storageclass.yaml index e1fa0f0..62655df 100644 --- a/docs/guides/volumesnapshot/statefulset/examples/storageclass.yaml +++ b/docs/guides/volumesnapshot/statefulset/examples/storageclass.yaml @@ -6,4 +6,4 @@ parameters: type: pd-standard provisioner: pd.csi.storage.gke.io reclaimPolicy: Delete -volumeBindingMode: Immediate +volumeBindingMode: Immediate \ No newline at end of file diff --git a/docs/guides/volumesnapshot/statefulset/examples/volumesnapshotclass.yaml b/docs/guides/volumesnapshot/statefulset/examples/volumesnapshotclass.yaml index 382ba7d..3bf5d3b 100644 --- a/docs/guides/volumesnapshot/statefulset/examples/volumesnapshotclass.yaml +++ b/docs/guides/volumesnapshot/statefulset/examples/volumesnapshotclass.yaml @@ -3,4 +3,4 @@ kind: VolumeSnapshotClass metadata: name: csi-snapshot-class driver: pd.csi.storage.gke.io -deletionPolicy: Delete +deletionPolicy: Delete \ No newline at end of file diff --git a/docs/guides/volumesnapshot/statefulset/images/gcp.png b/docs/guides/volumesnapshot/statefulset/images/gcp.png index 25a1536..ad66301 100644 Binary files a/docs/guides/volumesnapshot/statefulset/images/gcp.png and b/docs/guides/volumesnapshot/statefulset/images/gcp.png differ diff --git a/docs/guides/volumesnapshot/statefulset/images/gcp2.png b/docs/guides/volumesnapshot/statefulset/images/gcp2.png deleted file mode 100644 index aea779a..0000000 Binary files a/docs/guides/volumesnapshot/statefulset/images/gcp2.png and /dev/null differ diff --git a/docs/guides/volumesnapshot/statefulset/index.md b/docs/guides/volumesnapshot/statefulset/index.md index 39498c8..96aa1f5 100644 --- a/docs/guides/volumesnapshot/statefulset/index.md +++ b/docs/guides/volumesnapshot/statefulset/index.md @@ -1,5 +1,5 @@ --- -title: Snapshot StatefulSet Volumes | Stash +title: Snapshot StatefulSet Volumes | KubeStash description: An step by step guide showing how to snapshot the volumes of a StatefulSet menu: docs_{{ .version }}: @@ -14,13 +14,13 @@ section_menu_id: guides # Snapshotting the volumes of a StatefulSet -This guide will show you how to use Stash to snapshot the volumes of a StatefulSet and restore them from the snapshots using Kubernetes [VolumeSnapshot](https://kubernetes.io/docs/concepts/storage/volume-snapshots/) API. In this guide, we are going to backup the volumes in Google Cloud Platform with the help of [GCE Persistent Disk CSI Driver](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver). +This guide will show you how to use KubeStash to snapshot the volumes of a StatefulSet and restore them from the snapshots using Kubernetes [VolumeSnapshot](https://kubernetes.io/docs/concepts/storage/volume-snapshots/) API. In this guide, we are going to backup the volumes in Google Cloud Platform with the help of [GCE Persistent Disk CSI Driver](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver). ## Before You Begin - At first, you need to be familiar with the [GCE Persistent Disk CSI Driver](https://github.com/kubernetes-sigs/gcp-compute-persistent-disk-csi-driver). -- Install `Stash` in your cluster following the steps [here](/docs/setup/README.md). -- If you don't know how VolumeSnapshot works in Stash, please visit [here](/docs/guides/volumesnapshot/overview/index.md). +- Install `KubeStash` in your cluster following the steps [here](/docs/setup/README.md). +- If you don't know how VolumeSnapshot works in KubeStash, please visit [here](/docs/guides/volumesnapshot/overview/index.md). ## Prepare for VolumeSnapshot @@ -75,18 +75,17 @@ To keep everything isolated, we are going to use a separate namespace called `de $ kubectl create ns demo namespace/demo created ``` - ->Note: YAML files used in this tutorial are stored in [/docs/guides/volumesnapshot/statefulset/examples](/docs/guides/volumesnapshot/statefulset/examples/) directory of [kubestash/docs](https://github.com/kubestash/docs) repository. +> Note: YAML files used in this tutorial are stored in [/docs/guides/volumesnapshot/pvc/examples](/docs/guides/volumesnapshot/statefulset/examples/) directory of [kubestash/docs](https://github.com/kubestash/docs) repository. ## Take Volume Snapshot -When you create a Statefulset, there is no need to create PVCs separately, because all replicas in Statefulset use different PVCs to store data. Kubernetes allows us to define a `volumeClaimTemplates` in Statefulset so that new PVC is created for each replica automatically. We are going to take snapshot of those PVCs using Stash. +When you create a StatefulSet, there is no need to create PVCs separately, because all replicas in StatefulSet use different PVCs to store data. Kubernetes allows us to define a `volumeClaimTemplates` in StatefulSet so that new PVC is created for each replica automatically. We are going to take `VolumeSnapshot` of those PVCs using KubeStash. **Deploy StatefulSet :** -Now, we are going to deploy a Statefulset. This Statefulset will automatically generate sample data in `/source/data` directory. +Now, we are going to deploy a StatefulSet. This StatefulSet will automatically generate sample data in `/source/data` directory. -Below is the YAML of the Statefulset that we are going to create, +Below is the YAML of the StatefulSet that we are going to create, ```yaml apiVersion: v1 @@ -102,23 +101,23 @@ spec: name: web clusterIP: None selector: - app: stash + app: kubestash --- apiVersion: apps/v1 kind: StatefulSet metadata: - name: stash-demo + name: kubestash-demo namespace: demo spec: selector: matchLabels: - app: stash + app: kubestash serviceName: svc replicas: 3 template: metadata: labels: - app: stash + app: kubestash spec: containers: - args: ["echo $(POD_NAME) > /source/data/data.txt && sleep 3000"] @@ -149,604 +148,444 @@ spec: storage: 1Gi ``` -Let's create the Statefulset we have shown above. +Let's create the StatefulSet we have shown above. ```bash $ kubectl create -f ./docs/guides/volumesnapshot/statefulset/examples/statefulset.yaml service/svc created -statefulset.apps/stash-demo created +statefulset.apps/kubestash-demo created ``` -Now, wait for the pod of Statefulset to go into the `Running` state. +Now, wait for the pod of StatefulSet to go into the `Running` state. ```bash $ kubectl get pod -n demo -NAME READY STATUS RESTARTS AGE -stash-demo-0 1/1 Running 0 97s -stash-demo-1 1/1 Running 0 67s -stash-demo-2 1/1 Running 0 39s +NAME READY STATUS RESTARTS AGE +kubestash-demo-0 1/1 Running 0 62s +kubestash-demo-1 1/1 Running 0 41s +kubestash-demo-2 1/1 Running 0 19s ``` Let's find out the PVCs created for these replicas, ```bash kubectl get pvc -n demo -NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE -source-data-stash-demo-0 Bound pvc-760c1734-a6cc-11e9-9f3a-42010a800050 1Gi RWO standard 70s -source-data-stash-demo-1 Bound pvc-86f5b3bd-a6cc-11e9-9f3a-42010a800050 1Gi RWO standard 42s -source-data-stash-demo-2 Bound pvc-9c9f542f-a6cc-11e9-9f3a-42010a800050 1Gi RWO standard 5s +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE +source-data-kubestash-demo-0 Bound pvc-13d017a6-0df4-4874-9a80-886b63d7807f 1Gi RWO csi-standard 83s +source-data-kubestash-demo-1 Bound pvc-ad0a8f63-f715-40c8-a6af-8e361b9f76bf 1Gi RWO csi-standard 62s +source-data-kubestash-demo-2 Bound pvc-3ce8dfed-cea8-4456-a34d-4c8aa343c6a2 1Gi RWO csi-standard 40s ``` Verify that the sample data has been created in `/source/data` directory using the following command, ```bash -$ kubectl exec -n demo stash-demo-0 -- cat /source/data/data.txt -stash-demo-0 -$ kubectl exec -n demo stash-demo-1 -- cat /source/data/data.txt -stash-demo-1 -$ kubectl exec -n demo stash-demo-2 -- cat /source/data/data.txt -stash-demo-2 -``` - -**Create BackupConfiguration :** - -Now, create a `BackupConfiguration` crd to take snapshot of the PVCs of `stash-demo` Statefulset. - -Below is the YAML of the `BackupConfiguration` that we are going to create, - -```yaml -apiVersion: stash.appscode.com/v1beta1 -kind: BackupConfiguration -metadata: - name: statefulset-volume-snapshot - namespace: demo -spec: - schedule: "*/5 * * * *" - driver: VolumeSnapshotter - target: - ref: - apiVersion: apps/v1 - kind: StatefulSet - name: stash-demo - replicas : 3 - snapshotClassName: csi-snapshot-class - retentionPolicy: - name: 'keep-last-5' - keepLast: 5 - prune: true +$ kubectl exec -n demo kubestash-demo-0 -- cat /source/data/data.txt +kubestash-demo-0 +$ kubectl exec -n demo kubestash-demo-1 -- cat /source/data/data.txt +kubestash-demo-1 +$ kubectl exec -n demo kubestash-demo-2 -- cat /source/data/data.txt +kubestash-demo-2 ``` -Here, - -- `spec.schedule` is a [cron expression](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) indicates that `BackupSession` will be created at 5 minute interval. +## Backup -- `spec.driver` indicates the name of the agent to use to back up the target. Currently, Stash supports `Restic`, `VolumeSnapshotter` drivers. The `VolumeSnapshotter` is used to backup/restore PVC using `VolumeSnapshot` API. +Now, we are going to taking `VolumeSnapshot` of the StatefulSet `kubestash-demo` PVCs using KubeStash. We have to create a `Secret` and a `BackupStorage` object with access credentials and backend information respectively. -- `spec.target.ref` refers to the backup target. `apiVersion`, `kind` and `name` refers to the `apiVersion`, `kind` and `name` of the targeted workload respectively. Stash will use this information to create a Volume Snapshotter Job for creating VolumeSnapshot. +> For GCS backend, if the bucket does not exist, KubeStash needs `Storage Object Admin` role permissions to create the bucket. For more details, please check the following [guide](/docs/guides/backends/gcs/index.md). -- `spec.target.snapshotClassName` indicates the [VolumeSnapshotClass](https://kubernetes.io/docs/concepts/storage/volume-snapshot-classes/) to be used for volume snapshotting. +**Create Storage Secret:** -Let's create the `BackupConfiguration` crd we have shown above. +Let's create a Secret named `gcs-secret` with access credentials of our desired GCS backend, ```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/statefulset/examples/backupconfiguration.yaml -backupconfiguration.stash.appscode.com/statefulset-volume-snapshot created +$ echo -n '' > GOOGLE_PROJECT_ID +$ cat /path/to/downloaded/sa_key_file.json > GOOGLE_SERVICE_ACCOUNT_JSON_KEY +$ kubectl create secret generic -n demo gcs-secret \ + --from-file=./GOOGLE_PROJECT_ID \ + --from-file=./GOOGLE_SERVICE_ACCOUNT_JSON_KEY +secret/gcs-secret created ``` -**Verify CronJob :** +**Create BackupStorage:** -If everything goes well, Stash will create a `CronJob` to take periodic snapshot of `stash-demo-0` , `stash-demo-1` and `stash-demo-2` volumes of the Statefulset with the schedule specified in `spec.schedule` field of `BackupConfiguration` crd. +Now, create a `BackupStorage` custom resource specifying the desired bucket, and directory inside the bucket where the backed up data will be stored. - Check that the `CronJob` has been created using the following command, +Below is the YAML of `BackupStorage` object that we are going to create, -```bash -$ kubectl get cronjob -n demo -NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE -statefulset-volume-snapshot */1 * * * * False 0 18s +```yaml +apiVersion: storage.kubestash.com/v1alpha1 +kind: BackupStorage +metadata: + name: gcs-storage + namespace: demo +spec: + storage: + provider: gcs + gcs: + bucket: kubestash-qa + prefix: demo + secretName: gcs-secret + usagePolicy: + allowedNamespaces: + from: All + default: true + deletionPolicy: WipeOut ``` -**Wait for BackupSession :** - -The `statefulset-volume-snapshot` CronJob will trigger a backup on each schedule by creating a `BackupSession` crd. - -Wait for the next schedule for backup. Run the following command to watch `BackupSession` crd, +Let's create the `BackupStorage` object that we have shown above, ```bash -$ watch -n 1 kubectl get backupsession -n demo -Every 1.0s: kubectl get backupsession -n demo - -NAME INVOKER-TYPE INVOKER-NAME PHASE AGE -statefulset-volume-snapshot-fnbwz BackupConfiguration statefulset-volume-snapshot Succeeded 57s +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumes/statefulset/examples/backupstorage.yaml +backupstorage.storage.kubestash.com/gcs-storage created ``` -We can see above that the backup session has succeeded. Now, we are going to verify that the VolumeSnapshot has been created and the snapshots has been stored in the respective backend. - -**Verify Volume Snapshot :** - -Once a `BackupSession` crd is created, it creates volume snapshotter `Job`. Then the `Job` creates `VolumeSnapshot` crd for the targeted PVC. +Now, we have to create a `RetentionPolicy` custom resource which specifies how the old `Snapshots` should be cleaned up. -Check that the `VolumeSnapshot` has been created Successfully. +**Create RetentionPolicy:** -```bash -$ kubectl get volumesnapshot -n demo -NAME AGE -source-data-stash-demo-0-fnbwz 115s -source-data-stash-demo-1-fnbwz 115s -source-data-stash-demo-2-fnbwz 115s -``` - -Let's find out the actual snapshot name that has been saved in the Google Cloud by the following command, - -```bash -kubectl get volumesnapshot source-data-stash-demo-0-fnbwz -n demo -o yaml -``` +Below is the YAML of the `RetentionPolicy` object that we are going to create, ```yaml -apiVersion: snapshot.storage.k8s.io/v1 -kind: VolumeSnapshot +apiVersion: storage.kubestash.com/v1alpha1 +kind: RetentionPolicy metadata: - creationTimestamp: "2019-07-15T07:59:13Z" - finalizers: - - snapshot.storage.kubernetes.io/volumesnapshot-protection - generation: 4 - name: source-data-stash-demo-0-fnbwz + name: demo-retention namespace: demo - resourceVersion: "18764" - selfLink: /apis/snapshot.storage.k8s.io/v1/namespaces/demo/volumesnapshots/source-data-stash-demo-0-fnbwz - uid: 6f3b49a9-a6d6-11e9-9f3a-42010a800050 spec: - source: - persistentVolumeClaimName: source-data-stash-demo-0 - volumeSnapshotClassName: csi-snapshot-class -status: - boundVolumeSnapshotContentName: snapcontent-6f3b49a9-a6d6-11e9-9f3a-42010a800050 - creationTime: "2019-07-15T07:59:14Z" - readyToUse: true - restoreSize: 1Gi + default: true + failedSnapshots: + last: 2 + maxRetentionPeriod: 2mo + successfulSnapshots: + last: 5 + usagePolicy: + allowedNamespaces: + from: Same ``` -Here, `spec.snapshotContentName` field specifies the name of the `VolumeSnapshotContent` crd. It also represents the actual snapshot name that has been saved in Google Cloud. If we navigate to the `Snapshots` tab in the GCP console, we should see snapshot `snapcontent-6f3b49a9-a6d6-11e9-9f3a-42010a800050` has been stored successfully. - -
-  Stash Backup Flow -
Fig: Snapshots in GCP
-
- -## Restore PVC from VolumeSnapshot - -This section will show you how to restore PVCs from the snapshots we have taken in the earlier section. - -**Stop Taking Backup of the Old StatefulSet:** - -At first, let's stop taking any further backup of the old StatefulSet so that no backup is taken during the restore process. We are going to pause the `BackupConfiguration` that we created to backup the `stash-demo` StatefulSet. Then, Stash will stop taking any further backup for this StatefulSet. You can learn more how to pause a scheduled backup [here](/docs/guides/use-cases/pause-backup/index.md) - -Let's pause the `statefulset-volume-snapshot` BackupConfiguration, +Let's create the `RetentionPolicy` object that we have shown above, ```bash -$ kubectl patch backupconfiguration -n demo statefulset-volume-snapshot --type="merge" --patch='{"spec": {"paused": true}}' -backupconfiguration.stash.appscode.com/statefulset-volume-snapshot patched +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumes/statefulset/examples/retentionpolicy.yaml +retentionpolicy.storage.kubestash.com/demo-retention created ``` -Now, wait for a moment. Stash will pause the BackupConfiguration. Verify that the BackupConfiguration has been paused, +We are ready. Now, we have to create a `BackupConfiguration` custom resource targeting Statefulset `kubestash-demo`. -```bash -$ kubectl get backupconfiguration -n demo -NAME TASK SCHEDULE PAUSED AGE -statefulset-volume-snapshot */1 * * * * true 20m -``` - -Notice the `PAUSED` column. Value `true` for this field means that the BackupConfiguration has been paused. - -**Create RestoreSession :** +**Create BackupConfiguration :** -At first, we have to create a `RestoreSession` crd to restore PVCs from respective the snapshots. +Now, create a `BackupConfiguration` custom resource to take `volumeSnapshot` of the `kubestash-demo` Statefulset. -Below is the YAML of the `RestoreSesion` crd that we are going to create, +Below is the YAML of the `BackupConfiguration` object that we are going to create, ```yaml -apiVersion: stash.appscode.com/v1beta1 -kind: RestoreSession +apiVersion: core.kubestash.com/v1alpha1 +kind: BackupConfiguration metadata: - name: restore-pvc + name: statefulset-volume-snapshot namespace: demo spec: - driver: VolumeSnapshotter target: - replicas: 3 - volumeClaimTemplates: - - metadata: - name: restore-data-restore-demo-${POD_ORDINAL} - spec: - accessModes: ["ReadWriteOnce"] - storageClassName: "csi-standard" - resources: - requests: - storage: 1Gi - dataSource: - kind: VolumeSnapshot - name: source-data-stash-demo-${POD_ORDINAL}-fnbwz - apiGroup: snapshot.storage.k8s.io + apiGroup: apps + kind: StatefulSet + name: kubestash-demo + namespace: demo + backends: + - name: gcs-backend + storageRef: + namespace: demo + name: gcs-storage + retentionPolicy: + name: demo-retention + namespace: demo + sessions: + - name: frequent-backup + sessionHistoryLimit: 3 + scheduler: + schedule: "*/5 * * * *" + jobTemplate: + backoffLimit: 1 + repositories: + - name: gcs-repository + backend: gcs-backend + directory: /sts-volume-snapshot-repo + deletionPolicy: WipeOut + addon: + name: workload-addon + tasks: + - name: volume-snapshot + params: + volumeSnapshotClassName: csi-snapshot-class ``` - Here, +- `spec.target` specifies the backup target. `apiGroup`, `kind`, `namespace` and `name` refers to the `apiGroup`, `kind`, `namespace` and `name` of the targeted workload respectively. +- `spec.sessions[*].scheduler.schedule` specifies a [cron expression](https://kubernetes.io/docs/tasks/job/automated-tasks-with-cron-jobs/#schedule) indicates that `BackupSession` will be created at 5 minute interval. +- `spec.sessions[*].repositories[*].name` specifies the name of the `Repository` object that holds the backend information. +- `spec.sessions[*].repositories[*].backend` specifies the name of the backend that holds the `BackupStorage` information. +- `spec.sessions[*].repositories[*].directory` specifies the path of the `Repository` where the backed up data will be stored. +- `spec.sessions[*].addon.name` specifies the name of the `Addon` object that specifies addon configuration that will be used to perform backup of a stand-alone PVC. +- `spec.sessions[*].addon.tasks[*].name` specifies the name of the `Task` that holds the `Function` and their order of execution to perform backup of a stand-alone PVC. +- `spec.sessions[*].addon.tasks[*].params.volumeSnapshotClassName` indicates the [VolumeSnapshotClass](https://kubernetes.io/docs/concepts/storage/volume-snapshot-classes/) to be used for volume snapshotting. If we don't provide any then kubestash use default `volumeSnapshotClass` for volume snapshotting. -- `spec.target.replicas`: `spec.target.replicas` specify the number of replicas of a StatefulSet whose volumes were backed up and Stash uses this field to dynamically create the desired number of PVCs and initialize them from respective or Specific VolumeSnapShots. -- `spec.target.volumeClaimTemplates`: - - `metadata.name` is a template for the name of the restored PVC that will be created by Stash. You have to provide this named template to match with the desired PVC of a StatefulSet. For example, if you want to deploy a StatefulSet named `stash-demo` with `volumeClaimTemplate` name `my-volume`, the PVCs of your StatefulSet will be `my-volume-stash-demo-0`, `my-volume-stash-demo-1` and so on. In this case, you have to provide `volumeClaimTemplate` name in `RestoreSession` in the following format: - - ```bash - --${POD_ORDINAL} - ``` - - So for the above example, `volumeClaimTemplate` name for `RestoreSession` will be `my-volume-stash-demo-${POD_ORDINAL}`. - - `spec.dataSource`: `spec.dataSource` specifies the source of the data from where the newly created PVC will be initialized. It requires following fields to be set: - - `apiGroup` is the group for resource being referenced. Now, Kubernetes supports only `snapshot.storage.k8s.io`. - - `kind` is resource of the kind being referenced. Now, Kubernetes supports only `VolumeSnapshot`. - - `name` is the `VolumeSnapshot` resource name. In `RestoreSession` crd, You must provide the name in the following format: +Let's create the `BackupConfiguration` object we have shown above. - ```bash - -${POD_ORDINAL}- - ``` +```bash +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/statefulset/examples/backupconfiguration.yaml +backupconfiguration.core.kubestash.com/statefulset-volume-snapshot created +``` - The `${POD_ORDINAL}` variable is resolved by Stash. If you don't provide this variable and specify ordinal manually, all the PVC will be restored from the same VolumeSnapshot. +**Verify Backup Setup Successful** -Let's create the `RestoreSession` crd we have shown above. +If everything goes well, the phase of the `BackupConfiguration` should be in `Ready` state. The `Ready` phase indicates that the backup setup is successful. Let's check the `Phase` of the BackupConfiguration, ```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/statefulset/examples/restoresession.yaml -restoresession.stash.appscode.com/restore-pvc created +$ kubectl get backupconfiguration -n demo +NAME PHASE PAUSED AGE +statefulset-volume-snapshot Ready 12s ``` +**Verify Repository:** -Once, you have created the `RestoreSession` crd, Stash will create a job to restore. We can watch the `RestoreSession` phase to check if the restore process has succeeded or not. - -Run the following command to watch RestoreSession phase, +Verify that KubeStash has created `Repositories` that holds the `BackupStorage` information by the following command, ```bash -$ watch -n 1 kubectl get restore -n demo -Every 1.0s: kubectl get restore -n demo - -NAME REPOSITORY-NAME PHASE AGE -restore-pvc Running 10s -restore-pvc Succeeded 1m +$ kubectl get repositories -n demo +NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE +gcs-repository Ready 28s ``` -So, we can see from the output of the above command that the restore process succeeded. - -**Verify Restored PVC :** +**Verify CronJob:** -Once the restore process is complete, we are going to see that new PVCs with the name `restore-data-restore-demo-0` , `restore-data-restore-demo-1` and `restore-data-restore-demo-2` has been created. +If everything goes well, KubeStash will create a `CronJob` to take periodic `VolumeSnapshot` of `kubestash-demo-0` , `kubestash-demo-1` and `kubestash-demo-2` volumes of the StatefulSet with the schedule specified in `spec.schedule` field of `BackupConfiguration` object. -Verify that the PVCs has been created by the following command, +Check that the `CronJob` has been created using the following command, ```bash -$ kubectl get pvc -n demo -NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE -restore-data-restore-demo-0 Bound pvc-ed35c54d-a6dc-11e9-9f3a-42010a800050 1Gi RWO standard 13s -restore-data-restore-demo-1 Bound pvc-ed3bcb82-a6dc-11e9-9f3a-42010a800050 1Gi RWO standard 13s -restore-data-restore-demo-2 Bound pvc-ed3fed79-a6dc-11e9-9f3a-42010a800050 1Gi RWO standard 13s +$ kubectl get cronjob -n demo +NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE +trigger-statefulset-volume-snapshot-frequent-backup */5 * * * * False 0 99s ``` -Notice the `STATUS` field. It indicates that the respective PV has been provisioned and initialized from the respective VolumeSnapshot by CSI driver and the PVC has been bound with the PV. - ->The [volumeBindingMode](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode) field controls when volume binding and dynamic provisioning should occur. Kubernetes allows `Immediate` and `WaitForFirstConsumer` modes for binding volumes. The `Immediate` mode indicates that volume binding and dynamic provisioning occurs once the PVC is created and `WaitForFirstConsumer` mode indicates that volume binding and provisioning does not occur until a pod is created that uses this PVC. By default `volumeBindingMode` is `Immediate`. - ->If you use `volumeBindingMode: WaitForFirstConsumer`, respective PVC will be initialized from respective VolumeSnapshot after you create a workload with that PVC. In this case, Stash will mark the restore session as completed with phase `Unknown`. +**Wait for BackupSession:** -**Verify Restored Data :** +Now, wait for the next backup schedule. You can watch for `BackupSession` object using the following command, -We are going to create a new Statefulset with the restored PVCs to verify whether the backed up data has been restored. +```bash +$ watch -n 1 kubectl get backupsession -n demo -l=kubestash.com/invoker-name=statefulset-volume-snapshot +Every 1.0s: kubectl get backupsession -n demo -l=kubestash.com/invoker-name=statefulset-volume-sn... anisur: Tue Jan 16 17:40:07 2024 -Below, the YAML for the Statefulset we are going to create. +NAME INVOKER-TYPE INVOKER-NAME PHASE DURATION AGE +statefulset-volume-snapshot-frequent-backup-1705405201 BackupConfiguration statefulset-volume-snapshot Running 7s -```yaml -apiVersion: v1 -kind: Service -metadata: - name: restore-svc - labels: - app: restore-demo - namespace: demo -spec: - ports: - - port: 80 - name: web - clusterIP: None - selector: - app: restore-demo ---- -apiVersion: apps/v1 -kind: StatefulSet -metadata: - name: restore-demo - namespace: demo -spec: - selector: - matchLabels: - app: restore-demo - serviceName: svc - replicas: 3 - template: - metadata: - labels: - app: restore-demo - spec: - containers: - - args: - - sleep - - "3600" - name: nginx - image: nginx - ports: - - containerPort: 80 - name: web - volumeMounts: - - name: restore-data - mountPath: /restore/data - volumeClaimTemplates: - - metadata: - name: restore-data - namespace: demo - spec: - accessModes: - - ReadWriteOnce - storageClassName: csi-standard - resources: - requests: - storage: 1Gi ``` -Let's create the Statefulset we have shown above. +Once a `BackupSession` object is created, it creates a `Snapshot` custom resource for each `Repository`. This snapshot `spec.status.components` section represents the backup information of the targeted backup components. + +Verify created `Snapshot` object by the following command, ```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/statefulset/examples/restored-statefulset.yaml -service/svc created -statefulset.apps/restore-demo created -``` +$ watch -n 1 kubectl get snapshot -n demo -l=kubestash.com/repo-name=gcs-repository -Now, wait for the pod of the Statefulset to go into the `Running` state. +Every 2.0s: kubectl get snapshots.storage.kubestash.com -n demo -l=kubestash.com/repo-name=gcs-repository anisur: Tue Jan 16 17:03:06 2024 -```bash -$ kubectl get pod -n demo -NAME READY STATUS RESTARTS AGE -restore-demo-0 1/1 Running 0 65s -restore-demo-1 1/1 Running 0 46s -restore-demo-2 1/1 Running 0 26s +NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE VERIFICATION-STATUS AGE +gcs-repository-statefulset-volumshot-frequent-backup-1705405201 gcs-repository frequent-backup 2024-01-16T11:00:08Z Delete Succeeded 3m6s ``` - -Verify that the backed up data has been restored in `/restore/data` directory using the following command, +Now, retrieve the YAML representation of the above created `Snapshot`, and inspect the `spec.status` section to see the backup information of the targeted backup components. ```bash -$ kubectl exec -n demo restore-demo-0 -- cat /restore/data/data.txt -stash-demo-0 -$ kubectl exec -n demo restore-demo-1 -- cat /restore/data/data.txt -stash-demo-1 -$ kubectl exec -n demo restore-demo-2 -- cat /restore/data/data.txt -stash-demo-2 +$ kubectl get snapshots -n demo gcs-repository-statefulset-volumshot-frequent-backup-1705405201 -o yaml ``` - -## Advance Use-Case - -Stash can also backup only single replica or restore same data on all replicas of a StatefulSet. This is particularly useful when all replicas of the StatefulSet contains same data. For example, in [MongoDB ReplicaSet](https://kubedb.com/docs/0.12.0/guides/mongodb/clustering/replication_concept/) all the pod contains same data. In this case backup only single replica is enough. Similarly, it might be useful in some cases where all the replicas need to be initialized with same data. - -### Backup only Single Replica - -This section will show you how to snapshot only a single replica of a Statefulset volume. - -**Create BackupConfiguration :** - -Now, create a `BackupConfiguration` crd to take snapshot of a single PVC of the `stash-demo` Statefulset. - -Below is the YAML of the `BackupConfiguration` that we are going to create, - ```yaml -apiVersion: stash.appscode.com/v1beta1 -kind: BackupConfiguration +apiVersion: storage.kubestash.com/v1alpha1 +kind: Snapshot metadata: - name: statefulset-volume-snapshot + creationTimestamp: "2024-01-16T11:40:01Z" + name: gcs-repository-statefulset-volumshot-frequent-backup-1705405201 namespace: demo spec: - schedule: "*/5 * * * *" - driver: VolumeSnapshotter - target: - ref: - apiVersion: apps/v1 - kind: StatefulSet - name: stash-demo - replicas: 1 - snapshotClassName: csi-snapshot-class - retentionPolicy: - name: "keep-last-5" - keepLast: 5 - prune: true + --- +status: + components: + volumesnapshot-pod-0: + driver: VolumeSnapshotter + phase: Succeeded + volumeSnapshotterStats: + - hostPath: /source/data + pvcName: source-data-kubestash-demo-0 + volumeSnapshotName: source-data-kubestash-demo-0-1705405201 + volumesnapshot-pod-1: + driver: VolumeSnapshotter + phase: Succeeded + volumeSnapshotterStats: + - hostPath: /source/data + pvcName: source-data-kubestash-demo-1 + volumeSnapshotName: source-data-kubestash-demo-1-1705405201 + volumesnapshot-pod-2: + driver: VolumeSnapshotter + phase: Succeeded + volumeSnapshotterStats: + - hostPath: /source/data + pvcName: source-data-kubestash-demo-2 + volumeSnapshotName: source-data-kubestash-demo-2-1705405201 + conditions: + --- + phase: Succeeded + snapshotTime: "2024-01-16T11:40:45Z" + totalComponents: 3 ``` Here, +- `spec.status.components` specifies the backup information of the backup target components. +- `spec.status.components.[].volumeSnapshotterStats` specifies the information about the `VolumeSnapshotter` driver. In the case of a statefulset `PVC`, the component name is designated as `volumesnapshot-pod-`. + - `volumeSnapshotterStats.hostPath` specifies the corresponding path of `PVC` volume for which `volumeSnapshot` has created. + - `volumeSnapshotterStats.pvcName` specifies the name of the backed-up target `PVC` volume. + - `volumeSnapshotterStats.volumeSnapshotName` specifies the created `VolumeSnapshot` object name corresponding to the target `PVC` volume. -- `spec.replicas` specifies the number of replicas (starting from 0th) whose data should be backed up. If it is set to 1, Stash will take snapshot only the volumes of `--0` pod. For replica set to 2, Stash will take snapshot only the volumes of `--0` and `--1` pods and so on. - -Let's create the `BackupConfiguration` crd we have shown above. - -```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/statefulset/examples/backupconfiguration-single-replica.yaml -backupconfiguration.stash.appscode.com/statefulset-volume-snapshot created -``` - -**Verify CronJob :** - -If everything goes well, Stash will create a `CronJob` to take periodic snapshot of `stash-demo-0` volume of the Statefulset with the schedule specified in `spec.schedule` field of `BackupConfiguration` crd. - - Check that the `CronJob` has been created using the following command, - -```bash -$ kubectl get cronjob -n demo -NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE -statefulset-volume-snapshot */1 * * * * False 0 18s -``` - -**Wait for BackupSession :** - -The `statefulset-volume-snapshot` CronJob will trigger a backup on each schedule by creating a `BackupSession` crd. - -Wait for the next schedule for backup. Run the following command to watch `BackupSession` crd, - -```bash -$ watch -n 1 kubectl get backupsession -n demo -Every 1.0s: kubectl get backupsession -n demo - -NAME INVOKER-TYPE INVOKER-NAME PHASE AGE -statefulset-volume-snapshot-fnbwz BackupConfiguration statefulset-volume-snapshot Succeeded 57s -``` - -We can see above that the backup session has succeeded. Now, we are going to verify that the VolumeSnapshot has been created and the snapshot has been stored in the respective backend. -**Verify Volume Snapshotting and Backup :** +**Verify Volume Snapshot :** -Once a `BackupSession` crd is created, Stash creates a volume snapshotter `Job`. Then the `Job` creates `VolumeSnapshot` crd for the targeted PVC. +Once a `BackupSession` object is created, it creates volume snapshotter `Job`. Then the `Job` creates `VolumeSnapshot` object for each the targeted StatefulSet volumes. We can see above that the backup session has succeeded. Now, we are going to verify that the `VolumeSnapshot` has been created and the snapshot have been stored in the respective backend. -Check that the `VolumeSnapshot` has been created Successfully. +Check that all the `VolumeSnapshot` has been created Successfully. ```bash $ kubectl get volumesnapshot -n demo -NAME AGE -source-data-stash-demo-0-fnbwz 67s +NAME READYTOUSE SOURCEPVC SOURCESNAPSHOTCONTENT RESTORESIZE SNAPSHOTCLASS SNAPSHOTCONTENT CREATIONTIME AGE +source-data-kubestash-demo-0-1705405201 true source-data-kubestash-demo-0 1Gi csi-snapshot-class snapcontent-f4a199c2-eed5-4438-aa09-e9c9683556ef 17m 17m +source-data-kubestash-demo-1-1705405201 true source-data-kubestash-demo-1 1Gi csi-snapshot-class snapcontent-b7b7ffa1-809d-4c35-aa79-445f8fd39313 17m 17m +source-data-kubestash-demo-2-1705405201 true source-data-kubestash-demo-2 1Gi csi-snapshot-class snapcontent-bc84b998-0077-4458-bbf2-b85d06628296 17m 17m ``` -Let's find out the actual snapshot name that will be saved in the GCP by the following command, +Let's find out the actual `VolumeSnapshotContent` that has been saved in the Google Cloud by the following command, ```bash -kubectl get volumesnapshot source-data-stash-demo-0-fnbwz -n demo -o yaml +kubectl get volumesnapshot -n demo source-data-kubestash-demo-0-1705405201 -o yaml ``` ```yaml apiVersion: snapshot.storage.k8s.io/v1 kind: VolumeSnapshot metadata: - creationTimestamp: "2019-07-15T09:01:06Z" + creationTimestamp: "2024-01-16T11:40:04Z" finalizers: - - snapshot.storage.kubernetes.io/volumesnapshot-protection - generation: 4 - name: source-data-stash-demo-0-fnbwz + - snapshot.storage.kubernetes.io/volumesnapshot-as-source-protection + - snapshot.storage.kubernetes.io/volumesnapshot-bound-protection + generation: 1 + name: source-data-kubestash-demo-0-1705405201 namespace: demo - resourceVersion: "24310" - selfLink: /apis/snapshot.storage.k8s.io/v1/namespaces/demo/volumesnapshots/source-data-stash-demo-0-fnbwz - uid: 14984cd3-a6df-11e9-9f3a-42010a800050 + resourceVersion: "30453" + uid: f4a199c2-eed5-4438-aa09-e9c9683556ef spec: source: - persistentVolumeClaimName: source-data-stash-demo-0 + persistentVolumeClaimName: source-data-kubestash-demo-0 volumeSnapshotClassName: csi-snapshot-class status: - boundVolumeSnapshotContentName: snapcontent-14984cd3-a6df-11e9-9f3a-42010a800050 - creationTime: "2019-07-15T09:01:07Z" + boundVolumeSnapshotContentName: snapcontent-f4a199c2-eed5-4438-aa09-e9c9683556ef + creationTime: "2024-01-16T11:40:06Z" readyToUse: true restoreSize: 1Gi ``` -Here, `spec.snapshotContentName` field specifies the name of the `VolumeSnapshotContent` crd. It also represents the actual snapshot name that has been saved in GCP. If we navigate to the `Snapshots` tab in the GCP console, we should see the snapshot `snapcontent-14984cd3-a6df-11e9-9f3a-42010a800050` has been stored successfully. +Here, `spec.status.boundVolumeSnapshotContentName` field specifies the name of the `VolumeSnapshotContent` object. It also represents the actual snapshot name that has been saved in Google Cloud. If we navigate to the `Snapshots` tab in the GCP console, we are going to see snapshot `snapcontent-f4a199c2-eed5-4438-aa09-e9c9683556ef` has been stored successfully.
-  Stash Backup Flow -
Fig: Snapshot in GCP
+  Snapshots in GCP +
Fig: Snapshots in GCP
-### Restore Same Data in all Replicas - -This section will show you how to restore PVCs from the snapshot that we have taken in the earlier section. - -**Stop Taking Backup of the Old StatefulSet:** - -At first, let's stop taking any further backup of the old StatefulSet so that no backup is taken during the restore process. We are going to pause the `BackupConfiguration` that we created to backup the `stash-demo` StatefulSet. Then, Stash will stop taking any further backup for this StatefulSet. You can learn more how to pause a scheduled backup [here](/docs/guides/use-cases/pause-backup/index.md) - -Let's pause the `statefulset-volume-snapshot` BackupConfiguration, - -```bash -$ kubectl patch backupconfiguration -n demo statefulset-volume-snapshot --type="merge" --patch='{"spec": {"paused": true}}' -backupconfiguration.stash.appscode.com/statefulset-volume-snapshot patched -``` - -Now, wait for a moment. Stash will pause the BackupConfiguration. Verify that the BackupConfiguration has been paused, - -```bash -$ kubectl get backupconfiguration -n demo -NAME TASK SCHEDULE PAUSED AGE -statefulset-volume-snapshot */1 * * * * true 20m -``` +## Restore PVC from VolumeSnapshot -Notice the `PAUSED` column. Value `true` for this field means that the BackupConfiguration has been paused. +This section will show you how to restore the PVCs from the snapshots we have taken in the previous section. **Create RestoreSession :** -At first, we have to create a `RestoreSession` crd to restore PVCs from the respective snapshot. - -Below is the YAML of the `RestoreSesion` crd that we are going to create, +Now, we are going to create a `RestoreSession` custom resource to restore all backed-up `PVC` volumes individually from their respective `VolumeSnapshot`. Below is the YAML of the `RestoreSession` object that we are going to create, ```yaml -apiVersion: stash.appscode.com/v1beta1 +apiVersion: core.kubestash.com/v1alpha1 kind: RestoreSession metadata: - name: restore-pvc + name: restore-statefulset-pvc namespace: demo spec: - driver: VolumeSnapshotter - target: - replicas : 3 - volumeClaimTemplates: - - metadata: - name: restore-data-restore-demo-${POD_ORDINAL} - spec: - accessModes: [ "ReadWriteOnce" ] - storageClassName: "standard" - resources: - requests: - storage: 1Gi - dataSource: - kind: VolumeSnapshot - name: source-data-stash-demo-0-fnbwz - apiGroup: snapshot.storage.k8s.io + dataSource: + repository: gcs-repository + snapshot: latest + addon: + name: pvc-addon + tasks: + - name: VolumeSnapshotRestore + targetVolumes: + volumeClaimTemplates: + - metadata: + name: restore-data-restore-demo-${POD_ORDINAL} + spec: + accessModes: [ "ReadWriteOnce" ] + storageClassName: "csi-standard" + resources: + requests: + storage: 1Gi ``` Here, -- `spec.dataSource.name`: `spec.dataSource.name` is the `VolumeSnapshot` resource name. data will be restored in all replica from single VolumeSnapshot. +- `spec.dataSource.repository` specifies the name of the `Repository` from which the data will be restored. +- `spec.dataSource.snapshot` specifies that we want to restore the latest snapshot of the `gcs-repository`. KubeStash retrieves `VolumeSnapshot` information from the `status.components[*]` field of this snapshot. +- `spec.addon.name` specifies the name of the `Addon` object that specifies addon configuration that will be used to perform restore of a stand-alone PVC. +- `spec.addon.tasks[*].name` specifies the name of the `Task` that holds the `Function` and their order of execution to perform restore of a stand-alone PVC. +- `spec.addon.targetVolumes.volumeClaimTemplates[*]`: + - `metadata.name` is a template for the name of the restored PVC that will be created by KubeStash. You have to provide this named template to match with the desired PVC of a StatefulSet. For example, if you want to deploy a StatefulSet named `kubestash-demo` with `volumeClaimTemplate` name `my-volume`, the PVCs of your StatefulSet will be `my-volume-kubestash-demo-0`, `my-volume-kubestash-demo-1` and so on. In this case, you have to provide `volumeClaimTemplate` name in `RestoreSession` in the following format: + ```bash + --${POD_ORDINAL} + ``` + - The `${POD_ORDINAL}` variable is resolved by KubeStash. + - So for the above example, `volumeClaimTemplate` name for `RestoreSession` will be `restore-data-restore-demo-${POD_ORDINAL}`. -Let's create the `BackupConfiguration` crd we have shown above. +Let's create the `RestoreSession` object we have shown above. ```bash -$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/statefulset/examples/restoresession-single-replica.yaml -restoresession.stash.appscode.com/restore-pvc created -``` +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/statefulset/examples/restoresession.yaml -Once, you have created the `RestoreSession` crd, Stash will create a job to restore. We can watch the `RestoreSession` phase to check if the restore process has succeeded or not. +``` +Once, you have created the `RestoreSession` object, KubeStash will create a job to restore. We can watch the `RestoreSession` phase to check if the restore process has succeeded or not. -Run the following command to watch RestoreSession phase, +Run the following command to watch `RestoreSession` phase, ```bash $ watch -n 1 kubectl get restore -n demo -Every 1.0s: kubectl get restore -n demo suaas-appscode: Tue Jun 18 18:35:41 2019 +Every 1.0s: kubectl get restore -n demo anisur: Tue Jan 16 18:08:51 2024 + +NAME REPOSITORY FAILURE-POLICY PHASE DURATION AGE +restore-statefulset-pvc gcs-repository Running 50s +restore-statefulset-pvc gcs-repository Succeeded 50s -NAME REPOSITORY-NAME PHASE AGE -restore-pvc Running 10s -restore-pvc Succeeded 1m ``` So, we can see from the output of the above command that the restore process succeeded. **Verify Restored PVC :** -Once the restore process is complete, we are going to see that new PVCs with the name `restore-data-restore-demo-0` , `restore-data-restore-demo-1` and `restore-data-restore-demo-2` have been created successfully. +Once the restore process is complete, we are going to see that new PVCs with the name `restore-data-restore-demo-0` , `restore-data-restore-demo-1` and `restore-data-restore-demo-2` has been created. -check that the status of the PVCs are bound, +Verify that the PVCs has been created by the following command, ```bash $ kubectl get pvc -n demo -NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE -restore-data-restore-demo-0 Bound pvc-745e0f51-a6e0-11e9-9f3a-42010a800050 1Gi RWO standard 5m23s -restore-data-restore-demo-1 Bound pvc-746227e7-a6e0-11e9-9f3a-42010a800050 1Gi RWO standard 5m23s -restore-data-restore-demo-2 Bound pvc-74674656-a6e0-11e9-9f3a-42010a800050 1Gi RWO standard 5m23s +NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE +restore-data-restore-demo-0 Bound pvc-5ab92607-0c02-4763-b0ff-d1526d5aa845 1Gi RWO csi-standard 89s +restore-data-restore-demo-1 Bound pvc-93b0305f-500d-4d29-a8e4-661325bccabd 1Gi RWO csi-standard 66s +restore-data-restore-demo-2 Bound pvc-8632d3a2-e6a7-4104-86e6-1a90957be476 1Gi RWO csi-standard 44s ``` +Notice the `STATUS` field. It indicates that the respective PV has been provisioned and initialized from the respective VolumeSnapshot by CSI driver and the PVC has been bound with the PV. + +>The [volumeBindingMode](https://kubernetes.io/docs/concepts/storage/storage-classes/#volume-binding-mode) field controls when volume binding and dynamic provisioning should occur. Kubernetes allows `Immediate` and `WaitForFirstConsumer` modes for binding volumes. The `Immediate` mode indicates that volume binding and dynamic provisioning occurs once the PVC is created and `WaitForFirstConsumer` mode indicates that volume binding and provisioning does not occur until a pod is created that uses this PVC. By default `volumeBindingMode` is `Immediate`. + +>If you use `volumeBindingMode: WaitForFirstConsumer`, respective PVC will be initialized from respective VolumeSnapshot after you create a workload with that PVC. In this case, KubeStash will mark the restore session as completed with phase `Unknown`. + **Verify Restored Data :** -We are going to create a new Statefulset to verify whether the restored data has been restored successfully. +We are going to create a new Statefulset with the restored PVCs to verify whether the backed up data has been restored. Below, the YAML for the Statefulset we are going to create. @@ -760,8 +599,8 @@ metadata: namespace: demo spec: ports: - - port: 80 - name: web + - port: 80 + name: web clusterIP: None selector: app: restore-demo @@ -783,57 +622,57 @@ spec: app: restore-demo spec: containers: - - args: - - sleep - - "3600" - name: nginx - image: nginx - ports: - - containerPort: 80 - name: web - volumeMounts: - - name: restore-data - mountPath: /restore/data + - args: + - sleep + - "3600" + name: nginx + image: nginx + ports: + - containerPort: 80 + name: web + volumeMounts: + - name: restore-data + mountPath: /restore/data volumeClaimTemplates: - metadata: name: restore-data namespace: demo spec: accessModes: - - ReadWriteOnce - storageClassName: standard + - ReadWriteOnce + storageClassName: csi-standard resources: requests: storage: 1Gi ``` -Let's create the Statefulset we have shown above. +Let's create the StatefulSet we have shown above. ```bash - kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/statefulset/examples/restored-statefulset.yaml -service/svc created +$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/volumesnapshot/statefulset/examples/restored-statefulset.yaml +service/restore-svc created statefulset.apps/restore-demo created ``` -Now, wait for the pod of Statefulset to go into the `Running` state. +Now, wait for the pod of the Statefulset to go into the `Running` state. ```bash $ kubectl get pod -n demo NAME READY STATUS RESTARTS AGE -restore-demo-0 1/1 Running 0 3m9s -restore-demo-1 1/1 Running 0 2m50s -restore-demo-2 1/1 Running 0 2m30s +restore-demo-0 1/1 Running 0 65s +restore-demo-1 1/1 Running 0 46s +restore-demo-2 1/1 Running 0 26s ``` Verify that the backed up data has been restored in `/restore/data` directory using the following command, ```bash $ kubectl exec -n demo restore-demo-0 -- cat /restore/data/data.txt -stash-demo-0 +kubestash-demo-0 $ kubectl exec -n demo restore-demo-1 -- cat /restore/data/data.txt -stash-demo-0 +kubestash-demo-1 $ kubectl exec -n demo restore-demo-2 -- cat /restore/data/data.txt -stash-demo-0 +kubestash-demo-2 ``` ## Cleaning Up @@ -841,10 +680,14 @@ stash-demo-0 To clean up the Kubernetes resources created by this tutorial, run: ```bash -kubectl delete -n demo statefulset stash-demo +kubectl delete -n demo statefulset kubestash-demo kubectl delete -n demo statefulset restore-demo kubectl delete -n demo backupconfiguration statefulset-volume-snapshot +kubectl delete -n demo retentionpolicy demo-retention +kubectl delete -n demo gcs-storage kubectl delete -n demo restoresession restore-pvc +kubectl delete -n demo pvc --all +kubectl delete -n demo volumesnapshots --all kubectl delete -n demo storageclass csi-standard kubectl delete -n demo volumesnapshotclass csi-snapshot-class ```