Skip to content

Commit

Permalink
incorporate with review changes
Browse files Browse the repository at this point in the history
Signed-off-by: Anisur Rahman <[email protected]>
  • Loading branch information
anisurrahman75 committed Mar 18, 2024
1 parent 876a7d6 commit f6cc5cd
Show file tree
Hide file tree
Showing 32 changed files with 503 additions and 337 deletions.
2 changes: 1 addition & 1 deletion docs/guides/backends/azure/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ Create the `BackupStorage` we have shown above using the following command,

```bash
$ kubectl apply -f https://github.com/kubestash/docs/raw/{{< param "info.version" >}}/docs/guides/backends/azure/examples/azure.yaml
backupstorage.storage.kubestash.com/azure-storaqge created
backupstorage.storage.kubestash.com/azure-storage created
```

Now, we are ready to use this backend to backup our desired data using KubeStash.
Expand Down
21 changes: 3 additions & 18 deletions docs/guides/backends/overview/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,9 @@ section_menu_id: guides

## BackupStorage

KubeStash supports various backends for use as a BackupStorage. It can be a cloud storage like GCS bucket, AWS S3, Azure Blob Storage etc. or a Kubernetes persistent volume like [HostPath](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath), [PersistentVolumeClaim](https://kubernetes.io/docs/concepts/storage/volumes/#persistentvolumeclaim), [NFS](https://kubernetes.io/docs/concepts/storage/volumes/#nfs) etc.
KubeStash supports various backends for storing backup data. It can be a cloud storage like GCS bucket, AWS S3, Azure Blob Storage etc. or a Kubernetes persistent volume like [HostPath](https://kubernetes.io/docs/concepts/storage/volumes/#hostpath), [PersistentVolumeClaim](https://kubernetes.io/docs/concepts/storage/volumes/#persistentvolumeclaim), [NFS](https://kubernetes.io/docs/concepts/storage/volumes/#nfs) etc.

The following diagram shows how kubestash operator and backup container accesses and backs up data into a backend.
The following diagram shows how KubeStash operator and backup container accesses and backs up data into a backend.

<figure align="center">
<img alt="KubeStash Backend Overview" src="images/kubestash_backend_overview.svg">
Expand All @@ -30,7 +30,7 @@ The following diagram shows how kubestash operator and backup container accesses
The Backend process works in the following steps:

- At first user creates a [BackupStorage](/docs/concepts/crds/backupstorage/index.md) object that contains the backend information along with a `Secret` object containing the corresponding backend credentials required for accessing the backend.
- KubeStash operator watches for `BackupStorage` custom resources and `Secrets`. When it finds a `BackupStorage` object, it establishes a connection with the storage by uploading the `metadata.yaml` file into the target storage.
- KubeStash operator watches for `BackupStorage` custom resources and `Secrets`. When it finds a `BackupStorage` object, it initializes the storage by uploading the `metadata.yaml` file.

Below, a screenshot that shows initialization of a `BackupStorage` in a GCS bucket named `kubestash-qa`:

Expand All @@ -41,21 +41,6 @@ Below, a screenshot that shows initialization of a `BackupStorage` in a GCS buck

Here, `kubestash-qa` serves as the bucket name, and the presence of `metadata.yaml` indicates the successful initialization of the BackupStorage.

## Repository

- Once the `BackupStorage` is initialized and in ready phase then the next steps is creating [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md).
- User creates a `BackupConfiguration` custom resource. The `BackupConfiguration` object specifies the `Repository` pointing to a `BackupStorage` that contains backend information, indicating where to upload backup data. Multiple `Repository` objects can refer to a single `BackupStorage`.
- Then KubeStash operator retrieves `Repository` information from it and create `Repository` object. This Repository object serves as a container for effectively managing and storing the backup data.
- KubeStash operator triggers a `Backup` job according to the schedule defined in the `BackupConfiguration`. This job accesses the `BackupStorage` object and `Secret` and uploads backup data into the storage.

Below, a screenshot the shows a `Repository` located in the directory path `kubestash-qa/demo/monodb-backup/`, and the presence of `repository.yaml` indicates the successful initialization of the Repository.

<figure align="center">
  <img alt="Repository under a BackupStorage" src="./images/gcs_repository.png">
<figcaption align="center">Fig: Repository under a BackupStorage</figcaption>
</figure>


## Next Steps
- Learn how to configure `Kubernetes Volume` as backend from [here](/docs/guides/backends/local/index.md).
- Learn how to configure `AWS S3/Minio/Rook` backend from [here](/docs/guides/backends/s3/index.md).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ spec:
backend: gcs-backend
directory: /deployment-manifests
encryptionSecret:
name: encryption-secret
name: encrypt-secret
namespace: demo
deletionPolicy: WipeOut
addon:
Expand Down
81 changes: 36 additions & 45 deletions docs/guides/kubedump/application/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ You have to be familiar with the following custom resources:
- [BackupStorage](/docs/concepts/crds/backupstorage/index.md)
- [BackupConfiguration](/docs/concepts/crds/backupconfiguration/index.md)
- [BackupSession](/docs/concepts/crds/backupsession/index.md)
- [SnapShot](/docs/concepts/crds/snapshot/index.md)
- [Snapshot](/docs/concepts/crds/snapshot/index.md)
- [RestoreSession](/docs/concepts/crds/restoresession/index.md)
- [RetentionPolicy](/docs/concepts/crds/retentionpolicy/index.md)

Expand All @@ -44,13 +44,13 @@ namespace/demo created
#### Prepare Backend

Now, we are going backup of the YAMLs of an application in a GCS bucket using KubeStash. We have to create a `Secret` with necessary credentials and a `BackupStorage` object to use this backend. If you want to use a different backend, please read the respective backend configuration doc from [here](/docs/guides/backends/overview/index.md).
Now, we are going backup of the YAMLs of an application to a GCS bucket using KubeStash. For this, we have to create a `Secret` with necessary credentials and a `BackupStorage` object. If you want to use a different backend, please read the respective backend configuration doc from [here](/docs/guides/backends/overview/index.md).

> For GCS backend, if the bucket does not exist, KubeStash needs `Storage Object Admin` role permissions to create the bucket. For more details, please check the following [guide](/docs/guides/backends/gcs/index.md).
**Create Secret:**

At first, let's create a secret called `gcs-secret` with access credentials to our desired GCS bucket,
Let's create a Secret named `gcs-secret` with access credentials of our desired GCS backend,

```bash
$ echo -n '<your-project-id>' > GOOGLE_PROJECT_ID
Expand Down Expand Up @@ -119,7 +119,7 @@ spec:
from: Same
```

Notice the `spec.usagePolicy` that allows referencing the `RetentionPolicy` from all namespaces. To allow specific namespaces, we can configure it accordingly by following [RetentionPolicy usage policy](/docs/concepts/crds/retentionpolicy/index.md#retentionpolicy-spec).
Notice the `spec.usagePolicy` that allows referencing the `RetentionPolicy` from all namespaces.For more details on configuring it for specific namespaces, please refer to the following [RetentionPolicy usage policy](/docs/concepts/crds/retentionpolicy/index.md).

Let's create the `RetentionPolicy` object that we have shown above,

Expand All @@ -130,7 +130,7 @@ retentionpolicy.storage.kubestash.com/demo-retention created

#### Create RBAC

The `kubedump` plugin requires read permission for the application resources. By default, KubeStash does not grant such permissions. We have to provide the necessary permissions manually.
To take backup of the resource YAMLs of an application KubeStash creates a backup `Job`. This `Job` requires read permission for the application resources. By default, KubeStash does not grant such permissions. We have to provide the necessary permissions manually.

Here, is the YAML of the `ServiceAccount`, `ClusterRole`, and `ClusterRoleBinding` that we are going to use for granting the necessary permissions.

Expand Down Expand Up @@ -179,34 +179,35 @@ Now, we are ready for backup. In the next section, we are going to schedule a ba

### Backup

To schedule a backup, we have to create a `BackupConfiguration` object.
Now, we have to create a `BackupConfiguration` custom resource targeting an application.

At first, lets list available Deployment in `kubestash` namespace.
Lets list available `Deployment` in `kubestash` namespace.

```bash
$ kubectl get deployments -n kubestash
NAME READY UP-TO-DATE AVAILABLE AGE
kubestash-kubestash-operator 1/1 1 1 19h
```

Here, we are going to setup backup YAMLs for `kubestash-kubestash-operator` Deployment.
Here, we are going to take backup YAMLs for `kubestash-kubestash-operator` Deployment.

we also have to create another `Secret` with an encryption key `RESTIC_PASSWORD` for `Restic`. This secret will be used by `Restic` for both encrypting and decrypting the backup data during backup & restore.

**Create Secret:**

We also have to create another `Secret` with an encryption key `RESTIC_PASSWORD` for `Restic`. This secret will be used by `Restic` for both encrypting and decrypting the backup data during backup & restore.

Let's create a secret named `encry-secret` with the Restic password.

```bash
$ echo -n 'changeit' > RESTIC_PASSWORD
$ kubectl create secret generic -n demo encryption-secret \
$ kubectl create secret generic -n demo encrypt-secret \
--from-file=./RESTIC_PASSWORD
secret/encryption-secret created
secret/encrypt-secret created
```

#### Create BackupConfiguration

Below is the YAML for `BackupConfiguration` object we care going to use to backup the YAMLs of the cluster resources,
Below is the YAML of the `BackupConfiguration` object we are going to create to backup the YAMLs of the cluster resources,

```yaml
apiVersion: core.kubestash.com/v1alpha1
Expand Down Expand Up @@ -240,7 +241,7 @@ spec:
backend: gcs-backend
directory: /deployment-manifests
encryptionSecret:
name: encryption-secret
name: encrypt-secret
namespace: demo
deletionPolicy: WipeOut
addon:
Expand Down Expand Up @@ -268,27 +269,21 @@ backupconfiguration.core.kubestash.com/application-manifest-backup created

#### Verify Backup Setup Successful

If everything goes well, the phase of the `BackupConfiguration` should be `Ready`. The `Ready` phase indicates that the backup setup is successful. Let's verify the `Phase` of the BackupConfiguration,
If everything goes well, the phase of the `BackupConfiguration` should be in `Ready` state. The `Ready` phase indicates that the backup setup is successful.

Let's check the `Phase` of the BackupConfiguration

```bash
kubectl get backupconfiguration -n demo
$ kubectl get backupconfiguration -n demo
NAME PHASE PAUSED AGE
application-manifest-backup Ready 19s
```

**Verify Repository:**

Verify that KubeStash has created `Repositories` that holds the `BackupStorage` information by the following command,

```bash
$ kubectl get repositories -n demo
NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE
gcs-repository Ready 28s
```

**Verify CronJob:**

Verify that KubeStash has created a CronJob to trigger a periodic backup of the targeted PVC by the following command,
Verify that KubeStash has created a `CronJob` with the schedule specified in `spec.sessions[*].scheduler.schedule` field of `BackupConfiguration` object.

Check that the `CronJob` has been created using the following command,

```bash
$ kubectl get cronjob -n demo
Expand All @@ -298,7 +293,7 @@ trigger-application-manifest-backup-frequent-backup */5 * * * * False 0

**Wait for BackupSession:**

Now, wait for the next backup schedule. You can watch for `BackupSession` crd using the following command,
Now, wait for the next backup schedule. You can watch for `BackupSession` CR using the following command,

```bash
$ watch -n 1 kubectl get backupsession -n demo -l=kubestash.com/invoker-name=application-manifest-backup
Expand All @@ -313,27 +308,24 @@ application-manifest-backup-frequent-backup-1708677300 BackupConfiguration a

When backup session is created, KubeStash operator creates `Snapshot` which represents the state of backup run for each `Repository` which are provided in `BackupConfiguration`.

Run the following command to check `Snapshot` phase,

```bash
$ kubectl get snapshots -n demo
NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE AGE
gcs-repository-application-manifckup-frequent-backup-1708677300 gcs-repository frequent-backup 2024-02-23T08:35:00Z Delete Succeeded 43s
$ kubectl get repositories -n demo
NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE
gcs-repository true 1 2.262 KiB Ready 103s 8m
```

When backup session is completed, KubeStash will update the respective `Repository` to reflect the latest state of backed up data.
At this moment we have one `Snapshot`. Run the following command to check the respective `Snapshot`.

Run the following command to check if a backup snapshot has been stored in the backend,
Verify created `Snapshot` object by the following command,

```bash
$ kubectl get repositories -n demo
NAME INTEGRITY SNAPSHOT-COUNT SIZE PHASE LAST-SUCCESSFUL-BACKUP AGE
gcs-repository true 1 2.262 KiB Ready 103s 8m
$ kubectl get snapshots -n demo
NAME REPOSITORY SESSION SNAPSHOT-TIME DELETION-POLICY PHASE AGE
gcs-repository-application-manifckup-frequent-backup-1708677300 gcs-repository frequent-backup 2024-02-23T08:35:00Z Delete Succeeded 43s
```

From the output above, we can see that `1` snapshot has been stored in the backend specified by Repository `gcs-repository`.

If we navigate to `kubestash-qa/demo/deployment-manifests/repository/v1/frequent-backup/manifest` directory of our GCS bucket, we are going to see that the snapshot has been stored there.
Now, If we navigate to `kubestash-qa/demo/deployment-manifests/repository/v1/frequent-backup/manifest` directory of our GCS bucket, we are going to see that the snapshot has been stored there.

<figure align="center">
  <img alt="Backup YAMLs data of an Application in GCS storage" src="/docs/guides/kubedump/application/images/application_manifest_backup.png">
Expand All @@ -342,7 +334,6 @@ If we navigate to `kubestash-qa/demo/deployment-manifests/repository/v1/frequent

> KubeStash keeps all backup data encrypted. So, snapshot files in the bucket will not contain any meaningful data until they are decrypted.


## Restore

KubeStash does not provide any automatic mechanism to restore the cluster resources from the backed-up YAMLs. Your application might be managed by Helm or by an operator. In such cases, just applying the YAMLs is not enough to restore the application. Furthermore, there might be an order issue. Some resources must be applied before others. It is difficult to generalize and codify various application-specific logic.
Expand All @@ -356,13 +347,13 @@ KubeStash provides a [kubectl plugin](https://stash.run/docs/v2022.05.12/guides/
Now, let's download the latest Snapshot from our backed-up data into the `$HOME/Downloads/stash/applications/kube-system/stash-enterprise` folder of our local machine.

```bash
kubectl kubestash download gcs-repository-application-manifckup-frequent-backup-1708677300 --namespace=demo --destination=$HOME/Downloads/kubestash/applications/kubestash/kubestash-kubestash-operator
$ kubectl kubestash download gcs-repository-application-manifckup-frequent-backup-1708677300 --namespace=demo --destination=$HOME/Downloads/kubestash/applications/kubestash/kubestash-kubestash-operator
```

Now, lets use [tree](https://linux.die.net/man/1/tree) command to inspect downloaded YAMLs files.

```bash
tree $HOME/Downloads/kubestash/applications/kubestash/kubestash-kubestash-operator
$ tree $HOME/Downloads/kubestash/applications/kubestash/kubestash-kubestash-operator
/home/anisur/Downloads/kubestash/applications/kubestash/kubestash-kubestash-operator
└── gcs-repository-application-manifckup-frequent-backup-1708677300
└── manifest
Expand All @@ -385,7 +376,7 @@ As you can see that the Deployment has been backed up along with it's ReplicaSet
Let's inspect the YAML of `kubestash-kubestash-operator.yaml` file,

```yaml
cat /home/anisur/Downloads/kubestash/applications/kubestash/kubestash-kubestash-operator/gcs-repository-application-manifckup-frequent-backup-1708677300/manifest/tmp/manifest/kubestash-kubestash-operator.yaml
$ cat /home/anisur/Downloads/kubestash/applications/kubestash/kubestash-kubestash-operator/gcs-repository-application-manifckup-frequent-backup-1708677300/manifest/tmp/manifest/kubestash-kubestash-operator.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
Expand Down Expand Up @@ -527,6 +518,6 @@ kubectl delete clusterrole cluster-resource-reader
kubectl delete clusterrolebinding cluster-resource-reader
kubectl delete retentionPolicy -n demo demo-retention
kubectl delete -n demo backupstorage gcs-storage
kubectl delete secret -n demo encryption-secret
kubectl delete secret -n demo encrypt-secret
kubectl delete secret -n demo gcs-secret
```
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ spec:
backend: gcs-backend
directory: /cluster-manifests
encryptionSecret:
name: encryption-secret
name: encrypt-secret
namespace: demo
deletionPolicy: WipeOut
addon:
Expand Down
Loading

0 comments on commit f6cc5cd

Please sign in to comment.