docs: move older design docs into the git repo

Signed-off-by: Tiago Castro <[email protected]>

doc/design/k8s/diskpool-cr.md

@@ -0,0 +1,46 @@
+# DiskPool Custom Resource for K8s
+
+The DiskPool operator is a [K8s] specific component which managed pools in a K8s environment. \
+Simplistically, it drives pools between into the various states listed below.
+
+In [K8s], mayastor pools are represented as [Custom Resources][k8s-cr], which is an extension on top of the existing [K8s API][k8s-api]. \
+This allows users to declarative create [diskpool], and mayastor will not only eventually create the corresponding mayastor pool but will
+also ensure that it gets re-imported after pod restarts, node restarts, crashes, etc...
+
+> **NOTE**: mayastor pool (msp) has been renamed to diskpool (dsp)
+
+## DiskPool States
+
+> *NOTE*
+> Non-exhaustive enums could have additional variants added in the future. Therefore, when matching against variants of non-exhaustive enums, an extra > > wildcard arm must be added to account for future variants.
+
+- Creating \
+The pool is a new OR missing resource, and it has not been created or imported yet. The pool spec ***MAY*** be present but ***DOES NOT*** have a status field.
+
+- Created \
+The pool has been created in the designated i/o engine node by the control-plane.
+
+- Terminating \
+A deletion request has been issued by the user. The pool will eventually be deleted by the control-plane and eventually the DiskPool Custom Resource will also get removed from the K8s API.
+
+- Error (*Deprecated*) \
+Trying to converge to the next state has exceeded the maximum retry counts. The retry counts are implemented using an exponential back-off, which by default is set to 10. Once the error state is entered, reconciliation stops. Only external events (a new resource version) will trigger a new attempt. \
+  > NOTE: this State has been deprecated since API version **v1beta1**
+
+## Reconciler actions
+
+The operator responds to two types of events:
+
+- Scheduled \
+When, for example, we try to submit a new PUT request for a pool. On failure (i.e., network) we will reschedule the operation after 5 seconds.
+
+- CRD updates \
+When the CRD is changed, the resource version is changed. This will trigger a new reconcile loop. This process is typically known as “watching.”
+
+- Observability \
+During the transition, the operator will emit events to K8s, which can be obtained by kubectl. This gives visibility into the state and its transitions.
+
+[K8s]: https://kubernetes.io/
+[k8s-cr]: https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/
+[k8s-api]: https://kubernetes.io/docs/concepts/overview/kubernetes-api/
+[diskpool]: https://openebs.io/docs/user-guides/replicated-storage-user-guide/replicated-pv-mayastor/rs-configuration

doc/lvm.md → doc/design/lvm.md

@@ -98,9 +98,9 @@ graph TD;
    end
 
    subgraph Physical Volumes
-      PV_1 --> VG_1["Volume Group - VG 1"]
-      PV_2 --> VG_1
-      PV_3 --> VG_2["Volume Group - VG 2"]
+      PV_1["PV 1"] --> VG_1["Vol Group 1"]
+      PV_2["PV 2"] --> VG_1
+      PV_3["PV 3"] --> VG_2["Vol Group 2"]
    end
 
    subgraph Node1

doc/design/persistent-store.md

@@ -0,0 +1,63 @@
+# Persistent Configuration Storage for the control-plane
+
+The Mayastor Control Plane requires a persistent store for storing information about most useful things like nodes, pools, volumes, etc..
+
+A key-value store has been selected as the appropriate type of store. More specifically [etcd] will be used.
+
+<br>
+
+## etcd
+
+<br>
+
+etcd is widely used and is a fundamental component of Kubernetes itself. As such, it has been “battle hardened” in production making it a reasonable first choice for storing configuration.
+
+<br>
+
+> NOTE from their own documentation:
+>
+> etcd is designed to reliably store infrequently updated data…
+
+<br>
+
+This limitation is acceptable for the control plane as, by design, we shouldn’t be storing information at anywhere near the limits of etcd, given we
+want to use this store for configuration and not the volume data itself.
+
+Given all of the above, if there is a justifiable reason for moving away from etcd, the implementation should make this switch simple.
+
+<br>
+
+## Persistent Information
+
+<br>
+
+There are two categories of information that the control plane wishes to store:
+
+1. System state
+   - Volume states
+   - Node states
+   - Pool states
+
+2. Per volume policies
+   - Replica replacement policy
+   - Nexus replacement policy
+
+<br>
+
+### System State
+
+The control plane requires visibility of the state of the system in order to make autonomous decisions. \
+For example, should a volume transition from a healthy state to a degraded state, the control plane could inspect the state of its children and
+optionally (based on the policy) replace any that are unhealthy.
+
+Additionally, this state information would be useful for implementing an early warning system. If any resource (volume, node, pool) changed state,
+any etcd watchers would be notified. \
+We could then potentially have a service which watches for state changes and notifies the upper layers (i.e. operators) that an error has occurred.
+
+### Per Volume Policies
+
+When creating a volume, the REST API allows a set of nodes to be supplied which denotes the placement of nexuses/replicas. This information is placed in the persistent store and is used as the basis for the replacement policy.
+
+Should a volume become degraded, the control plane can look up the unhealthy replica, the nodes that replicas are allowed to be placed on (the policy) and can replace the unhealthy replica with a new one.
+
+[etcd]: https://etcd.io/docs