Understanding Harvester Reserved Resources Calculation

Signed-off-by: Jian Wang <[email protected]>
harvester · Jan 23, 2024 · 20e4140 · 20e4140
1 parent cb6902f
commit 20e4140
Show file tree

Hide file tree

Showing 3 changed files with 204 additions and 0 deletions.
diff --git a/kb/2024-01-23/imgs/cluster-resource-usage.png b/kb/2024-01-23/imgs/cluster-resource-usage.png
diff --git a/kb/2024-01-23/imgs/host-resource-usage.png b/kb/2024-01-23/imgs/host-resource-usage.png
diff --git a/kb/2024-01-23/reserved_resources_calculation.md b/kb/2024-01-23/reserved_resources_calculation.md
@@ -0,0 +1,204 @@
+---
+title: Understanding Harvester Reserved Resources Calculation
+description: Know hwo the resources reseravation is made and calculated
+slug: harvester_reserved_resources_calculation
+authors:
+  - name: Jian Wang
+    title: Staff Software Engineer
+    url: https://github.com/w13915984028
+    image_url: https://github.com/w13915984028.png
+tags: [harvester, reserved resources, calculation]
+hide_table_of_contents: false
+---
+
+# Understanding Harvester Reserved Resources Calculation
+
+In Harvester Web UI menu `Dashboard`, you can view the cluster level resources usage.
+
+![cluster level resources usage](./imgs/cluster-resource-usage.png).
+
+In Harvester Web UI menu `Hosts`, you can also view the host level resources usage.
+
+![host level resources usage](./imgs/host-resource-usage.png).
+
+How the resources usages are computed?
+
+Undoubtly the resources usages are computed from a series of data collected from the system dynamically. This article will reveal all the details under the hood.
+
+## Host Level Resource Usage
+
+### CPU and Memory Usage
+
+#### Data Source
+
+##### Resources Capacity
+
+In Kubernetes, a `Node` object is created for each host.
+
+The `.status.allocatable.cpu` and `.status.allocatable.memory` represent the available CPU and Memory resources of a host.
+
+
+```
+# kubectl get nodes -A -oyaml
+apiVersion: v1
+items:
+- apiVersion: v1
+  kind: Node
+  metadata:
+..
+      management.cattle.io/pod-limits: '{"cpu":"12715m","devices.kubevirt.io/kvm":"1","devices.kubevirt.io/tun":"1","devices.kubevirt.io/vhost-net":"1","memory":"17104951040"}'
+      management.cattle.io/pod-requests: '{"cpu":"5657m","devices.kubevirt.io/kvm":"1","devices.kubevirt.io/tun":"1","devices.kubevirt.io/vhost-net":"1","ephemeral-storage":"50M","memory":"9155862208","pods":"78"}'
+      node.alpha.kubernetes.io/ttl: "0"
+..
+    name: harv41
+    resourceVersion: "2170215"
+    uid: b6f5850a-2fbc-4aef-8fbe-121dfb671b67
+  spec:
+    podCIDR: 10.52.0.0/24
+    podCIDRs:
+    - 10.52.0.0/24
+    providerID: rke2://harv41
+  status:
+    addresses:
+    - address: 192.168.122.141
+      type: InternalIP
+    - address: harv41
+      type: Hostname
+    allocatable:
+      cpu: "10"
+      devices.kubevirt.io/kvm: 1k
+      devices.kubevirt.io/tun: 1k
+      devices.kubevirt.io/vhost-net: 1k
+      ephemeral-storage: "149527126718"
+      hugepages-1Gi: "0"
+      hugepages-2Mi: "0"
+      memory: 20464216Ki
+      pods: "200"
+    capacity:
+      cpu: "10"
+      devices.kubevirt.io/kvm: 1k
+      devices.kubevirt.io/tun: 1k
+      devices.kubevirt.io/vhost-net: 1k
+      ephemeral-storage: 153707984Ki
+      hugepages-1Gi: "0"
+      hugepages-2Mi: "0"
+      memory: 20464216Ki
+      pods: "200"
+```
+
+##### Resources Usage
+
+The dynamic CPU and Memory usage are collected from `NodeMetrics` object, the data is stored in `usage.cpu` and `usage.memory`.
+
+```
+# kubectl get NodeMetrics -A -oyaml
+apiVersion: v1
+items:
+- apiVersion: metrics.k8s.io/v1beta1
+  kind: NodeMetrics
+  metadata:
+...
+    name: harv41
+  timestamp: "2024-01-23T12:04:44Z"
+  usage:
+    cpu: 891736742n
+    memory: 9845008Ki
+  window: 10.149s
+```
+
+##### Resources Reservation
+
+Harvester does some smart works on the backgroud, it computes the resources requests and limits of all PODs in this host dynamically, and updates the information to annotataions of the aforementioned `Node` object.
+
+```
+      management.cattle.io/pod-limits: '{"cpu":"12715m",...,"memory":"17104951040"}'
+      management.cattle.io/pod-requests: '{"cpu":"5657m",...,"memory":"9155862208"}'
+```
+
+Refer [Kubernetes resources requests and limits](https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/#requests-and-limits) for more details.
+
+#### Calculation
+
+The `Resource Capacity` is the base data.
+The `Resource Usage` is converted to `Used`.
+The `Resource Reservation` is converted to `Reserved`.
+And the Web UI also does some final rendings.
+
+### Storage Usage
+
+The Longhorn is the default CSI of Harvester, it manages the storage.
+
+#### Reserved Storage in Longhorn
+
+The reserved percentage specifies the percentage of disk space that will not be allocated to the default disk on each new Longhorn node.
+
+This setting only affects the default disk of a new adding node or nodes when installing Longhorn.
+
+Defautl value: 30
+
+Refer [Longhorn reserved disks](https://longhorn.io/docs/1.5.3/references/settings/#storage-reserved-percentage-for-default-disk) for more details.
+
+Depends on the default disk size, you may adjust this value via [the embedded Longhorn UI](https://docs.harvesterhci.io/v1.2/troubleshooting/harvester/#access-embedded-rancher-and-longhorn-dashboards).
+
+::: note
+
+You are suggested to confirm with Harvester engineering team before change the settings.
+
+:::
+
+#### Data Source
+
+```
+# kubectl get nodes.longhorn.io -n longhorn-system -oyaml
+
+apiVersion: v1
+items:
+- apiVersion: longhorn.io/v1beta2
+  kind: Node
+  metadata:
+..
+    name: harv41
+    namespace: longhorn-system
+..
+  spec:
+    allowScheduling: true
+    disks:
+      default-disk-ef11a18c36b01132:
+        allowScheduling: true
+        diskType: filesystem
+        evictionRequested: false
+        path: /var/lib/harvester/defaultdisk
+        storageReserved: 24220101427
+        tags: []
+..
+  status:
+..
+    diskStatus:
+      default-disk-ef11a18c36b01132:
+..
+        diskType: filesystem
+        diskUUID: d2788933-8817-44c6-b688-dee414cc1f73
+        scheduledReplica:
+          pvc-95561210-c39c-4c2e-ac9a-4a9bd72b3100-r-20affeca: 2147483648
+          pvc-9e83b2dc-6a4b-4499-ba70-70dc25b2d9aa-r-4ad05c86: 32212254720
+          pvc-bc25be1e-ca4e-4818-a16d-48353a0f2f96-r-c7b88c60: 3221225472
+          pvc-d9d3e54d-8d67-4740-861e-6373f670f1e4-r-f4c7c338: 2147483648
+          pvc-e954b5fe-bbd7-4d44-9866-6ff6684d5708-r-ba6b87b6: 5368709120
+        storageAvailable: 77699481600
+        storageMaximum:   80733671424
+        storageScheduled: 45097156608
+    region: ""
+    snapshotCheckStatus: {}
+    zone: ""
+```
+
+#### Calculation
+
+Storage Capacity: The sum of `storageMaximum` of all status.diskStatus.disk-name.
+Storage Usage: `Stoage Capacity` - the sum of `storageAvailable` of all status.diskStatus.disk-name.
+Storage Reservation: The sum of `storageReserved` of all spec.disks.
+
+## Cluster Level Resources Usage
+
+All above resources in each `host`, are summarized and calculated correspondingly to get the cluster level resources usage.
+