Merge pull request #102 from taorye/dev

fix: limit keypoints_window_size to 1 for working on multi target and add docs

docs/doc/en/sidebar.yaml

@@ -85,6 +85,8 @@ items:
         label: YOLOv5 model offline training
     -   file: vision/customize_model_yolov8.md
         label: YOLO11/v8 model offline training
+    -   file: vision/body_pose_classification.md
+        label: YOLO11 pose model based classifier
     -   file: ai_model_converter/maixcam.md
         label: ONNX model to MaixCAM's
     -   file: pro/datasets.md

docs/doc/en/vision/body_pose_classification.md

@@ -0,0 +1,55 @@
+---
+title: MaixCAM MaixPy YOLO11-based Pose Estimation and Preliminary Human Posture Classification
+---
+
+
+## Introduction
+
+The `MaixCAM MaixPy Pose Estimation` can estimate `17` human keypoints.
+
+![](../../assets/body_keypoints.jpg)
+
+The connections between specific keypoints can simulate the human body, such as:
+
+```markdown
+3-1-0-2-4 form the head (Head)
+5-6-12-11-5 form the torso (Torso)
+5-7-9 or 6-8-10 form the upper limbs (Upper Limbs)
+11-13-15 or 12-14-16 form the lower limbs (Lower Limbs)
+
+"Thigh": "大腿",
+"Shin": "小腿",
+"Upper Arm": "大臂",
+"Forearm": "小臂",
+```
+
+Each limb can be represented as a vector, and the angle between different limbs can be calculated.
+
+For example, the angle between the thigh and shin can be used to determine whether the shin is straight or bent. Since a human cannot stand upright with a bent shin, similar logic can be applied to classify the posture of the human body.
+
+Current classifications include:
+
+1. "Lying Down": "躺下",
+2. "Standing Up": "直立",
+3. "Sitting Down": "坐下",
+4. "Reclining": "斜躺",
+5. "To Left 1": "向左1",
+6. "To Right 1": "向右1",
+7. "Both Hands Raised Horizontally": "双手平举",
+8. "Left Hand Raised": "举左手",
+9. "Right Hand Raised": "举右手",
+10. "Both Hands Raised Up": "举双手",
+11. "Both Hands Forming a Heart": "双手比心",
+12. "Big 'T' Shape": "大字型",
+
+
+Example Image
+![](../../assets/body_pose_classification.jpg)
+
+## Usage
+
+The app `Human Pose Classifier` packaged in `projects/app_human_pose_classifier/` can be directly run.
+
+The file `nn_yolo11_pose_cls.py` under `examples/vision/ai_vision/` is a standalone implementation and can be run directly on MaixVision by just clicking the run button.
+
+It is recommended to refer to `PoseEstimation.py` for customization.

docs/doc/zh/sidebar.yaml

@@ -85,6 +85,8 @@ items:
         label: 离线训练 YOLOv5 模型
     -   file: vision/customize_model_yolov8.md
         label: 离线训练 YOLO11/YOLOv8 模型
+    -   file: vision/body_pose_classification.md
+        label: 基于 YOLO11 pose 模型的人体姿态分类器
     -   file: ai_model_converter/maixcam.md
         label: ONNX 模型转给 MaixCAM 用
     -   file: pro/datasets.md

docs/doc/zh/vision/body_pose_classification.md

@@ -0,0 +1,57 @@
+---
+title: MaixCAM MaixPy 基于yolo11 pose估计人体关键点初步分类人体姿态
+---
+
+
+## 简介
+
+由`MaixCAM MaixPy 检测人体关键点姿态检测`可估计人体的 `17` 个关键点。
+
+![](../../assets/body_keypoints.jpg)
+
+特定关键点之间的连线可以简单模拟人的肢体，如
+
+```markdown
+3-1-0-2-4 构成 头部 (Head)
+5-6-12-11-5 构成 躯干 (Torso)
+5-7-9 或 6-8-10 构成 上肢 (Upper Limbs)
+11-13-15 或 12-14-16 构成 下肢 (Lower Limbs)
+
+"大腿": "Thigh",
+"小腿": "Shin",
+"大臂": "Upper Arm",
+"小臂": "Forearm",
+```
+
+每个肢体为一个向量，可由此计算不同肢体间夹角，如
+
+大腿和小腿间夹角，然后可以判断小腿是伸直还是弯曲，且人类不可能在小腿弯曲的情况下直立，以此类推
+
+等等，可对人体的姿态进行初步分类。
+
+当前有以下几个分类：
+
+1. "躺下": "Lying Down",
+2. "直立": "Standing Up",
+3. "坐下": "Sitting Down",
+4. "斜躺": "Reclining",
+5. "向左1": "To Left 1",
+6. "向右1": "To Right 1",
+7. "双手平举": "Both Hands Raised Horizontally",
+8. "举左手": "Left Hand Raised",
+9. "举右手": "Right Hand Raised",
+10. "举双手": "Both Hands Raised Up",
+11. "双手比心": "Both Hands Forming a Heart",
+12. "大字型": "Big 'T' Shape",
+
+
+示例图
+![](../../assets/body_pose_classification.jpg)
+
+## 使用
+
+`projects/app_human_pose_classifier/` 打包的 app `Human Pose Classifier` 可直接运行。
+
+`examples/vision/ai_vision/` 下的 `nn_yolo11_pose_cls.py` 是单单文件实现，可以在 MaixVision 直接点击 run 按钮运行。
+
+建议参考 `PoseEstimation.py` 进行定制修改。

projects/app_human_pose_classifier/PoseEstimation.py

@@ -2,7 +2,8 @@
 from collections import deque
 
 class PoseEstimation:
-    def __init__(self, keypoints_window_size=5):
+    # need redesign, PoseEstimation cant work on multi target unless keypoints_window_size=1
+    def __init__(self, keypoints_window_size=1):
         self.keypoints_map_deque = deque(maxlen=keypoints_window_size)
         self.status = []