126 lines
4.4 KiB
Markdown
126 lines
4.4 KiB
Markdown
# contracts/skeleton-topology.md — 骨骼拓扑契约 v0.1.0
|
||
|
||
**唯一拓扑：MediaPipe Pose 33 点。禁止引入 SMPL 及其衍生拓扑（许可风险）。**
|
||
|
||
## 1. 33 点定义与索引
|
||
|
||
| idx | name | 中文 | 说明 |
|
||
|---|---|---|---|
|
||
| 0 | nose | 鼻 | |
|
||
| 1 | left_eye_inner | 左眼内 | |
|
||
| 2 | left_eye | 左眼 | |
|
||
| 3 | left_eye_outer | 左眼外 | |
|
||
| 4 | right_eye_inner | 右眼内 | |
|
||
| 5 | right_eye | 右眼 | |
|
||
| 6 | right_eye_outer | 右眼外 | |
|
||
| 7 | left_ear | 左耳 | |
|
||
| 8 | right_ear | 右耳 | |
|
||
| 9 | mouth_left | 左嘴角 | |
|
||
| 10 | mouth_right | 右嘴角 | |
|
||
| 11 | left_shoulder | 左肩 | |
|
||
| 12 | right_shoulder | 右肩 | |
|
||
| 13 | left_elbow | 左肘 | |
|
||
| 14 | right_elbow | 右肘 | |
|
||
| 15 | left_wrist | 左腕 | |
|
||
| 16 | right_wrist | 右腕 | |
|
||
| 17 | left_pinky | 左小指 | |
|
||
| 18 | right_pinky | 右小指 | |
|
||
| 19 | left_index | 左食指 | |
|
||
| 20 | right_index | 右食指 | |
|
||
| 21 | left_thumb | 左拇指 | |
|
||
| 22 | right_thumb | 右拇指 | |
|
||
| 23 | left_hip | 左髋 | |
|
||
| 24 | right_hip | 右髋 | |
|
||
| 25 | left_knee | 左膝 | |
|
||
| 26 | right_knee | 右膝 | |
|
||
| 27 | left_ankle | 左踝 | |
|
||
| 28 | right_ankle | 右踝 | |
|
||
| 29 | left_heel | 左足跟 | |
|
||
| 30 | right_heel | 右足跟 | |
|
||
| 31 | left_foot_index | 左足尖 | |
|
||
| 32 | right_foot_index | 右足尖 | |
|
||
|
||
## 2. 虚拟关节（由 33 点派生，索引 33/34，仅用于层级与角度特征）
|
||
|
||
| idx | name | 计算 |
|
||
|---|---|---|
|
||
| 33 | neck | `midpoint(left_shoulder(11), right_shoulder(12))` |
|
||
| 34 | pelvis | `midpoint(left_hip(23), right_hip(24))` |
|
||
|
||
## 3. 父节点表（`parent[i]`，-1 表示世界根）
|
||
|
||
对称树，便于骨骼重定向与关节角计算：
|
||
|
||
```
|
||
34 pelvis -> -1 (world root)
|
||
33 neck -> 34
|
||
23 left_hip -> 34
|
||
24 right_hip -> 34
|
||
11 left_shoulder -> 33
|
||
12 right_shoulder -> 33
|
||
0 nose -> 33
|
||
1 left_eye_inner -> 0
|
||
2 left_eye -> 0
|
||
3 left_eye_outer -> 0
|
||
4 right_eye_inner -> 0
|
||
5 right_eye -> 0
|
||
6 right_eye_outer -> 0
|
||
7 left_ear -> 3
|
||
8 right_ear -> 6
|
||
9 mouth_left -> 0
|
||
10 mouth_right -> 0
|
||
25 left_knee -> 23
|
||
26 right_knee -> 24
|
||
27 left_ankle -> 25
|
||
28 right_ankle -> 26
|
||
29 left_heel -> 27
|
||
30 right_heel -> 28
|
||
31 left_foot_index -> 29
|
||
32 right_foot_index -> 30
|
||
13 left_elbow -> 11
|
||
14 right_elbow -> 12
|
||
15 left_wrist -> 13
|
||
16 right_wrist -> 14
|
||
17 left_pinky -> 15
|
||
18 right_pinky -> 16
|
||
19 left_index -> 15
|
||
20 right_index -> 16
|
||
21 left_thumb -> 15
|
||
22 right_thumb -> 16
|
||
```
|
||
|
||
## 4. 坐标系
|
||
|
||
- **归一化图像坐标**：`x, y ∈ [0, 1]`，原点左上角，`y` 向下；`x` 相对图像宽，`y` 相对图像高。
|
||
- **深度**：`z` 与 `x` 同尺度，原点位于髋中心，**负值朝向相机**。
|
||
- **镜像**：Android 前置摄像头默认输出镜像画面；是否镜像由帧头 `flags.bit0` 声明，
|
||
接收端据此决定是否翻转 `x`（`x' = 1 - x`）。
|
||
- **置信度**：`visibility ∈ [0, 1]`，落盘/传输为 `uint8`（`round(v * 255)`）。
|
||
归一化比例统一使用 `32767`（见 `websocket-keypoints.md`）。
|
||
|
||
## 5. 派生特征（DTW 输入，Windows 端实现，服务端预处理产物同构）
|
||
|
||
服务端 T-WEB-006 预处理产出的关键点序列与 Android 端输出的帧**结构完全一致**，
|
||
因此 Windows 端 DTW 引擎只有一条输入通路。
|
||
|
||
关节角特征（弧度，余弦值便于插值）：
|
||
|
||
```
|
||
shoulder_L = angle(left_elbow, left_shoulder, left_hip)
|
||
shoulder_R = angle(right_elbow, right_shoulder, right_hip)
|
||
elbow_L = angle(left_shoulder, left_elbow, left_wrist)
|
||
elbow_R = angle(right_shoulder, right_elbow, right_wrist)
|
||
hip_L = angle(left_shoulder, left_hip, left_knee)
|
||
hip_R = angle(right_shoulder, right_hip, right_knee)
|
||
knee_L = angle(left_hip, left_knee, left_ankle)
|
||
knee_R = angle(right_hip, right_knee, right_ankle)
|
||
neck = angle(nose, neck, pelvis)
|
||
spine = angle(neck, pelvis, midpoint(left_ankle, right_ankle))
|
||
```
|
||
|
||
`angle(a, b, c)` = 向量 `ba` 与 `bc` 的夹角余弦，即
|
||
`dot(normalize(a-b), normalize(c-b))`，落在 `[-1, 1]`。
|
||
|
||
**DTW 序列结构（冻结）**：每帧 = `[33 × (x, y, z), 9 个关节角余弦] = 108 维`。
|
||
低置信度点（`visibility < 0.5`）在插值阶段由相邻帧线性补全，不参与代价计算。
|