604 lines
17 KiB
Markdown
604 lines
17 KiB
Markdown
# Dokumentasi Struktur Folder `lib/`
|
||
|
||
## Gambaran Umum
|
||
|
||
Folder `lib/` berisi semua utility libraries, helper functions, dan core logic aplikasi. Ini adalah "otak" dari aplikasi yang menangani AI pose detection, exercise recognition, authentication, dan database operations.
|
||
|
||
---
|
||
|
||
## 📁 Struktur Direktori
|
||
|
||
```
|
||
lib/
|
||
├── auth.tsx # Authentication Context & Utilities
|
||
├── mediapipe-shim.js # MediaPipe Polyfill
|
||
├── prisma.ts # Prisma Client Instance
|
||
├── pose/ # AI & Exercise Recognition Engine
|
||
│ ├── ExerciseRules.ts # Exercise Configuration & Rules
|
||
│ ├── HARCore.ts # Human Activity Recognition Core
|
||
│ ├── MathUtils.ts # Mathematical Utilities
|
||
│ ├── RehabCore.ts # Exercise Recognition & Counting
|
||
│ ├── RehabFSM.ts # Finite State Machines for Reps
|
||
│ ├── RepetitionCounter.ts # Legacy Rep Counter
|
||
│ └── XGBoostPredictor.ts # ML Model Predictor
|
||
├── prisma/ # Prisma Generated Client (Symlink)
|
||
└── prisma-gen/ # Prisma Generated Types
|
||
```
|
||
|
||
---
|
||
|
||
## 📄 File Root Level
|
||
|
||
### **`auth.tsx`**
|
||
|
||
**Fungsi**: Context Provider untuk authentication dan session management
|
||
|
||
**Isi Utama**:
|
||
|
||
- **Interface**:
|
||
- `User`: Type definition untuk user object (id, name, role, coach_id)
|
||
- `AuthContextType`: Type untuk auth context
|
||
- `UserRole`: 'COACH' | 'CLIENT'
|
||
|
||
- **Context**:
|
||
- `AuthContext`: React Context untuk state auth global
|
||
- `AuthProvider`: Provider component yang wrap seluruh app
|
||
|
||
- **Methods**:
|
||
- `login(userId: string)`: Login user berdasarkan ID, fetch data dari `/api/users/[id]`
|
||
- `logout()`: Clear session, hapus localStorage, reset state
|
||
- Auto-load session dari localStorage saat app mount
|
||
|
||
**Tech**: React Context API, localStorage persistence
|
||
|
||
**Size**: ~1.8 KB
|
||
|
||
---
|
||
|
||
### **`mediapipe-shim.js`**
|
||
|
||
**Fungsi**: Polyfill/Shim untuk MediaPipe compatibility
|
||
|
||
**Isi**:
|
||
|
||
- Menyediakan global objects yang dibutuhkan MediaPipe di browser
|
||
- Mengatasi module resolution issues
|
||
- Bridge antara MediaPipe WASM dan JavaScript environment
|
||
|
||
**Use Case**: Diload sebelum MediaPipe library untuk ensure compatibility
|
||
|
||
**Size**: ~381 bytes
|
||
|
||
---
|
||
|
||
### **`prisma.ts`**
|
||
|
||
**Fungsi**: Singleton instance dari Prisma Client untuk database operations
|
||
|
||
**Isi**:
|
||
|
||
```typescript
|
||
import { PrismaClient } from "../app/generated/client/client";
|
||
|
||
const globalForPrisma = global as unknown as { prisma: PrismaClient };
|
||
|
||
export const prisma = globalForPrisma.prisma || new PrismaClient();
|
||
|
||
if (process.env.NODE_ENV !== "production") globalForPrisma.prisma = prisma;
|
||
```
|
||
|
||
**Logic**:
|
||
|
||
- Menggunakan singleton pattern untuk avoid multiple instances
|
||
- Di development, menyimpan instance di `global` untuk hot-reload compatibility
|
||
- Di production, create new instance sekali saja
|
||
|
||
**Import Path**: `@/lib/prisma`
|
||
|
||
**Size**: ~300 bytes
|
||
|
||
---
|
||
|
||
## 📁 Folder `pose/` - AI Exercise Recognition Engine
|
||
|
||
Ini adalah core engine yang mendeteksi dan menganalisis gerakan exercise menggunakan AI dan heuristics.
|
||
|
||
### **`ExerciseRules.ts`**
|
||
|
||
**Fungsi**: Konfigurasi dan aturan untuk setiap jenis latihan
|
||
|
||
**Exports**:
|
||
|
||
1. **`interface Landmark`**
|
||
- Representasi titik pose dari MediaPipe
|
||
- Fields: `x`, `y`, `z`, `visibility?`
|
||
|
||
2. **`interface ExerciseConfig`**
|
||
- Schema konfigurasi untuk setiap exercise
|
||
- Fields:
|
||
- `name`: Nama display
|
||
- `detection`: Kriteria untuk mendeteksi exercise type
|
||
- `shoulder_static`: Range angle untuk static shoulder
|
||
- `shoulder_down`: Range untuk shoulder down
|
||
- `hip_static`: Range untuk hip position
|
||
- `phase_type`: 'start_down' | 'start_up' (starting position)
|
||
- `dynamic_angles`: Range angle untuk setiap fase movement
|
||
- Contoh: `elbow_down: [140, 180]`, `elbow_up: [0, 85]`
|
||
- `static_angles`: Ideal angle untuk joints yang harus tetap
|
||
- `wrist_distance`: Range jarak pergelangan tangan
|
||
- `convex_hull`: Area coverage body saat up/down
|
||
|
||
3. **`EXERCISE_CONFIGS`** (Constant Object)
|
||
- Berisi konfigurasi untuk 7 exercise core:
|
||
1. **`bicep_curl`**: Bicep Curl
|
||
- Phase: start_down (arms extended)
|
||
- Dynamic: Elbow flexion 140-180° (down) → 0-85° (up)
|
||
- Shoulder harus static (0-30°)
|
||
2. **`hammer_curl`**: Hammer Curl
|
||
- Similar to bicep curl
|
||
- Wrist distance lebih ketat (0-0.2m)
|
||
3. **`shoulder_press`**: Overhead Press
|
||
- Phase: start_down (shoulders)
|
||
- Elbow: 20-100° (down) → 150-180° (up)
|
||
- Hip harus tetap 165° (standing straight)
|
||
4. **`lateral_raises`**: Lateral Raises
|
||
- Phase: start_down (arms at sides)
|
||
- Shoulder: 0-30° → 80-110° (T-pose)
|
||
- Elbow harus tetap straight (140-180°)
|
||
5. **`squat`**: Squat
|
||
- Phase: start_up (standing)
|
||
- Hip/Knee: 160-180° (up) → 50-100° (down)
|
||
6. **`deadlift`**: Deadlift
|
||
- Phase: start_down (floor)
|
||
- Hip: 45-100° → 160-180°
|
||
- Elbow tetap straight (170°)
|
||
7. **`lunges`**: Lunges
|
||
- Phase: start_up
|
||
- Knee: 160-180° → 70-110°
|
||
|
||
**Use Case**: Digunakan oleh `RehabCore` untuk validasi form dan counting reps
|
||
|
||
**Size**: ~5.1 KB
|
||
|
||
---
|
||
|
||
### **`HARCore.ts`**
|
||
|
||
**Fungsi**: Human Activity Recognition - mendeteksi aktivitas umum (Standing/Sitting/Fall)
|
||
|
||
**Class**: `HARCore`
|
||
|
||
**Fields**:
|
||
|
||
- `predictor: XGBoostPredictor` - ML model untuk klasifikasi
|
||
- `rehab: RehabCore` - Exercise counter instance
|
||
- `currentExercise: string | null` - Exercise yang sedang ditrack
|
||
|
||
**Methods**:
|
||
|
||
1. **`setExercise(name: string)`**
|
||
- Set exercise yang akan ditrack
|
||
- Normalize nama dari UI ke config key
|
||
- Contoh: "Bicep Curl" → "bicep_curl"
|
||
|
||
2. **`resetParams()`**
|
||
- Reset counter tanpa menghapus exercise selection
|
||
- Calls `rehab.reset()`
|
||
|
||
3. **`async process(landmarks, worldLandmarks)`** ⭐ Main Method
|
||
- **Input**:
|
||
- `landmarks`: Array 33 pose keypoints (normalized)
|
||
- `worldLandmarks`: 3D coordinates in meters
|
||
- **Output**:
|
||
```typescript
|
||
{
|
||
status: "Standing" | "Sitting" | "Fall Detected",
|
||
confidence: number,
|
||
exercise: string | null,
|
||
reps: number,
|
||
feedback: string,
|
||
debug: { angles, scores }
|
||
}
|
||
```
|
||
- **Logic**:
|
||
1. Extract 141 features dari landmarks
|
||
2. Classify activity menggunakan XGBoost
|
||
3. Jika exercise active, process dengan RehabCore
|
||
4. Return combined result
|
||
|
||
4. **`extractFeatures(landmarks): number[]`** (Private)
|
||
- Extract 141 features:
|
||
- **132 Raw Features**: 33 landmarks × 4 (x, y, z, visibility)
|
||
- **9 Derived Features**:
|
||
- 6 Angles: Left/Right Elbow, Hip, Knee
|
||
- 2 Ratios: Shoulder width, Hip width (relative to torso)
|
||
- 1 Alignment: Torso vertical cosine similarity
|
||
|
||
**Tech Stack**:
|
||
|
||
- XGBoost model untuk activity classification
|
||
- Heuristic rules untuk exercise recognition
|
||
- Angle calculation dari Math Utils
|
||
|
||
**Size**: ~5.9 KB
|
||
|
||
---
|
||
|
||
### **`RehabCore.ts`**
|
||
|
||
**Fungsi**: Exercise Recognition & Rep Counting dengan form validation
|
||
|
||
**Class**: `RehabCore`
|
||
|
||
**Fields**:
|
||
|
||
- `counters: { [key: string]: RepFSM[] }` - FSM instances per exercise
|
||
- `worldLandmarksCache: Vec3[]` - Cache untuk 3D coordinates
|
||
- `DEVIATION_THRESHOLD = 15.0` - Batas toleransi deviasi form (degrees)
|
||
|
||
**Counter Map**:
|
||
|
||
```typescript
|
||
{
|
||
'bicep_curl': [BicepCurlCounter('left'), BicepCurlCounter('right')],
|
||
'hammer_curl': [HammerCurlCounter('left'), HammerCurlCounter('right')],
|
||
'shoulder_press': [OverheadPressCounter()],
|
||
'lateral_raises': [LateralRaiseCounter()],
|
||
'squat': [SquatCounter()],
|
||
'deadlift': [DeadliftCounter()],
|
||
'lunges': [LungeCounter()]
|
||
}
|
||
```
|
||
|
||
**Methods**:
|
||
|
||
1. **`reset()`**
|
||
- Reset semua counters ke initial state
|
||
- Clear cache
|
||
|
||
2. **`validateExerciseType(configKey, features): string | null`**
|
||
- ⚠️ **6-Way Wrong Exercise Detection**
|
||
- Cek apakah pose sesuai dengan exercise yang diharapkan
|
||
- Return error message jika salah exercise
|
||
- Contoh: Jika program bicep curl tapi user lakukan squat → "Wrong Exercise: Detected lower body"
|
||
|
||
3. **`calculateDeviation(configKey, features, fsmState): { mae, isDeviating, details }`**
|
||
- ⭐ **Per-Rep Form Scoring**
|
||
- Hitung MAE (Mean Absolute Error) dari ideal angles
|
||
- Compare current angles dengan config ranges
|
||
- Return:
|
||
- `mae`: Average deviation dalam degrees
|
||
- `isDeviating`: Boolean (> DEVIATION_THRESHOLD)
|
||
- `details`: Array pesan koreksi spesifik
|
||
|
||
4. **`process(exerciseName, landmarks, worldLandmarks, frameTime)`** ⭐ Main Method
|
||
- **Core rep counting logic**
|
||
- **Workflow**:
|
||
1. Normalize exercise name
|
||
2. Lazy-load counters jika belum ada
|
||
3. Compute features (angles, distances, dll)
|
||
4. **Wrong Exercise Detection**
|
||
5. Update FSM untuk bilateral sides (left/right)
|
||
6. **Form Deviation Analysis** (per frame)
|
||
7. Generate composite feedback
|
||
- **Output**:
|
||
```typescript
|
||
{
|
||
left: { stage, angle, reps },
|
||
right: { stage, angle, reps },
|
||
scores: { deviation_mae },
|
||
feedback: "L: UP | R: DOWN | Knees Inward 🔴"
|
||
}
|
||
```
|
||
|
||
5. **`getReps(exName): number`**
|
||
- Get total reps (max dari left/right atau sum)
|
||
- Untuk bilateral: ambil yang lebih besar
|
||
- Untuk unilateral: langsung return
|
||
|
||
**Key Features**:
|
||
|
||
- ✅ Real-time rep detection menggunakan FSM
|
||
- ✅ Form quality scoring (MAE calculation)
|
||
- ✅ Wrong exercise detection
|
||
- ✅ Specific corrective feedback per frame
|
||
- ✅ Bilateral tracking (left/right independent)
|
||
|
||
**Size**: ~16.1 KB
|
||
|
||
---
|
||
|
||
### **`RehabFSM.ts`**
|
||
|
||
**Fungsi**: Finite State Machines untuk setiap jenis exercise
|
||
|
||
**Exports**:
|
||
|
||
1. **Helper Functions**:
|
||
- `vec3(landmark): Vec3` - Convert Landmark to 3D vector
|
||
- `computeFeatures(landmarks, worldLandmarks): PoseFeatures`
|
||
- Extract semua angles dan distances
|
||
- Return object dengan 20+ features
|
||
|
||
2. **Base Class**: `RepFSM` (Abstract)
|
||
- Fields:
|
||
- `state: "LOW" | "HIGH"`
|
||
- `reps: number`
|
||
- `lastAngle: number`
|
||
- Methods:
|
||
- `abstract shouldTransition(features): boolean`
|
||
- `update(features): void`
|
||
- `reset(): void`
|
||
|
||
3. **Concrete FSMs** (7 Classes):
|
||
|
||
**`BicepCurlCounter(side: 'left' | 'right')`**
|
||
- Track elbow flexion angle
|
||
- LOW (140-180°) ↔ HIGH (0-85°)
|
||
- +1 rep saat kembali ke LOW
|
||
|
||
**`HammerCurlCounter(side: 'left' | 'right')`**
|
||
- Similar to Bicep, sedikit berbeda thresholds
|
||
|
||
**`OverheadPressCounter()`**
|
||
- Bilateral (min of left/right elbow)
|
||
- LOW: hands at shoulders
|
||
- HIGH: arms extended overhead
|
||
|
||
**`LateralRaiseCounter()`**
|
||
- Track shoulder abduction (max of left/right)
|
||
- LOW: arms at sides → HIGH: T-pose
|
||
|
||
**`SquatCounter()`**
|
||
- Track hip/knee flexion (min)
|
||
- HIGH: standing → LOW: squat depth
|
||
|
||
**`DeadliftCounter()`**
|
||
- Track hip extension
|
||
- LOW: floor position → HIGH: lockout
|
||
|
||
**`LungeCounter()`**
|
||
- Track knee flexion (min of both)
|
||
- Similar to squat
|
||
|
||
**Logic Pattern**:
|
||
|
||
```typescript
|
||
if (state === "LOW" && angle < UP_THRESHOLD) {
|
||
state = "HIGH";
|
||
} else if (state === "HIGH" && angle > DOWN_THRESHOLD) {
|
||
state = "LOW";
|
||
reps++; // Rep completed!
|
||
}
|
||
```
|
||
|
||
**Size**: Variable (likely 5-10 KB based on 7 classes)
|
||
|
||
---
|
||
|
||
### **`MathUtils.ts`**
|
||
|
||
**Fungsi**: Mathematical utility functions untuk pose analysis
|
||
|
||
**Functions**:
|
||
|
||
1. **`calculateAngle(a, b, c): number`**
|
||
- Hitung angle di point B dari triangle ABC
|
||
- Input: 3 points dengan {x, y}
|
||
- Output: Angle dalam degrees (0-180)
|
||
- Formula: `arctangent` menggunakan vectors
|
||
|
||
2. **`calculateRangeDeviation(value, range, weight = 1.0): number`**
|
||
- Hitung deviasi dari ideal range
|
||
- Jika value dalam range → return 0
|
||
- Jika di luar → return distance × weight
|
||
- Contoh: `value=90, range=[80,100]` → return 0
|
||
- Contoh: `value=110, range=[80,100]` → return 10
|
||
|
||
3. **`computeMAE(deviations): number`**
|
||
- Mean Absolute Error
|
||
- Average dari array deviasi
|
||
- Digunakan untuk overall form score
|
||
|
||
**Use Case**:
|
||
|
||
- Digunakan di semua modules untuk angle calculation
|
||
- Form validation dan scoring
|
||
|
||
**Size**: ~500 bytes (estimasi)
|
||
|
||
---
|
||
|
||
### **`XGBoostPredictor.ts`**
|
||
|
||
**Fungsi**: XGBoost Machine Learning model predictor untuk activity classification
|
||
|
||
**Class**: `XGBoostPredictor`
|
||
|
||
**Purpose**:
|
||
|
||
- Classify pose sebagai "Standing", "Sitting", atau "Fall Detected"
|
||
- Menggunakan pre-trained XGBoost model
|
||
|
||
**Methods**:
|
||
|
||
- `predict(features: number[]): number[]`
|
||
- Input: 141 features dari `HARCore.extractFeatures()`
|
||
- Output: Probability array [standingProb, sittingProb, fallProb]
|
||
- Model di-load dari embedded weights
|
||
|
||
**Tech**:
|
||
|
||
- XGBoost model serialized ke JavaScript
|
||
- Likely menggunakan ONNX atau custom JSON format
|
||
|
||
**Size**: Likely large (10-50 KB) karena contain model weights
|
||
|
||
---
|
||
|
||
### **`RepetitionCounter.ts`**
|
||
|
||
**Fungsi**: Legacy rep counter (kemungkinan deprecated)
|
||
|
||
**Status**: ⚠️ Likely tidak digunakan, digantikan oleh `RehabFSM.ts`
|
||
|
||
**Reason**:
|
||
|
||
- `RehabCore` menggunakan FSM dari `RehabFSM.ts`
|
||
- File ini mungkin versi lama sebelum refactor
|
||
|
||
---
|
||
|
||
## 📁 Folder `prisma/` dan `prisma-gen/`
|
||
|
||
**Status**: Auto-generated by Prisma CLI
|
||
|
||
### **`prisma/client/`**
|
||
|
||
- Symlink atau copy dari `/app/generated/client`
|
||
- Prisma Client untuk browser compatibility
|
||
|
||
### **`prisma-gen/`**
|
||
|
||
- Type definitions dan models
|
||
- Files:
|
||
- `browser.ts` - Browser-compatible client
|
||
- `client.ts` - Main client
|
||
- `models.ts` - Type definitions
|
||
- `enums.ts` - Enum types
|
||
- `models/` - Individual model files:
|
||
- `activity_logs.ts`
|
||
- `training_menus.ts`
|
||
- `user_recaps.ts`
|
||
|
||
**PENTING**: ❌ **JANGAN EDIT MANUAL** - Will be regenerated by `prisma generate`
|
||
|
||
---
|
||
|
||
## 🔑 Key Concepts & Data Flow
|
||
|
||
### **Complete Exercise Recognition Flow**:
|
||
|
||
```
|
||
1. MediaPipe Pose Detection
|
||
└─> 33 Landmarks (x, y, z, visibility)
|
||
|
||
2. HARCore.process()
|
||
├─> Extract 141 Features
|
||
├─> XGBoost Prediction (Standing/Sitting/Fall)
|
||
└─> RehabCore.process()
|
||
├─> Compute Angles & Features
|
||
├─> Wrong Exercise Detection
|
||
├─> FSM Update (Rep Counting)
|
||
├─> Form Deviation Calculation (MAE)
|
||
└─> Generate Feedback Text
|
||
|
||
3. Return to UI
|
||
└─> { status, reps, feedback, scores: {deviation_mae} }
|
||
```
|
||
|
||
### **Per-Rep Tracking Flow** (NEW):
|
||
|
||
```
|
||
Training Page (client/training/page.tsx)
|
||
├─> Frame Loop: predictWebcam()
|
||
│ ├─> Call har.process()
|
||
│ ├─> Accumulate MAE to repBuffer
|
||
│ └─> Capture feedback to repFeedbackBuffer
|
||
│
|
||
├─> Rep Completion Detection (res.reps > lastRepCount)
|
||
│ ├─> Calculate avgRepScore
|
||
│ ├─> Find dominant feedback
|
||
│ └─> Push to currentSetReps[{rep, score, feedback}]
|
||
│
|
||
└─> Set Completion
|
||
├─> Save to results state
|
||
└─> API POST to /api/recap
|
||
```
|
||
|
||
---
|
||
|
||
## 💡 Important Notes
|
||
|
||
### **Exercise Configuration**:
|
||
|
||
- Semua config di `ExerciseRules.ts`
|
||
- Untuk tambah exercise baru:
|
||
1. Add config di `EXERCISE_CONFIGS`
|
||
2. Create FSM class di `RehabFSM.ts`
|
||
3. Add entry di `COUNTER_MAP` di `RehabCore.ts`
|
||
|
||
### **Form Scoring**:
|
||
|
||
- MAE (Mean Absolute Error) dalam degrees
|
||
- Thresholds:
|
||
- < 8° = **Excellent**
|
||
- < 15° = **Good**
|
||
- < 25° = **Fair**
|
||
- ≥ 25° = **Needs Improvement**
|
||
|
||
### **Wrong Exercise Detection**:
|
||
|
||
- 6-way classification:
|
||
1. Upper body dynamic (curls/press)
|
||
2. Upper body static T-pose (lateral raises)
|
||
3. Lower body squat pattern
|
||
4. Lower body hinge pattern (deadlift)
|
||
5. Lower body lunge pattern
|
||
6. Unrecognized pose
|
||
|
||
### **Performance**:
|
||
|
||
- Frame processing: ~10-30ms
|
||
- Feature extraction: O(n) dengan n=33 landmarks
|
||
- FSM updates: O(1) per exercise
|
||
- Real-time capable at 30 FPS
|
||
|
||
---
|
||
|
||
## 📊 Dependencies
|
||
|
||
```
|
||
lib/pose/
|
||
├── ExerciseRules.ts (Pure config, no deps)
|
||
├── MathUtils.ts (Pure math, no deps)
|
||
├── RehabFSM.ts
|
||
│ └── depends on: ExerciseRules, MathUtils
|
||
├── RehabCore.ts
|
||
│ └── depends on: ExerciseRules, RehabFSM, MathUtils
|
||
├── XGBoostPredictor.ts (Standalone ML model)
|
||
└── HARCore.ts
|
||
└── depends on: ALL above
|
||
```
|
||
|
||
```
|
||
lib/ (Root)
|
||
├── auth.tsx (React Context)
|
||
├── prisma.ts (Database client)
|
||
└── mediapipe-shim.js (Standalone polyfill)
|
||
```
|
||
|
||
---
|
||
|
||
## 🛠️ Maintenance Tips
|
||
|
||
1. **Update Exercise Config**: Edit `ExerciseRules.ts` → Test di Monitor mode
|
||
2. **Adjust Thresholds**: Tweak angle ranges di config atau FSM classes
|
||
3. **Debug Form Scores**: Check `res.debug.scores` di browser console
|
||
4. **Add New Exercise**:
|
||
- Step 1: Config di `ExerciseRules.ts`
|
||
- Step 2: FSM class di `RehabFSM.ts`
|
||
- Step 3: Register di `RehabCore.ts` COUNTER_MAP
|
||
- Step 4: Update UI exercise dropdown
|
||
5. **Performance Tuning**: Optimize `extractFeatures()` di HARCore jika lag
|
||
|
||
---
|
||
|
||
## 📈 Future Enhancements
|
||
|
||
- [ ] Add more exercises (Pull-ups, Push-ups, etc.)
|
||
- [ ] Implement velocity-based training (track rep speed)
|
||
- [ ] Add form correction animations/overlays
|
||
- [ ] Machine learning untuk auto-tune thresholds per user
|
||
- [ ] Export workout data ke CSV/PDF
|