# Dokumentasi Struktur Folder `lib/` ## Gambaran Umum Folder `lib/` berisi semua utility libraries, helper functions, dan core logic aplikasi. Ini adalah "otak" dari aplikasi yang menangani AI pose detection, exercise recognition, authentication, dan database operations. --- ## 📁 Struktur Direktori ``` lib/ ├── auth.tsx # Authentication Context & Utilities ├── mediapipe-shim.js # MediaPipe Polyfill ├── prisma.ts # Prisma Client Instance ├── pose/ # AI & Exercise Recognition Engine │ ├── ExerciseRules.ts # Exercise Configuration & Rules │ ├── HARCore.ts # Human Activity Recognition Core │ ├── MathUtils.ts # Mathematical Utilities │ ├── RehabCore.ts # Exercise Recognition & Counting │ ├── RehabFSM.ts # Finite State Machines for Reps │ ├── RepetitionCounter.ts # Legacy Rep Counter │ └── XGBoostPredictor.ts # ML Model Predictor ├── prisma/ # Prisma Generated Client (Symlink) └── prisma-gen/ # Prisma Generated Types ``` --- ## 📄 File Root Level ### **`auth.tsx`** **Fungsi**: Context Provider untuk authentication dan session management **Isi Utama**: - **Interface**: - `User`: Type definition untuk user object (id, name, role, coach_id) - `AuthContextType`: Type untuk auth context - `UserRole`: 'COACH' | 'CLIENT' - **Context**: - `AuthContext`: React Context untuk state auth global - `AuthProvider`: Provider component yang wrap seluruh app - **Methods**: - `login(userId: string)`: Login user berdasarkan ID, fetch data dari `/api/users/[id]` - `logout()`: Clear session, hapus localStorage, reset state - Auto-load session dari localStorage saat app mount **Tech**: React Context API, localStorage persistence **Size**: ~1.8 KB --- ### **`mediapipe-shim.js`** **Fungsi**: Polyfill/Shim untuk MediaPipe compatibility **Isi**: - Menyediakan global objects yang dibutuhkan MediaPipe di browser - Mengatasi module resolution issues - Bridge antara MediaPipe WASM dan JavaScript environment **Use Case**: Diload sebelum MediaPipe library untuk ensure compatibility **Size**: ~381 bytes --- ### **`prisma.ts`** **Fungsi**: Singleton instance dari Prisma Client untuk database operations **Isi**: ```typescript import { PrismaClient } from "../app/generated/client/client"; const globalForPrisma = global as unknown as { prisma: PrismaClient }; export const prisma = globalForPrisma.prisma || new PrismaClient(); if (process.env.NODE_ENV !== "production") globalForPrisma.prisma = prisma; ``` **Logic**: - Menggunakan singleton pattern untuk avoid multiple instances - Di development, menyimpan instance di `global` untuk hot-reload compatibility - Di production, create new instance sekali saja **Import Path**: `@/lib/prisma` **Size**: ~300 bytes --- ## 📁 Folder `pose/` - AI Exercise Recognition Engine Ini adalah core engine yang mendeteksi dan menganalisis gerakan exercise menggunakan AI dan heuristics. ### **`ExerciseRules.ts`** **Fungsi**: Konfigurasi dan aturan untuk setiap jenis latihan **Exports**: 1. **`interface Landmark`** - Representasi titik pose dari MediaPipe - Fields: `x`, `y`, `z`, `visibility?` 2. **`interface ExerciseConfig`** - Schema konfigurasi untuk setiap exercise - Fields: - `name`: Nama display - `detection`: Kriteria untuk mendeteksi exercise type - `shoulder_static`: Range angle untuk static shoulder - `shoulder_down`: Range untuk shoulder down - `hip_static`: Range untuk hip position - `phase_type`: 'start_down' | 'start_up' (starting position) - `dynamic_angles`: Range angle untuk setiap fase movement - Contoh: `elbow_down: [140, 180]`, `elbow_up: [0, 85]` - `static_angles`: Ideal angle untuk joints yang harus tetap - `wrist_distance`: Range jarak pergelangan tangan - `convex_hull`: Area coverage body saat up/down 3. **`EXERCISE_CONFIGS`** (Constant Object) - Berisi konfigurasi untuk 7 exercise core: 1. **`bicep_curl`**: Bicep Curl - Phase: start_down (arms extended) - Dynamic: Elbow flexion 140-180° (down) → 0-85° (up) - Shoulder harus static (0-30°) 2. **`hammer_curl`**: Hammer Curl - Similar to bicep curl - Wrist distance lebih ketat (0-0.2m) 3. **`shoulder_press`**: Overhead Press - Phase: start_down (shoulders) - Elbow: 20-100° (down) → 150-180° (up) - Hip harus tetap 165° (standing straight) 4. **`lateral_raises`**: Lateral Raises - Phase: start_down (arms at sides) - Shoulder: 0-30° → 80-110° (T-pose) - Elbow harus tetap straight (140-180°) 5. **`squat`**: Squat - Phase: start_up (standing) - Hip/Knee: 160-180° (up) → 50-100° (down) 6. **`deadlift`**: Deadlift - Phase: start_down (floor) - Hip: 45-100° → 160-180° - Elbow tetap straight (170°) 7. **`lunges`**: Lunges - Phase: start_up - Knee: 160-180° → 70-110° **Use Case**: Digunakan oleh `RehabCore` untuk validasi form dan counting reps **Size**: ~5.1 KB --- ### **`HARCore.ts`** **Fungsi**: Human Activity Recognition - mendeteksi aktivitas umum (Standing/Sitting/Fall) **Class**: `HARCore` **Fields**: - `predictor: XGBoostPredictor` - ML model untuk klasifikasi - `rehab: RehabCore` - Exercise counter instance - `currentExercise: string | null` - Exercise yang sedang ditrack **Methods**: 1. **`setExercise(name: string)`** - Set exercise yang akan ditrack - Normalize nama dari UI ke config key - Contoh: "Bicep Curl" → "bicep_curl" 2. **`resetParams()`** - Reset counter tanpa menghapus exercise selection - Calls `rehab.reset()` 3. **`async process(landmarks, worldLandmarks)`** ⭐ Main Method - **Input**: - `landmarks`: Array 33 pose keypoints (normalized) - `worldLandmarks`: 3D coordinates in meters - **Output**: ```typescript { status: "Standing" | "Sitting" | "Fall Detected", confidence: number, exercise: string | null, reps: number, feedback: string, debug: { angles, scores } } ``` - **Logic**: 1. Extract 141 features dari landmarks 2. Classify activity menggunakan XGBoost 3. Jika exercise active, process dengan RehabCore 4. Return combined result 4. **`extractFeatures(landmarks): number[]`** (Private) - Extract 141 features: - **132 Raw Features**: 33 landmarks × 4 (x, y, z, visibility) - **9 Derived Features**: - 6 Angles: Left/Right Elbow, Hip, Knee - 2 Ratios: Shoulder width, Hip width (relative to torso) - 1 Alignment: Torso vertical cosine similarity **Tech Stack**: - XGBoost model untuk activity classification - Heuristic rules untuk exercise recognition - Angle calculation dari Math Utils **Size**: ~5.9 KB --- ### **`RehabCore.ts`** **Fungsi**: Exercise Recognition & Rep Counting dengan form validation **Class**: `RehabCore` **Fields**: - `counters: { [key: string]: RepFSM[] }` - FSM instances per exercise - `worldLandmarksCache: Vec3[]` - Cache untuk 3D coordinates - `DEVIATION_THRESHOLD = 15.0` - Batas toleransi deviasi form (degrees) **Counter Map**: ```typescript { 'bicep_curl': [BicepCurlCounter('left'), BicepCurlCounter('right')], 'hammer_curl': [HammerCurlCounter('left'), HammerCurlCounter('right')], 'shoulder_press': [OverheadPressCounter()], 'lateral_raises': [LateralRaiseCounter()], 'squat': [SquatCounter()], 'deadlift': [DeadliftCounter()], 'lunges': [LungeCounter()] } ``` **Methods**: 1. **`reset()`** - Reset semua counters ke initial state - Clear cache 2. **`validateExerciseType(configKey, features): string | null`** - ⚠️ **6-Way Wrong Exercise Detection** - Cek apakah pose sesuai dengan exercise yang diharapkan - Return error message jika salah exercise - Contoh: Jika program bicep curl tapi user lakukan squat → "Wrong Exercise: Detected lower body" 3. **`calculateDeviation(configKey, features, fsmState): { mae, isDeviating, details }`** - ⭐ **Per-Rep Form Scoring** - Hitung MAE (Mean Absolute Error) dari ideal angles - Compare current angles dengan config ranges - Return: - `mae`: Average deviation dalam degrees - `isDeviating`: Boolean (> DEVIATION_THRESHOLD) - `details`: Array pesan koreksi spesifik 4. **`process(exerciseName, landmarks, worldLandmarks, frameTime)`** ⭐ Main Method - **Core rep counting logic** - **Workflow**: 1. Normalize exercise name 2. Lazy-load counters jika belum ada 3. Compute features (angles, distances, dll) 4. **Wrong Exercise Detection** 5. Update FSM untuk bilateral sides (left/right) 6. **Form Deviation Analysis** (per frame) 7. Generate composite feedback - **Output**: ```typescript { left: { stage, angle, reps }, right: { stage, angle, reps }, scores: { deviation_mae }, feedback: "L: UP | R: DOWN | Knees Inward 🔴" } ``` 5. **`getReps(exName): number`** - Get total reps (max dari left/right atau sum) - Untuk bilateral: ambil yang lebih besar - Untuk unilateral: langsung return **Key Features**: - ✅ Real-time rep detection menggunakan FSM - ✅ Form quality scoring (MAE calculation) - ✅ Wrong exercise detection - ✅ Specific corrective feedback per frame - ✅ Bilateral tracking (left/right independent) **Size**: ~16.1 KB --- ### **`RehabFSM.ts`** **Fungsi**: Finite State Machines untuk setiap jenis exercise **Exports**: 1. **Helper Functions**: - `vec3(landmark): Vec3` - Convert Landmark to 3D vector - `computeFeatures(landmarks, worldLandmarks): PoseFeatures` - Extract semua angles dan distances - Return object dengan 20+ features 2. **Base Class**: `RepFSM` (Abstract) - Fields: - `state: "LOW" | "HIGH"` - `reps: number` - `lastAngle: number` - Methods: - `abstract shouldTransition(features): boolean` - `update(features): void` - `reset(): void` 3. **Concrete FSMs** (7 Classes): **`BicepCurlCounter(side: 'left' | 'right')`** - Track elbow flexion angle - LOW (140-180°) ↔ HIGH (0-85°) - +1 rep saat kembali ke LOW **`HammerCurlCounter(side: 'left' | 'right')`** - Similar to Bicep, sedikit berbeda thresholds **`OverheadPressCounter()`** - Bilateral (min of left/right elbow) - LOW: hands at shoulders - HIGH: arms extended overhead **`LateralRaiseCounter()`** - Track shoulder abduction (max of left/right) - LOW: arms at sides → HIGH: T-pose **`SquatCounter()`** - Track hip/knee flexion (min) - HIGH: standing → LOW: squat depth **`DeadliftCounter()`** - Track hip extension - LOW: floor position → HIGH: lockout **`LungeCounter()`** - Track knee flexion (min of both) - Similar to squat **Logic Pattern**: ```typescript if (state === "LOW" && angle < UP_THRESHOLD) { state = "HIGH"; } else if (state === "HIGH" && angle > DOWN_THRESHOLD) { state = "LOW"; reps++; // Rep completed! } ``` **Size**: Variable (likely 5-10 KB based on 7 classes) --- ### **`MathUtils.ts`** **Fungsi**: Mathematical utility functions untuk pose analysis **Functions**: 1. **`calculateAngle(a, b, c): number`** - Hitung angle di point B dari triangle ABC - Input: 3 points dengan {x, y} - Output: Angle dalam degrees (0-180) - Formula: `arctangent` menggunakan vectors 2. **`calculateRangeDeviation(value, range, weight = 1.0): number`** - Hitung deviasi dari ideal range - Jika value dalam range → return 0 - Jika di luar → return distance × weight - Contoh: `value=90, range=[80,100]` → return 0 - Contoh: `value=110, range=[80,100]` → return 10 3. **`computeMAE(deviations): number`** - Mean Absolute Error - Average dari array deviasi - Digunakan untuk overall form score **Use Case**: - Digunakan di semua modules untuk angle calculation - Form validation dan scoring **Size**: ~500 bytes (estimasi) --- ### **`XGBoostPredictor.ts`** **Fungsi**: XGBoost Machine Learning model predictor untuk activity classification **Class**: `XGBoostPredictor` **Purpose**: - Classify pose sebagai "Standing", "Sitting", atau "Fall Detected" - Menggunakan pre-trained XGBoost model **Methods**: - `predict(features: number[]): number[]` - Input: 141 features dari `HARCore.extractFeatures()` - Output: Probability array [standingProb, sittingProb, fallProb] - Model di-load dari embedded weights **Tech**: - XGBoost model serialized ke JavaScript - Likely menggunakan ONNX atau custom JSON format **Size**: Likely large (10-50 KB) karena contain model weights --- ### **`RepetitionCounter.ts`** **Fungsi**: Legacy rep counter (kemungkinan deprecated) **Status**: ⚠️ Likely tidak digunakan, digantikan oleh `RehabFSM.ts` **Reason**: - `RehabCore` menggunakan FSM dari `RehabFSM.ts` - File ini mungkin versi lama sebelum refactor --- ## 📁 Folder `prisma/` dan `prisma-gen/` **Status**: Auto-generated by Prisma CLI ### **`prisma/client/`** - Symlink atau copy dari `/app/generated/client` - Prisma Client untuk browser compatibility ### **`prisma-gen/`** - Type definitions dan models - Files: - `browser.ts` - Browser-compatible client - `client.ts` - Main client - `models.ts` - Type definitions - `enums.ts` - Enum types - `models/` - Individual model files: - `activity_logs.ts` - `training_menus.ts` - `user_recaps.ts` **PENTING**: ❌ **JANGAN EDIT MANUAL** - Will be regenerated by `prisma generate` --- ## 🔑 Key Concepts & Data Flow ### **Complete Exercise Recognition Flow**: ``` 1. MediaPipe Pose Detection └─> 33 Landmarks (x, y, z, visibility) 2. HARCore.process() ├─> Extract 141 Features ├─> XGBoost Prediction (Standing/Sitting/Fall) └─> RehabCore.process() ├─> Compute Angles & Features ├─> Wrong Exercise Detection ├─> FSM Update (Rep Counting) ├─> Form Deviation Calculation (MAE) └─> Generate Feedback Text 3. Return to UI └─> { status, reps, feedback, scores: {deviation_mae} } ``` ### **Per-Rep Tracking Flow** (NEW): ``` Training Page (client/training/page.tsx) ├─> Frame Loop: predictWebcam() │ ├─> Call har.process() │ ├─> Accumulate MAE to repBuffer │ └─> Capture feedback to repFeedbackBuffer │ ├─> Rep Completion Detection (res.reps > lastRepCount) │ ├─> Calculate avgRepScore │ ├─> Find dominant feedback │ └─> Push to currentSetReps[{rep, score, feedback}] │ └─> Set Completion ├─> Save to results state └─> API POST to /api/recap ``` --- ## 💡 Important Notes ### **Exercise Configuration**: - Semua config di `ExerciseRules.ts` - Untuk tambah exercise baru: 1. Add config di `EXERCISE_CONFIGS` 2. Create FSM class di `RehabFSM.ts` 3. Add entry di `COUNTER_MAP` di `RehabCore.ts` ### **Form Scoring**: - MAE (Mean Absolute Error) dalam degrees - Thresholds: - < 8° = **Excellent** - < 15° = **Good** - < 25° = **Fair** - ≥ 25° = **Needs Improvement** ### **Wrong Exercise Detection**: - 6-way classification: 1. Upper body dynamic (curls/press) 2. Upper body static T-pose (lateral raises) 3. Lower body squat pattern 4. Lower body hinge pattern (deadlift) 5. Lower body lunge pattern 6. Unrecognized pose ### **Performance**: - Frame processing: ~10-30ms - Feature extraction: O(n) dengan n=33 landmarks - FSM updates: O(1) per exercise - Real-time capable at 30 FPS --- ## 📊 Dependencies ``` lib/pose/ ├── ExerciseRules.ts (Pure config, no deps) ├── MathUtils.ts (Pure math, no deps) ├── RehabFSM.ts │ └── depends on: ExerciseRules, MathUtils ├── RehabCore.ts │ └── depends on: ExerciseRules, RehabFSM, MathUtils ├── XGBoostPredictor.ts (Standalone ML model) └── HARCore.ts └── depends on: ALL above ``` ``` lib/ (Root) ├── auth.tsx (React Context) ├── prisma.ts (Database client) └── mediapipe-shim.js (Standalone polyfill) ``` --- ## 🛠️ Maintenance Tips 1. **Update Exercise Config**: Edit `ExerciseRules.ts` → Test di Monitor mode 2. **Adjust Thresholds**: Tweak angle ranges di config atau FSM classes 3. **Debug Form Scores**: Check `res.debug.scores` di browser console 4. **Add New Exercise**: - Step 1: Config di `ExerciseRules.ts` - Step 2: FSM class di `RehabFSM.ts` - Step 3: Register di `RehabCore.ts` COUNTER_MAP - Step 4: Update UI exercise dropdown 5. **Performance Tuning**: Optimize `extractFeatures()` di HARCore jika lag --- ## 📈 Future Enhancements - [ ] Add more exercises (Pull-ups, Push-ups, etc.) - [ ] Implement velocity-based training (track rep speed) - [ ] Add form correction animations/overlays - [ ] Machine learning untuk auto-tune thresholds per user - [ ] Export workout data ke CSV/PDF