Smartphone-Aided VR Exergames for HIIT: Gamifying Fitness with Real-Time Pose Estimation
Organizations: Human Computer Interaction Lab, Indian Institute of Information Technology Sri City, Andhra Pradesh 517646, India · Technical University of Munich, Munich, Germany
Abstract
This paper introduces an innovative virtual reality (VR) exergame system designed to gamify High-Intensity Interval Training (HIIT) workouts through 11 predefined combo moves, including squat-punch combinations, bilateral arm strikes, and diagonal stretch combos. Unlike conventional tracking methods that often rely on wearable motion capture suits or limited headset-based sensors, our system uses a standard smartphone camera placed on a tripod to capture full-body movement. This approach offers a cost-effective and scalable alternative, particularly well-suited to the fast, dynamic motions typical of HIIT routines. The smartphone footage is processed in real time using a pose estimation algorithm, with extracted biomechanical data analyzed by a backend server built on the Model Context Protocol (MCP), Anthropic's open standard for connecting AI models to structured data sources and tools, and a GPT-4o large language model. The system generates personalized assessments of form accuracy, movement efficiency, exercise intensity, and potential injury risks, tailored specifically to high-impact, time-sensitive exercises. A within-subject study involving 20 participants (ages 18-25) compared experiences in VR and Non-VR HIIT sessions. Results indicated significantly higher enjoyment (Mean = 3.5 vs 2.8, p = 0.02), improved immersion/focus (Mean = 3.2 vs 2.8, p = 0.03), and increased likelihood to recommend (Mean = 3.2 vs 2.7, p = 0.04) under the VR condition. Confidence to continue did not differ significantly between conditions (p = 0.15), while perceived intensity remained comparable. The MCP system showed strong agreement with expert evaluation (87.5%, N = 10 sessions, one certified trainer), supporting its reliability as a feedback tool.
Figures & tables
| Combo | Orbs | Direction | Movement Type |
|---|---|---|---|
| Combo_1 | Orb_1, Orb_2 | Horizontal | Lateral punch or side jab |
| Combo_2 | Orb_3, Orb_4 | Horizontal | Torso twist or mid strike |
| Combo_3 | Orb_5, Orb_6 | Horizontal | Side squat-punch |
| Combo_4 | Orb_1, Orb_4 | Diagonal | Upper diagonal hand stretch |
| Combo_5 | Orb_3, Orb_2 | Diagonal | Upper diagonal hand stretch |
| Combo_6 | Orb_3, Orb_6 | Diagonal | Lower diagonal stretch with squats |
| Metric | VR Group | Non-VR Group |
|---|---|---|
| Motivation | 2.8 | 2.6 |
| Enjoyment | 3.5 | 2.8 |
| Focus/Immersion | 3.2 | 2.8 |
| Perceived Intensity | 3.3 | 3.4 |
| Confidence to Continue | 2.9 | 2.8 |
| Recommendation Likelihood | 3.2 | 2.7 |
Virtual Reality
Virtual reality studies immersive, interactive environments and the technologies and human experiences that make them useful.
See all 46 papers