Impact of Patient Orientation in Single- and Multi-View Camera Environments for AI-based Rehabilitation Monitoring

Miriama Jánošová; Andreas Lang; Petra Budikova; Jan Sedmidubsky

cs.CVSep 28, 2026

Impact of Patient Orientation in Single- and Multi-View Camera Environments for AI-based Rehabilitation Monitoring

Authors: Miriama Jánošová, Andreas Lang, Petra Budikova, Jan Sedmidubsky

Organizations: Faculty of Informatics, Masaryk University, Brno, Czech Republic · TU Dortmund University, Dortmund, Germany · VisionCraft, Brno, Czech Republic

Abstract

Automated quality assessment of rehabilitation exercises relies heavily on accurate human pose estimation from video data. Although numerous RGB-based pose estimation methods have been proposed, the impact of camera placement on detecting clinically relevant movement errors remains insufficiently explored. To address this gap, we introduce REHAB26-ViewAngles, a dataset comprising correct and incorrect rehabilitation exercise executions captured from a wide range of camera angles. Furthermore, we propose a novel separability metric to quantify an algorithm's ability to distinguish between valid and faulty exercise repetitions. Using these tools, we analyze how various RGB-based pose-estimation strategies are suitable for exercise quality assessment under varying camera placements. In particular, we analyze single-camera 2D and 3D pose estimation and four multi-camera strategies: a combination of two orthogonal 2D views, 3D triangulation, weighted 3D fusion, and an AI-based pose-estimation transformer model specifically trained from two synchronized cameras. Our findings reveal that an optimally placed 2D camera can improve the separability by 16.9% over the commonly used 0° frontal view and frequently outperforms single-camera 3D estimation, while combining two views can further improve accuracy by up to 13.1%. These results offer practical guidance for deploying rehabilitation monitoring in both home and clinical settings.

Figures & tables

Figure 1: High-level scheme of personalized monitoring of rehabilitation exercises. The highlighted red parts are the main focus of this paper – we are especially interested in the optimal positioning of camera(s) to capture the patient’s movements.

Figure 2: The REHAB26-ViewAngles data acquisition setup. Three cameras are positioned at fixed angles ( $\theta_{Cam1}=-90^{\circ}$ , $\theta_{Cam2}=-45^{\circ}$ , $\theta_{Cam3}=0^{\circ}$ ) while the participant systematically changes their standing orientation (angle $\alpha$ with respect to Cam3). At each designated orientation, the participant performs a continuous set of 10 repetitions, consisting of 5 correct and 5 erroneous repetitions, each with a specific clinical error. The camera viewpoint ( $VP_{Cam_{i}}$ ) for any given recording is mathematically defined as the sum of the participant’s rotation and the static camera position ( $VP_{Cam_{i}}=\alpha+\theta_{i}$ ).

Exercise	Error Description	Feature Code	What Angle is Measured?
Ex1	Arm positioned too far forward	F1	Angle between the hand, shoulder, and torso
Ex1, Ex2	Shoulder elevation	F2	Angle between both shoulders, and hip
Ex1	Lateral head tilt	F3	Angle between the head, the torso, and vertical axis
Ex1, Ex2	Elbow flexion (frontal)	F4	Angle between the upper arm and forearm
Ex1, Ex2	Wrong arm positioning	F5	Angle between the hand and both shoulders
Ex2, Ex3	Lateral torso bending	F6	Angle between the torso and the lower limbs

Table 1: Overview of movement errors, their associated joint-angle measurements, and the corresponding feature codes. Some features (F2, F4-F6) are shared across multiple exercises. Features F9–F14 are defined separately for the left and right sides.

Panel of Figure 3 — Figure 3: Representative examples from the REHAB26-ViewAngles dataset. (a) Temporal progression of Exercise 1 (Arm Abduction) recorded from a single camera. (b) Exercise 2 (Shoulder Extension) recorded simultaneously from three distinct camera perspectives. (c) Exercise 3 (Squat) captured from the same camera across three different patient orientations, with the viewpoint shifting by 10-degree increments.

	$-90^{\circ}$	$-80^{\circ}$	$-70^{\circ}$	$-60^{\circ}$	$-50^{\circ}$	$-45^{\circ}$	$-40^{\circ}$	$-35^{\circ}$	$-30^{\circ}$	$-25^{\circ}$	$-20^{\circ}$	$-15^{\circ}$	$-10^{\circ}$	$-5^{\circ}$	$0^{\circ}$	$5^{\circ}$	$10^{\circ}$	$15^{\circ}$	$20^{\circ}$	$25^{\circ}$	$30^{\circ}$	$35^{\circ}$	$40^{\circ}$	$45^{\circ}$	$50^{\circ}$	$60^{\circ}$	$70^{\circ}$	$80^{\circ}$	$90^{\circ}$
Cam1	✓	✓	✓	✓	✓	✓	✓		✓		✓		✓		✓
Cam2						✓		✓		✓		✓		✓	✓	✓		✓		✓		✓		✓
Cam3															✓		✓		✓		✓		✓	✓	✓	✓	✓	✓	✓

Table 2: Overview of all viewpoints available in REHAB26-ViewAngles dataset. A check mark indicates that the viewpoint (column header) was recorded for the given camera.

Figure 4: Comparison of a feature F2 value (angle) across Exercise1 poses for correct vs. erroneous repetitions, including the mean and 95% confidence interval of the correct repetitions.

Figure 5: Possible approaches to pose data estimation in single and multiple camera environments.

Figure 6: Illustration of feature separability scores ( $\Psi^{k}$ ) for Exercises 1, 2, and 3 at different viewpoint angles. In each heatmap, rows correspond to camera angles and columns to individual features. The uppermost heatmaps present features computed from 2D poses. The second row of heatmaps shows features computed from 3D poses. In the last row, rotation heatmaps visualize how deviations from the optimal viewing angle affect the separability scores. Darker green indicates higher separability scores, while dark red corresponds to situations where correct and incorrect repetitions are indistinguishable.

Cams	Dims	Method	-90	-60	-40	-30	-20	-15	-10	0	10	15	20	30	40	60	90
S	2D	MPP	0.36	0.39	0.55	0.57	0.56	0.57	0.54	0.58	0.60	0.66	0.62	0.61	0.61	0.63	0.41
	3D	MPP	0.31	0.44	0.49	0.46	0.51	0.57	0.50	0.47	0.54	0.55	0.57	0.59	0.47	0.51	0.42
	2D	YOLO	0.21	0.38	0.55	0.60	0.53	0.57	0.53	0.56	0.61	0.63	0.64	0.62	0.60	0.59	0.41
M	2D	Ortho.	0.58	0.61	0.65	0.66	0.64	–	0.63	0.66	–	–	–	–	–	–	–
	3D	Triang.	0.43	0.61	0.67	0.63	0.60	–	0.60	0.54	–	–	–	–	–	–	–
	3D	Merged	0.55	0.63	0.63	0.61	0.58	–	0.62	0.51	–	–	–	–	–	–	–

Table 3: Overall separability scores across single (S) and multi-camera (M) methods and various viewpoint angles for all three exercises. For multi-camera approaches, the column header corresponds to the Cam1 viewpoint; the Cam3 viewpoint is always orthogonal (i.e., increased by

90^{\circ}

). Higher values indicate better distinguishability between correct and incorrect movements. Bold values denote the highest separability score within each row. Underlined values highlight the absolute best score within the single-camera and multi-camera configurations, respectively. The results indicate that a single camera is often sufficient for accurate assessment, provided the patient maintains the optimal standing orientation.

Panel of Figure 7 — Figure 7: Repetition analysis of feature F3 (Lateral head tilt) from Exercise 1 across three simultaneous viewpoints ( $VP_{Cam1}=-90^{\circ}$ , $VP_{Cam2}=-45^{\circ}$ , and $VP_{Cam3}=0^{\circ}$ ). Baseline values shift distinctly with the camera perspective. In the middle plot of (a), normal repetitions spread out to create a wider confidence interval, yet the erroneous repetition trajectory remains completely separate and easily detectable. Conversely, Person 2 in (b) shows a smaller separation between correct and erroneous trajectories due to a less pronounced error execution, highlighting the need for optimal camera placement. Across all setups, the frontal $0^{\circ}$ viewpoint provides the most distinct signal, while a comparison between (a) and (c) demonstrates that MediaPipe’s 3D estimates introduce substantial noise and signal instability compared to 2D tracking.

Explore similar work

Jun 22, 2026cs.CV

Can Single-View Mesh Reconstruction Generalize to Robot Camera Rotation?

Single-view mesh reconstruction predicts object meshes and spatial layouts from a single observation, making it attractive for fast robot spatial reasoning and real-to-sim digital twins. However, robot-mounted cameras naturally rotate during manipulation and navigation, while learned single-view reconstruction models often rely on view-dependent priors and may generalize poorly to out-of-distribution camera rotations. Such rotations can introduce 3D inconsistencies, incorrect layouts, and violations of physical constraints, but this failure mode remains under-evaluated. We introduce an evaluation protocol with controlled axis-wise roll, pitch, and yaw sweeps to trace errors in monocular depth estimation (MDE), canonical object meshes, camera-space layout, and physical plausibility within a representative SAM3D-style pipeline. On the Aria Digital Twin dataset and a real Franka wrist-camera sequence, camera rotations induce MDE distortion, layout drift, and collision penetration, while canonical mesh predictions remain relatively stable. A two-stage SAM3D+FoundationPose pipeline is more robust than one-stage feed-forward layout prediction, and our Gravity-Aware Refinement reduces one-stage pairwise ICP-based layout-orientation error by 47.1

\%

. Our evaluation reveals that current single-view mesh reconstruction methods generalize poorly to robot camera rotation, and suggests that explicit gravity cues are important for reliable robotic single-view mesh reconstruction.

Yu Zhan, Guangcheng Chen, Hanjing Ye +4

Shenzhen Key Laboratory of Robotics and Computer Vision, Southern University of Science and Technology (SUSTech) · Pengcheng Laboratory

3D Reconstruction

Ask PDF

Sep 21, 2026cs.CV

STA-TFM: Spatio-Temporal Aggregation Across Views TransForMer for Pose Estimation

Monocular 3D human pose estimation (HPE) remains challenging due to depth ambiguity, occlu- sions, and the need for temporal consistency. While multi-view methods provide superior accuracy over monocular approaches, they often require complex setups. We introduce STA-TFM, a transformer-based architecture that combines spatial and temporal information for multi-view pose estimation. The approach leverages DSTformer, a monocular feature extractor, to capture long-range pose dependencies within each view. A fusion transformer then aggregates information across views to produce coherent 3D estimates. To address training data scarcity, we use a data generation pipeline that transforms any existing 3D pose dataset into multi-view setups with controllable parameters. Experiments on various datasets demonstrate that STA-TFM outperforms existing camera-parameter-free multi-view methods. STA-TFM achieves 50.9% and 49.5% reductions in mean per joint position error (MPJPE) and mean per joint velocity error (MPJVE) on the DHP19 dataset. Furthermore, it achieves 6.7% and 7.7% respective reductions on HAA4D, and a 15.2% MPJPE reduction on TotalCapture. STA-TFM handles noisy and missing 2D inputs, supporting potential deployment in healthcare monitoring, athletic assessment, and immersive technologies. Code, training checkpoints, and data are available at https://zenodo.org/records/22832620.

Mena Kamel, Natalie Won, Amrut Sarangi +2

SanofiDigital R&D, K703, Industriepark Höchst, 65929 Frankfurt am Main, Germany · SanofiDigital R&D, Carrer de Rosselló i Pòrcel, 21, Nou Barris, 08016 Barcelona, Spain

Wifi-Csi 3D Human Pose Estimation Multi-View Uncertain Joint Stiffness Transformer Architectures

Ask PDF

Sep 30, 2026cs.CV

Agentic Relative Camera Pose Estimation via Learned Ranking and Verification

A wide range of approaches have been developed for camera pose estimation, including correspondence-based methods, end-to-end pose regression, and recent 3D geometric foundation models. Our key observation is that no single estimator is optimal for diverse challenges, such as wide baselines, lack of texture, appearance changes, and occlusions. Further analysis reveals substantial performance variation across both benchmarks and individual image pairs, with different estimators exhibiting complementary strengths. We introduce PoseAgent, an agentic framework for relative camera pose estimation that dynamically orchestrates pose estimators through learnable ranking and verification. Given an image pair, a profiling agent first extracts appearance, semantic, and geometric features relevant to pose estimation, e.g., scene type. A learned ranking agent then predicts the relative competence of multiple pose estimators given the image-pair profile. The top-ranked estimator is executed, and its predicted pose is assessed by a learned verification agent that estimates the corresponding pose error. When verification fails, PoseAgent adaptively invokes lower-ranked estimators until a candidate is accepted or the execution budget is reached. For pose verification, our verification network predicts pose errors more accurately than prior models. For pose estimation, PoseAgent improves AUC@5 degree up to 4.2% over the strongest standalone estimator on each of ARKitScenes, MegaDepth, ScanNet++, and RealEstate10K. On ARKitScenes, PoseAgent also outperforms VLM-based agents, which include a VLM ranker with the same verifier and fallback policy. These results demonstrate the effectiveness of our learned ranking and verification.

Zhining Gu, Shangjie Du, Weimin Qiu +3

Arizona State University · University of California, Merced · Lund University +1

Robotic Perception Agent2Agent

Ask PDF