Nonlinear Methods for Analyzing Pose in Behavioral Research
Authors: Carter Sale, Margaret C. Macpherson, Gaurav Patil, Kelly Miles, Rachel W. Kallen, Sebastian Wallot, Michael J. Richardson
Organizations: Modeling and Engineering Risk and Complexity, Scuola Superiore Meridionale, Naples, Italy. · 2*School of Psychological Sciences, Macquarie University, Sydney, Australia. · Performance and Expertise Research Centre, Macquarie University, Sydney, Australia. · Lifespan Health and Wellbeing Research Centre, Macquarie University, Sydney, Australia. · Macquarie University Hearing Research Centre, Macquarie University, Sydney, Australia. · Department of Linguistics, Macquarie University, Sydney, Australia. · Institute for Sustainability Psychology, Leuphana University of Lüneburg, Lüneburg, Germany.
Advances in markerless pose estimation have made it possible to capture detailed human movement in naturalistic settings using standard video, enabling new forms of behavioral analysis at scale. However, the high dimensionality, noise, and temporal complexity of pose data raise significant challenges for extracting meaningful patterns of coordination and behavioral change. This paper presents a general-purpose analysis pipeline for human pose data, designed to support both linear and nonlinear characterizations of movement across diverse experimental contexts. The pipeline combines principled preprocessing, dimensionality reduction, and recurrence-based time series analysis to quantify the temporal structure of movement dynamics. To illustrate the pipeline's flexibility, we present three case studies spanning facial and full-body movement, 2D and 3D data, and individual versus multi-agent behavior. Together, these examples demonstrate how the same analytic workflow can be adapted to extract theoretically meaningful insights from complex pose time series.
Figures & tables
Figure 1: Effects of gap length on RQA stability and recurrence structure. Top Left: Raw time series showing original signal used. Bottom Left: Relative error in Recurrence Rate (RR) and Determinism (DET) as a function of the embedding delay τ . Curves show the mean error across 30 trials, with error bars indicating standard deviation. The dashed line marks the theoretical threshold (m−1)τ , beyond which embedding vectors begin to span the gap; the dotted line marks a 5% error reference. RR error increases markedly once gaps exceed (m−1)τ , while DET remains comparatively stable. Right: Recurrence plots illustrating how interpolation affects the underlying recurrence structure. The top-left panel shows the baseline RP (no gaps). Remaining panels show discrete difference maps (gaps-baseline) for centered gaps of 0.5 τ , 2 τ , and 4 τ . Colors denote recurrence changes: blue = lost recurrence, white = unchanged, red = gained recurrence. Small gaps introduce minimal distortions, whereas larger gaps generate structured artifacts, including diagonal streaks and localized regions of false recurrence.
Figure 2: Procrustes alignment of 2D pose data. Synthetic pose skeletons illustrate the three stages of Procrustes alignment. (A) Raw: Two participants with different positions, scales, and orientations relative to the template (black). (B) After Translation: Poses are centered to the template centroid. (C) After Scale: Poses are normalized to match the template size. (D) After Rotation: Poses are rotated to align with the template orientation. This alignment procedure removes between-participant differences in position, body size, and camera angle, enabling direct comparison of pose dynamics.
Figure 3: Illustration of recurrence quantification analysis (RQA) workflows. Each approach begins with a time series (left), which is reconstructed into a state space via time-delay embedding (middle), and then converted into a recurrence plot (right). Auto-RQA (top row) quantifies recurrent structure within a single trajectory, capturing the temporal organization of one system’s dynamics. Cross-RQA (middle row) embeds two trajectories into a shared phase space and identifies when one system revisits states previously visited by the other, allowing coordinated patterns to be quantified. Multidimensional RQA (bottom row) embeds multiple time series simultaneously, treating them as components of a single dynamical system to assess collective recurrence structure across signals. Together, these methods extend recurrence analysis from individual dynamics to coupled and high-dimensional systems, providing flexible tools for movement and coordination research.
Figure 4: Illustrative Average Mutual Information (AMI) curves demonstrating common patterns encountered when selecting the embedding delay τ for pose-derived time series. (A) A clear first minimum, the canonical textbook case, where τ may be chosen directly at the first local minimum. (B) A broad plateau or shallow trough, characteristic of many natural movement signals; here a range of delays provides comparable information content, and robustness should be assessed across this interval. (C) An oscillatory or quasi-periodic AMI profile with multiple local minima; in such cases, the earliest stable minimum or plateau region is typically preferred. (D) A slow, almost flat decay indicating strong temporal redundancy at the sampling rate; delays may be selected at the onset of the plateau or based on a fixed AMI threshold. Across all panels, shaded bands denote plausible delay ranges, emphasizing that for non-rhythmic pose data, selecting a single τ is rarely justified. Robust analyses should verify that substantive results are stable across a small set of reasonable delays. The shaded bands are illustrative rather than prescriptive and are intended to denote plausible regions rather than precise optima.
Figure 5: Analysis pipeline for pose-based behavioral research. The five-stage workflow is applied across three case studies to demonstrate adaptability across recording configurations and movement scales. Stages 1–3 represent the common preprocessing sequence, including low-pass filtering (LPF), confidence-based masking, and Procrustes alignment to ensure spatial and temporal standardization. Stage 4 extracts linear kinematic features to quantify movement magnitude and variability. Stage 5 employs recurrence quantification analysis (RQA/CRQA) to characterize the temporal organization and coordination dynamics of the behavioral signal.
Figure 6: Experimental setup and facial pose features used for analysis. Left: Participant performing the Multi-Attribute Task Battery (MATB) aviation simulation while facial behavior was recorded with a consumer-grade webcam during multitasking under varying cognitive load. Right: Schematic illustration of the derived facial pose features used in analysis. Two-dimensional facial landmarks were grouped into functionally meaningful composites capturing blink aperture, mouth opening, pupil displacement, and rigid head motion (translation and rotation). These features formed the basis for subsequent linear kinematic analysis, recurrence quantification analysis (RQA), and cross-recurrence quantification analysis (CRQA) of gaze-head coordination.
Figure 7: Representative linear and recurrence-based facial dynamics across cognitive load levels. Group-averaged metrics (mean ± SEM) illustrating load-dependent changes in facial behavior during the MATB task. From left to right: average pupil velocity, pupil recurrence rate (%REC), cross-recurrence rate between gaze (pupil) and head motion, and pupil determinism (%DET), shown for low, moderate, and high workload conditions. Linear kinematic measures show increased movement magnitude under high load. Recurrence-based measures reveal an initial reduction in repetitiveness from low to moderate load followed by structural reorganization at high load, reflected in increased determinism. In contrast, cross-recurrence between gaze and head decreases under load and does not exhibit comparable recovery, indicating progressive inter-system decoupling despite intra-system reorganization.
Figure 8: The experimental set-up for case study 2. Interactants sat facing one another and conversed while exposed to various background noise conditions: Office (60 dB SPL), Cafe (70 dB SPL), Food Court (80 dB SPL) and Party (87.5 dB SPL).
Figure 9: Select group-averaged linear metrics (mean ± SEM) illustrating changes in arm and upper body velocity magnitude across the background noise levels. From left to right: RMS of individual-level arm velocity magnitude and dyadic cross-correlation of upper body velocity magnitude. Individual linear kinematic measures generally increased as background noise level rose. However, the effects for dyadic linear measures were often inconsistent, with only the upper body showing a reduction in coupling as noise level increased.
Figure 10: Select group-averaged CRQA metrics (mean ± SEM) illustrating changes in arm velocity magnitude across the background noise levels. From left to right: %REC of arm velocity magnitude and %DET of arm velocity magnitude. Non-linear interpersonal coordination of the arms and upper body generally increased as background noise level rose.
Figure 11: The three experimental conditions for the mirror game: face-to-face (FTF; mutual visual access), unidirectional (uni; asymmetric visual coupling with leader designation A or B), and back-to-back (BTB; no visual information).
Figure 12: Principal Movements (PMs) extracted from Mirror Game full-body coordination. The first 14 principal components visualized as min (gray) vs. max (black) postures in frontal view. Each PM is scaled by an amplification factor to achieve a target RMS displacement of 0.25 units for visual clarity. The 14 components together account for the majority of postural variance across all dyads and conditions.
Movement Type
Feature
Uni β ( SE )
p
FTF β ( SE )
p
Displacement
Mean
0.0022 (0.0005)
<.001
0.0031 (0.0005)
<.001
RMS
0.0037 (0.0007)
<.001
0.0047 (0.0007)
<.001
Velocity
Mean
0.067 (0.015)
<.001
0.093 (0.015)
<.001
RMS
0.110 (0.021)
<.001
0.141 (0.021)
<.001
Acceleration
Mean
0.740 (0.162)
<.001
0.889 (0.162)
<.001
RMS
1.237 (0.252)
<.001
1.380 (0.252)
<.001
Table 1: Condition effects on subset-level kinematic features from linear mixed-effects models
Figure 13: Effects of visual coupling on movement magnitude and dyadic coordination during the 3D Mirror Game. Bar plots display linear mixed-effects model coefficients ( β±SE ) for unidirectional (Uni) and face-to-face (FTF) conditions relative to the back-to-back (BTB) baseline. From left to right: acceleration (RMS), cross-recurrence rate (%REC), and maximum diagonal line length (Lmax). All metrics were computed from a predefined five-keypoint subset (head, left/right wrists, left/right ankles) and averaged across keypoints prior to model fitting (see Methods for details).
Resistance training can be a high risk activity, and safe form is essential to avoiding injury. Laboratory-based movement analysis provides quantitive technique assessment, yet is not easily accessible. Markerless pose estimation infers body landmarks from images or video without physical markers and could offer a feasible alternative for technique assessment. We present a pose estimation framework to evaluate resistance-training technique from ordinary video footage. Using BlazePose, anatomical landmarks were extracted from squat, bench press, and deadlift videos and converted into joint-angle trajectories, with the squat serving as the primary case study. Trajectories were assessed against a defined reference repetition using root mean square error (RMSE). Results show that the framework recovers meaningful kinematic patterns for the squat and deadlift, enabling quantitative comparison between repetitions and identification of technique variability within a set. Performance depended strongly on camera orientation and visual occlusion, with non-sagittal views distorting 2D joint-angle estimates. The findings demonstrate that markerless pose estimation can support accessible biomechanical assessment outside laboratory environments.
Joseph Turner, Jeff Clark, Nawid Keshtmand
School of Engineering Mathematics and Technology, University of Bristol, UK
Human pose estimation has advanced significantly due to the development of deep learning models, increased data availability, and improved computing resources. These developments have led to highly accurate body tracking systems with direct applications in sports analysis and performance evaluation. The VideoRun2D Demo performs a biomechanical analysis during sprints using different human pose estimators. The proposed framework was evaluated using human pose trackers and expert manual annotations. The tested framework uses 314 sprints from 44 professional runners, focusing on two key joint angles in sprint biomechanics: 1) hip flexion/extension and 2) knee flexion/extension. The framework also includes a post-processing module for outlier detection. The tested results demonstrate that the average root-mean-square errors range from 11.46° to 5.83° for the best trackers. When integrated with the post-processing modules, these errors can be reduced to 9.87° and 5.30°, respectively. The VideoRun2D Demo findings suggest that human pose-tracking approaches can be valuable resources for the biomechanical analysis of running.
Luis F. Gomez, Julian Fierrez, Roberto Daza +5
BiometricsAI, Universidad Autonoma de Madrid (UAM), Spain
High-quality dynamic human pose annotation equips AI with precise motion kinematics to enable human behavior mastery, yet remains labor-intensive and time-consuming. Current annotation tools either lack temporal correction propagation or fail in multi-person scenarios, necessitating excessive manual intervention. In this paper, we introduce IMPose, an interactive tool for multi-person dynamic pose annotation. It features a dual-level tracking mechanism that propagates one-frame multi-person pose corrections from annotators across entire videos. The keypoint-level ensures corrections temporal propagation via sequential modeling, while the instance-level employs keypoint-aware embedding with relative positional encoding to maintain multi-person cross-frame consistency. To further improve robustness, IMPose maintains historical pose and instance cues in a trajectory bank, which enhances long-range temporal association and stabilizes annotation in challenging cases such as occlusion and motion blur. By converting sparse human corrections into dense and coherent pose trajectories, our framework significantly reduces repeated manual refinement across frames. Extensive experiments show that IMPose consistently achieves a strong accuracy efficiency trade off under different interaction budgets, demonstrating particular advantages in low click annotation settings. IMPose achieves high precision annotation with high efficiency, requiring only 27 clicks per 1,050 frame video on 3DPW and 3 clicks per tracklet per 84-frame on PoseTrack21. We further expand PoseTrack21 with 188K pose instances (3.55M keypoints) at a minimal cost of 10 annotators in 10 hours. The annotation tool, codes, and extended dataset will be open-sourced.
Haoyang Ge, Jian Ma, Ziwen Wang +5
Tianjin University, Tianjin 300350, China · Tiandi, Tianjin, China · Zhongguancun Academy, Beijing 100094, China