cs.CVApr 1, 2026
SaveNonlinear Methods for Analyzing Pose in Behavioral Research
Organizations: Modeling and Engineering Risk and Complexity, Scuola Superiore Meridionale, Naples, Italy. · 2*School of Psychological Sciences, Macquarie University, Sydney, Australia. · Performance and Expertise Research Centre, Macquarie University, Sydney, Australia. · Lifespan Health and Wellbeing Research Centre, Macquarie University, Sydney, Australia. ·
Abstract
Advances in markerless pose estimation have made it possible to capture detailed human movement in naturalistic settings using standard video, enabling new forms of behavioral analysis at scale. However, the high dimensionality, noise, and temporal complexity of pose data raise significant challenges for extracting meaningful patterns of coordination and behavioral change. This paper presents a general-purpose analysis pipeline for human pose data, designed to support both linear and nonlinear characterizations of movement across diverse experimental contexts. The pipeline combines principled preprocessing, dimensionality reduction, and recurrence-based time series analysis to quantify the temporal structure of movement dynamics. To illustrate the pipeline's flexibility, we present three case studies spanning facial and full-body movement, 2D and 3D data, and individual versus multi-agent behavior. Together, these examples demonstrate how the same analytic workflow can be adapted to extract theoretically meaningful insights from complex pose time series.
Figures & tables
Figure 1: Effects of gap length on RQA stability and recurrence structure. Top Left: Raw time series showing original signal used. Bottom Left: Relative error in Recurrence Rate (RR) and Determinism (DET) as a function of the embedding delay . Curves show the mean error across 30 trials, with error bars indicating standard deviation. The dashed line marks the theoretical threshold , beyond which embedding vectors begin to span the gap; the dotted line marks a 5% error reference. RR error increases markedly once gaps exceed , while DET remains comparatively stable. Right: Recurrence plots illustrating how interpolation affects the underlying recurrence structure. The top-left panel shows the baseline RP (no gaps). Remaining panels show discrete difference maps (gaps-baseline) for centered gaps of 0.5 , 2 , and 4 . Colors denote recurrence changes: blue = lost recurrence, white = unchanged, red = gained recurrence. Small gaps introduce minimal distortions, whereas larger gaps generate structured artifacts, including diagonal streaks and localized regions of false recurrence.
Figure 2: Procrustes alignment of 2D pose data. Synthetic pose skeletons illustrate the three stages of Procrustes alignment. (A) Raw: Two participants with different positions, scales, and orientations relative to the template (black). (B) After Translation: Poses are centered to the template centroid. (C) After Scale: Poses are normalized to match the template size. (D) After Rotation: Poses are rotated to align with the template orientation. This alignment procedure removes between-participant differences in position, body size, and camera angle, enabling direct comparison of pose dynamics.
Figure 3: Illustration of recurrence quantification analysis (RQA) workflows. Each approach begins with a time series (left), which is reconstructed into a state space via time-delay embedding (middle), and then converted into a recurrence plot (right). Auto-RQA (top row) quantifies recurrent structure within a single trajectory, capturing the temporal organization of one system’s dynamics. Cross-RQA (middle row) embeds two trajectories into a shared phase space and identifies when one system revisits states previously visited by the other, allowing coordinated patterns to be quantified. Multidimensional RQA (bottom row) embeds multiple time series simultaneously, treating them as components of a single dynamical system to assess collective recurrence structure across signals. Together, these methods extend recurrence analysis from individual dynamics to coupled and high-dimensional systems, providing flexible tools for movement and coordination research.
Figure 4: Illustrative Average Mutual Information (AMI) curves demonstrating common patterns encountered when selecting the embedding delay for pose-derived time series. (A) A clear first minimum, the canonical textbook case, where may be chosen directly at the first local minimum. (B) A broad plateau or shallow trough, characteristic of many natural movement signals; here a range of delays provides comparable information content, and robustness should be assessed across this interval. (C) An oscillatory or quasi-periodic AMI profile with multiple local minima; in such cases, the earliest stable minimum or plateau region is typically preferred. (D) A slow, almost flat decay indicating strong temporal redundancy at the sampling rate; delays may be selected at the onset of the plateau or based on a fixed AMI threshold. Across all panels, shaded bands denote plausible delay ranges, emphasizing that for non-rhythmic pose data, selecting a single is rarely justified. Robust analyses should verify that substantive results are stable across a small set of reasonable delays. The shaded bands are illustrative rather than prescriptive and are intended to denote plausible regions rather than precise optima.
Figure 5: Analysis pipeline for pose-based behavioral research. The five-stage workflow is applied across three case studies to demonstrate adaptability across recording configurations and movement scales. Stages 1–3 represent the common preprocessing sequence, including low-pass filtering (LPF), confidence-based masking, and Procrustes alignment to ensure spatial and temporal standardization. Stage 4 extracts linear kinematic features to quantify movement magnitude and variability. Stage 5 employs recurrence quantification analysis (RQA/CRQA) to characterize the temporal organization and coordination dynamics of the behavioral signal.
Figure 6: Experimental setup and facial pose features used for analysis. Left: Participant performing the Multi-Attribute Task Battery (MATB) aviation simulation while facial behavior was recorded with a consumer-grade webcam during multitasking under varying cognitive load. Right: Schematic illustration of the derived facial pose features used in analysis. Two-dimensional facial landmarks were grouped into functionally meaningful composites capturing blink aperture, mouth opening, pupil displacement, and rigid head motion (translation and rotation). These features formed the basis for subsequent linear kinematic analysis, recurrence quantification analysis (RQA), and cross-recurrence quantification analysis (CRQA) of gaze-head coordination.
Figure 7: Representative linear and recurrence-based facial dynamics across cognitive load levels. Group-averaged metrics (mean SEM) illustrating load-dependent changes in facial behavior during the MATB task. From left to right: average pupil velocity, pupil recurrence rate (%REC), cross-recurrence rate between gaze (pupil) and head motion, and pupil determinism (%DET), shown for low, moderate, and high workload conditions. Linear kinematic measures show increased movement magnitude under high load. Recurrence-based measures reveal an initial reduction in repetitiveness from low to moderate load followed by structural reorganization at high load, reflected in increased determinism. In contrast, cross-recurrence between gaze and head decreases under load and does not exhibit comparable recovery, indicating progressive inter-system decoupling despite intra-system reorganization.
Figure 8: The experimental set-up for case study 2. Interactants sat facing one another and conversed while exposed to various background noise conditions: Office (60 dB SPL), Cafe (70 dB SPL), Food Court (80 dB SPL) and Party (87.5 dB SPL).
Figure 9: Select group-averaged linear metrics (mean SEM) illustrating changes in arm and upper body velocity magnitude across the background noise levels. From left to right: RMS of individual-level arm velocity magnitude and dyadic cross-correlation of upper body velocity magnitude. Individual linear kinematic measures generally increased as background noise level rose. However, the effects for dyadic linear measures were often inconsistent, with only the upper body showing a reduction in coupling as noise level increased.
Figure 10: Select group-averaged CRQA metrics (mean SEM) illustrating changes in arm velocity magnitude across the background noise levels. From left to right: %REC of arm velocity magnitude and %DET of arm velocity magnitude. Non-linear interpersonal coordination of the arms and upper body generally increased as background noise level rose.
Figure 11: The three experimental conditions for the mirror game: face-to-face (FTF; mutual visual access), unidirectional (uni; asymmetric visual coupling with leader designation A or B), and back-to-back (BTB; no visual information).
Figure 12: Principal Movements (PMs) extracted from Mirror Game full-body coordination. The first 14 principal components visualized as min (gray) vs. max (black) postures in frontal view. Each PM is scaled by an amplification factor to achieve a target RMS displacement of 0.25 units for visual clarity. The 14 components together account for the majority of postural variance across all dyads and conditions.
| Movement Type | Feature | Uni ( ) | FTF ( ) | ||
|---|---|---|---|---|---|
| Displacement | Mean | 0.0022 (0.0005) | 0.0031 (0.0005) | ||
| RMS | 0.0037 (0.0007) | 0.0047 (0.0007) | |||
| Velocity | Mean | 0.067 (0.015) | 0.093 (0.015) | ||
| RMS | 0.110 (0.021) | 0.141 (0.021) | |||
| Acceleration | Mean | 0.740 (0.162) | 0.889 (0.162) | ||
| RMS | 1.237 (0.252) | 1.380 (0.252) |
Table 1: Condition effects on subset-level kinematic features from linear mixed-effects models
Figure 13: Effects of visual coupling on movement magnitude and dyadic coordination during the 3D Mirror Game. Bar plots display linear mixed-effects model coefficients ( ) for unidirectional (Uni) and face-to-face (FTF) conditions relative to the back-to-back (BTB) baseline. From left to right: acceleration (RMS), cross-recurrence rate (%REC), and maximum diagonal line length (Lmax). All metrics were computed from a predefined five-keypoint subset (head, left/right wrists, left/right ankles) and averaged across keypoints prior to model fitting (see Methods for details).