How learning progresses over time and varies across individuals is a longstanding question in cognitive and behavioral science. However, a major challenge is identifying observable behavioral measures that faithfully capture learning progression. Existing approaches commonly rely on predefined behavioral features or task-level performance, while fine-scale posture and movement in freely behaving animals remain less explored. In this paper, we ask: can short-term movement trajectories reveal learning progress and individual identity? We introduce a self-supervised representation learning framework built on a simple hypothesis: behavioral characteristics associated with identity and learning remain relatively stable over nearby time intervals. Using temporal proximity as supervision, our framework learns compact representations directly from movement trajectories without manually defining behavioral states or kinematic features. Using behavior data from freely behaving rats, we show that the learned representations not only improve prediction of identity and learning-related readouts over existing methods, but also provide a tool for investigating broader scientific questions about when and how these signals appear in behavior. Specifically, trajectories as short as one second contain informative learning and identity signals. Egocentric pose dynamics are more predictive of identity, whereas allocentric motion is more predictive of learning-related variation, and both signals are strongest at decision junctions in the alternation cohort. We further use a representation-conditioned diffusion model to inspect movement patterns associated with identity and learning progress. Our findings establish short-term movement as an informative behavioral "fingerprint" for studying learning and individual differences. Our code is available at https://github.com/pizhn/TRAIL.
Figures & tables
Figure 1: Dataset and problem setting. Left: Repeated pose recordings span animals and days under two reward rules over the Y-shape spatial learning dataset: alternation and probabilistic rewards. L, C, and R denote the left, center, and right arms, respectively. Right: A compact representation of short-term movement trajectories is learned and evaluated for inferring identity, recording day, and performance (e.g., proficiency, success rate, and reward).
Figure 2: TRAIL high-level architecture.
Method
Identity
Tasks
Tasks
Frame
Seq.
Acc ↑
MSE ↓
F1% ↑
F1% ↑
F1% ↑
Short-term Context (2 s)
Classic
LG-SSM
65.32
9.42
21.01
13.17
56.25
AR-HMM
45.62
9.36
22.50
14.70
57.61
ProMPs
67.65
9.43
22.59
14.73
57.96
Keypoint-MoSeq
63.50
9.39
22.42
13.67
61.79
Table 2: Performance on the MABe22 Mouse Triplets dataset with short- and long-term contexts. MSE values scaled by 100 .
Figure 3: Effects of temporal settings. Left: Positive-pair window width, expressed as a percentage of the recording partition. Right: Trajectory length. Dashed lines mark the default settings.
Figure 5
Figure 6: t-SNE of 3-second trajectories. Left: Across animals. Right: Across days per rat.
Table 3: Behavior analysis across maze regions, recording days, and movement motifs. Day prediction is omitted from the per-day analysis. Bold and underlined values indicate the best and second-best results within each subtable, based on unrounded values.
Figure 7: Turning movement transformations across days and individuals. Left: Cross-day Body Trajectory with body speed, the magnitude of body-keypoint velocity. Right: Cross-individual Local Pose with body ratio, the nose–neck distance divided by the neck–body distance. Obs., Recon., Trans., and Ref. denote observed, reconstructed, transformed, and reference movements, respectively.
Appendix figures & tables25 assets
Supplementary material from the paper’s appendix.
Appendix
Identity
Animal-days
Sessions
Frames
Duration (min)
Alternation Reward
Rat A
5
15
98,708
148.9
Rat B
5
20
135,646
205.2
Rat C
5
20
131,844
198.9
Rat D
5
20
127,143
191.8
Rat E
5
20
133,732
202.2
Appendix
Table 4: Statistics for the two reward datasets. Frame counts include all data partitions; durations are computed from session timestamps.
Reward rule
Train
Validation
Test
Alternation Reward
604,223
37,752
113,357
Probabilistic Reward
823,294
51,436
154,462
Total
1,427,517
89,188
267,819
Appendix
Table 5: Frame counts by partition across two datasets. Rounding at session boundaries produces small deviations from the nominal split proportions.
Table 6: Hyperparameter search. Candidate values are grouped by model. Loss weights are listed as (reconstruction, contrastive); Keypoint-MoSeq iteration counts are listed as (AR, full model). Hidden width and HoA weight are fixed at the values shown.
Setting
Primary
Supplementary
Coordinate projection
Linear
Two-layer MLP
Maze tokens
Waypoints
Vertices and edge descriptors
First-query key normalization
None
Per-head layer normalization
Reconstruction input
Restored representation and seven queries
Full encoder query bank
Reconstruction coefficient
1
0.5
Identity
84.00
84.07
Appendix
Table 7: Encoder configurations used in the additional experiments. Both use multi-frequency coordinate features, 64-dimensional tokens, eight queries, four encoder layers, and contrastive temperature 0.1. The performance rows use the same 105,882 test windows and state-space Proficiency labels. Identity is percentage accuracy; the remaining targets are MSE.
Cohort
Target
Estimate
Confidence interval
Alternation
Identity (%)
84.00
[76.24,91.10]
Alternation
Day
1.2579
[0.9569,1.5937]
Alternation
Proficiency
.0156
[.0109,.0223]
Alternation
Success Rate
0.13
[0.09,0.18]
Alternation
Reward
0.6090
[0.4488,0.8493]
Probabilistic
Identity (%)
85.87
[82.59,88.81]
Appendix
Table 9: TRAIL prediction across reward conditions. Estimates use the held-out predictions from the cohort comparisons; confidence intervals are obtained by bootstrapping animals at the 95% level. Identity is percentage accuracy; the remaining targets are MSE.
Figure 8: Paired comparisons in the primary alternation cohort. Points show pooled MSE reductions relative to each baseline; horizontal bars show animal-bootstrap 95% confidence intervals. All methods use the same 105,882 test windows.
Representation
Accuracy (%)
Confidence interval
BAMS, short context
63.08
[61.72,64.46]
TRAIL , short context
73.65
[72.28,75.01]
Accuracy improvement (pp)
10.56
[9.68,11.45]
Appendix
Table 11: Within-clip identity on MABe22 Mouse Triplets. Estimates average the accuracies of the 1,058 test clips. The 95% confidence intervals use paired clip-bootstrap resampling; mouse slots are defined separately in each clip. The final row reports TRAIL’s accuracy improvement over BAMS in percentage points.
Target
TRAIL
BAMS
Reduction
Confidence interval
Animals
Day
1.765
1.932
0.167
[0.071,0.255]
5 of 6
Proficiency
.02032
.02460
.00428
[.00124,.00897]
5 of 6
Reward
0.760
1.179
0.419
[0.131,0.788]
6 of 6
Appendix
Table 13: Prediction on held-out sessions. MSE is averaged equally across animals. MSE reductions measure the improvement with TRAIL over BAMS, with paired animal-bootstrap 95% confidence intervals. The comparison uses 183,884 windows, except proficiency, which has 181,639 complete cases. BAMS uses local coordinates and maze context. The final column counts animals with lower MSE under TRAIL.
Readout
Day
Proficiency
Animal-specific mean
2.012
.02571
14 kinematic features
1.950
.02541
TRAIL
1.756
.02046
Appendix
Table 14: Day and proficiency readouts on held-out sessions. MSE is averaged equally across six animals. The lower block reports the reduction in MSE achieved by TRAIL relative to each baseline, with paired animal-bootstrap 95% confidence intervals. Day uses 190,814 test windows, and Proficiency uses 188,261 complete cases.
Representation
Balanced acc. (%)
Gain (pp)
Confidence interval
Chance
16.67%
58.21 pp
[44.08,72.33]
Kinematic features
25.90%
48.98 pp
[33.91,62.35]
BAMS (local + maze)
38.89%
35.99 pp
[20.80,55.76]
TRAIL
74.87%
–
–
Appendix
Table 15: Identity and rewarded-count prediction on a held-out day. Days 1–3 train the encoders and probes; day 4 selects regularization. Results use 139,875 day-5 windows. TRAIL improves identity accuracy in all six animals; gains are measured against each listed baseline in percentage points. Rewarded-count MSE is averaged equally across animals; five of six animals improve. Intervals are paired animal-bootstrap 95% confidence intervals.
Representation
Aligned
Permuted
Reduction
Confidence interval
BAMS (local + maze)
1.593
1.661
0.068
[0.0031,0.1604]
TRAIL
0.948
0.980
0.032
[0.0029,0.0683]
Appendix
Table 16: Within-recording time alignment. Errors are mean per-animal rewarded-count MSE. The reduction compares correctly aligned features with permuted features; intervals are animal-bootstrap 95% confidence intervals. Each method improves in five of six animals (one-sided permutation p=0.001 ).
Target
TRAIL
Kinematics
Reduction
Confidence interval
Day
1.621
1.861
0.241
[0.182,0.284]
Proficiency
.00944
.01112
.00167
[.00073,.00261]
Reward
0.860
1.025
0.165
[0.082,0.249]
Success Rate
0.05
0.07
0.02
[0.01,0.02]
Appendix
Table 17: Transfer with calibration on four new animals. MSE is averaged equally across the four animals using 74,768 test windows. MSE reductions measure the improvement with TRAIL over kinematic features, with paired animal-bootstrap 95% confidence intervals.
Table 18: Ablation studies on loss function and multi-frequency modeling . Bold : best based on unrounded values.
Figure 9: Latent representation trade-off. The default representation has 16 dimensions.
View
Identity
Day
Proficiency
Local Pose
77.1
1.6047
.0189
[66.2,87.4]
[1.3303,1.9455]
[.0124,.0277]
Global Trajectory
72.1
1.4733
.0172
[61.4,82.3]
[1.3063,1.6259]
[.0128,.0231]
Full Movement
84.0
1.2579
.0156
[76.2,91.1]
[0.9569,1.5937]
[.0109,.0223]
Appendix
Table 21: Prediction from local pose and global trajectory. Pooled estimates use the same 105,882 test windows as figure 5 . Intervals below the estimates are animal-bootstrap 95% confidence intervals. Identity is percentage accuracy; Day and Proficiency are MSE.
Figure 10: Movement inputs and reconstruction. Points show pooled MSE reductions relative to each ablated model; horizontal bars show animal-bootstrap 95% confidence intervals. All methods use the same 105,882 test windows. The comparisons use the runs in figures 5 and 18(a) .
Group
Rat A
Rat B
Rat C
Rat D
Rat E
Rat F
Proficiency SD
Arm segment
63
97
101
122
96
79
.174
Arm endpoint
320
449
396
376
404
432
.155
Junction
42
40
72
52
77
43
.157
Day 1
82
158
110
114
120
116
.105
Day 2
84
107
114
109
113
108
.165
Day 3
88
97
115
109
116
111
.111
Appendix
Table 23: Composition and target spread in the primary subgroup analysis. Counts include only scored windows. Rat labels follow table 22 . The final column gives the standard deviation of Proficiency across these windows.
Group
n
Identity
Day
Proficiency
Maze region
Arm segment
558
81.7
1.59
.0211
[72.4,90.1]
[1.04,2.11]
[.0109,.0310]
Arm endpoint
2377
83.0
1.20
.0149
[74.1,91.0]
[0.92,1.50]
[.0113,.0203]
Junction
326
88.7
1.20
.0135
Appendix
Table 24: Prediction across maze regions, recording days, and movement motifs. Values are pooled estimates, with animal-bootstrap 95% confidence intervals directly below. Identity is percentage accuracy; Day and Proficiency are MSE. Day prediction is omitted when day defines the group.
Classifier
Rat A
Rat B
Rat C
Rat D
Rat E
Rat F
kNN
17
50
21
40
81
32
SVM-RBF
6
43
11
44
85
49
Decision tree
12
49
24
37
78
34
Random forest
11
49
21
42
84
35
TabPFN v2.6
11
49
16
44
84
42
Appendix
Table 28: Per-animal recall from mean segment lengths. Values are percentages on the default split in table 27 . Rat labels follow table 22 .
Figure 11: t-SNE visualizations of 3-second representations for each baseline. Left: A joint projection colored by rat. Right: Six per-rat projections colored by day. The figure continues on the next page.
Figure 12: t-SNE visualizations of 3-second representations (continued). The final panel repeats TRAIL from figure 6 for direct comparison. Colors follow the same rat and day conventions as above.
Figure 13: Cross-day transformation of turning in local pose. Skeleton sequences and body ratio for the same transformation as the left panel of figure 7 .
Figure 14: Cross-individual transformation of turning in body trajectory. Body-keypoint trajectories and body speed for the same transformation as the right panel of figure 7 .
Figure 15: Cross-day transformation of going straight. Left: Body Trajectory with body speed. Right: Local Pose with body ratio.
Figure 16: Cross-individual transformation of going straight. Left: Body Trajectory with body speed. Right: Local Pose with body ratio.
Human mobility appears highly diverse, yet much of a person's daily mobility can be explained by a small set of recurring behavioral templates, such as commuting, school-centered activities, caregiving, nightlife, or errand patterns. We present IBAD (Interpretable Behavioral Anomaly Detection), a framework that learns interpretable daily mobility templates and represents each individual as a distribution over mixtures of these templates. Rather than focusing on specific locations, IBAD characterizes activities that individuals perform across locations. This approach first discovers global behavioral templates using Latent Dirichlet Allocation (LDA), then employs a hierarchical self-supervised model to learn normal behavior of individuals from their soft behavioral templates. We also introduce a splicing benchmark that creates controlled behavioral mismatches between an individual's historical profile and injected mobility patterns. Experiments on real-world and synthetic datasets show that daily behavior can be effectively decomposed into a small number of interpretable templates. Crucially, we show that the learned behavioral archetypes transfer across distinct geographic and demographic contexts. Furthermore, IBAD maintains a robust competitive performance across all settings. For reproducibility purposes, the code is accessible at ~https://github.com/USC-InfoLab/IBAD.
Bita Azarijoo, John Krumm, Cyrus Shahabi
University of Southern California Los Angeles, California, USA
Understanding animal behavior at an algorithmic level -- what animals attend to, how they form internal representations, and how this maps to action -- remains a central challenge in neuroethology. We explore how well large generative models capture properties of fly social behavior. We combine transformers with agent-centric representations. Models input egocentric sensory observations and output egocentric movements, mirroring biological constraints. Social behavior emerges from agents independently sensing and responding to one another. We demonstrate that this simple model captures many properties of fly behavior, outperforming the world-centric representation and achieving SOTA on MABe 2022-fly. We systematically evaluate properties of the representation important for capturing the behavior distribution through a series of lesioning experiments. These comparisons were enabled by a new, general-purpose library for animal-centric pose forecasting.
Trajectory representation learning underpins a wide range of trajectory analytics tasks; however, most existing self-supervised approaches, whether discriminative or generative, adopt an open-loop paradigm, relying on fixed data augmentations or random masking without feedback, which limits their ability to generalize and scale. We propose REFINE, a simple yet effective Representation lEarning Framework vIa closed-loop traNscription rEfinement for trajectory data. Drawing upon feedback control theory, REFINE tightly couples road-network-aware generative reconstruction with feedback-driven contrastive learning, enabling the model to capture fine-grained local movement semantics and global spatio-temporal dependencies without manually designed augmentation views. We further provide a control-theoretic analysis that establishes convergence guarantees for the proposed closed-loop optimization. Extensive experiments on four real-world datasets demonstrate that REFINE consistently outperforms state-of-the-art methods across multiple downstream tasks while remaining computationally efficient and scalable. This paper is an extended version of REFINE: Trajectory Representation Learning via Closed-Loop Transcription, to appear in KDD 2026.
Sean Bin Yang, Ying Sun, Jilin Hu +5
Aalborg University Aalborg, Denmark · Chongqing Universtiy of Posts and Telecommunications Chongqing, China · East China Normal University Shanghai, China +1