Representing short-term movement to reveal animal identity and learning progress

Ping Zhang; Quang-Huy Nguyen; Yingtie Lei; Sameera Shridhar; Zachary Leveroni; Ellie Quattrocchi; Jai Y Yu; Wei-Lun Chao

cs.LGOct 19, 2026

Representing short-term movement to reveal animal identity and learning progress

Authors: Ping Zhang, Quang-Huy Nguyen, Yingtie Lei, Sameera Shridhar, Zachary Leveroni, Ellie Quattrocchi, Jai Y Yu, Wei-Lun Chao

Organizations: Boston University · The Ohio State University · University of Chicago

Abstract

How learning progresses over time and varies across individuals is a longstanding question in cognitive and behavioral science. However, a major challenge is identifying observable behavioral measures that faithfully capture learning progression. Existing approaches commonly rely on predefined behavioral features or task-level performance, while fine-scale posture and movement in freely behaving animals remain less explored. In this paper, we ask: can short-term movement trajectories reveal learning progress and individual identity? We introduce a self-supervised representation learning framework built on a simple hypothesis: behavioral characteristics associated with identity and learning remain relatively stable over nearby time intervals. Using temporal proximity as supervision, our framework learns compact representations directly from movement trajectories without manually defining behavioral states or kinematic features. Using behavior data from freely behaving rats, we show that the learned representations not only improve prediction of identity and learning-related readouts over existing methods, but also provide a tool for investigating broader scientific questions about when and how these signals appear in behavior. Specifically, trajectories as short as one second contain informative learning and identity signals. Egocentric pose dynamics are more predictive of identity, whereas allocentric motion is more predictive of learning-related variation, and both signals are strongest at decision junctions in the alternation cohort. We further use a representation-conditioned diffusion model to inspect movement patterns associated with identity and learning progress. Our findings establish short-term movement as an informative behavioral "fingerprint" for studying learning and individual differences. Our code is available at https://github.com/pizhn/TRAIL.

Figures & tables

Figure 1: Dataset and problem setting. Left: Repeated pose recordings span animals and days under two reward rules over the Y-shape spatial learning dataset: alternation and probabilistic rewards. L, C, and R denote the left, center, and right arms, respectively. Right: A compact representation of short-term movement trajectories is learned and evaluated for inferring identity, recording day, and performance (e.g., proficiency, success rate, and reward).

Figure 2: TRAIL high-level architecture.

Short-term Context (2 s)
	Method	Identity	Tasks	Tasks	Frame	Seq.
		Acc $\uparrow$	MSE $\downarrow$	F1% $\uparrow$	F1% $\uparrow$	F1% $\uparrow$
Classic	LG-SSM	65.32	9.42	21.01	13.17	56.25
	AR-HMM	45.62	9.36	22.50	14.70	57.61
	ProMPs	67.65	9.43	22.59	14.73	57.96
	Keypoint-MoSeq	63.50	9.39	22.42	13.67	61.79

Table 2: Performance on the MABe22 Mouse Triplets dataset with short- and long-term contexts. MSE values scaled by

100

Figure 3: Effects of temporal settings. Left: Positive-pair window width, expressed as a percentage of the recording partition. Right: Trajectory length. Dashed lines mark the default settings.

Figure 5

Figure 6: t-SNE of 3-second trajectories. Left: Across animals. Right: Across days per rat.

Table 3: Behavior analysis across maze regions, recording days, and movement motifs. Day prediction is omitted from the per-day analysis. Bold and underlined values indicate the best and second-best results within each subtable, based on unrounded values.

Figure 7: Turning movement transformations across days and individuals. Left: Cross-day Body Trajectory with body speed, the magnitude of body-keypoint velocity. Right: Cross-individual Local Pose with body ratio, the nose–neck distance divided by the neck–body distance. Obs., Recon., Trans., and Ref. denote observed, reconstructed, transformed, and reference movements, respectively.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Identity	Animal-days	Sessions	Frames	Duration (min)
Alternation Reward
Rat A	5	15	98,708	148.9
Rat B	5	20	135,646	205.2
Rat C	5	20	131,844	198.9
Rat D	5	20	127,143	191.8
Rat E	5	20	133,732	202.2

Appendix

Table 4: Statistics for the two reward datasets. Frame counts include all data partitions; durations are computed from session timestamps.

Reward rule	Train	Validation	Test
Alternation Reward	604,223	37,752	113,357
Probabilistic Reward	823,294	51,436	154,462
Total	1,427,517	89,188	267,819

Appendix

Table 5: Frame counts by partition across two datasets. Rounding at session boundaries produces small deviations from the nominal split proportions.

Model	Hyperparameter	Candidate values
TRAIL	Learning rate	$10^{-4}$ , $3\times 10^{-4}$ , $10^{-3}$
	Loss weights	$(1,1)$ , $(0.5,1)$ , $(1,0.5)$ , $(0.25,0.75)$ , $(0.75,0.25)$
Classic methods
LG-SSM	Latent dimension	4, 8, 16, 32, 64, 128
	EM iterations	100, 200
AR-HMM	Number of states	32, 64, 128

Appendix

Table 6: Hyperparameter search. Candidate values are grouped by model. Loss weights are listed as (reconstruction, contrastive); Keypoint-MoSeq iteration counts are listed as (AR, full model). Hidden width and HoA weight are fixed at the values shown.

Setting	Primary	Supplementary
Coordinate projection	Linear	Two-layer MLP
Maze tokens	Waypoints	Vertices and edge descriptors
First-query key normalization	None	Per-head layer normalization
Reconstruction input	Restored representation and seven queries	Full encoder query bank
Reconstruction coefficient	1	0.5
Identity	84.00	84.07

Appendix

Table 7: Encoder configurations used in the additional experiments. Both use multi-frequency coordinate features, 64-dimensional tokens, eight queries, four encoder layers, and contrastive temperature 0.1. The performance rows use the same 105,882 test windows and state-space Proficiency labels. Identity is percentage accuracy; the remaining targets are MSE.

Cohort	Target	Estimate	Confidence interval
Alternation	Identity (%)	84.00	$[76.24,\,91.10]$
Alternation	Day	1.2579	$[0.9569,\,1.5937]$
Alternation	Proficiency	.0156	$[.0109,\,.0223]$
Alternation	Success Rate	0.13	$[0.09,\,0.18]$
Alternation	Reward	0.6090	$[0.4488,\,0.8493]$
Probabilistic	Identity (%)	85.87	$[82.59,\,88.81]$

Appendix

Table 9: TRAIL prediction across reward conditions. Estimates use the held-out predictions from the cohort comparisons; confidence intervals are obtained by bootstrapping animals at the 95% level. Identity is percentage accuracy; the remaining targets are MSE.

Figure 8: Paired comparisons in the primary alternation cohort. Points show pooled MSE reductions relative to each baseline; horizontal bars show animal-bootstrap 95% confidence intervals. All methods use the same 105,882 test windows.

Representation	Accuracy (%)	Confidence interval
BAMS, short context	63.08	$[61.72,\,64.46]$
TRAIL , short context	73.65	$[72.28,\,75.01]$
Accuracy improvement (pp)	10.56	$[9.68,\,11.45]$

Appendix

Table 11: Within-clip identity on MABe22 Mouse Triplets. Estimates average the accuracies of the 1,058 test clips. The 95% confidence intervals use paired clip-bootstrap resampling; mouse slots are defined separately in each clip. The final row reports TRAIL’s accuracy improvement over BAMS in percentage points.

Target	TRAIL	BAMS	Reduction	Confidence interval	Animals
Day	1.765	1.932	$0.167$	$[0.071,\,0.255]$	5 of 6
Proficiency	.02032	.02460	$.00428$	$[.00124,\,.00897]$	5 of 6
Reward	0.760	1.179	$0.419$	$[0.131,\,0.788]$	6 of 6

Appendix

Table 13: Prediction on held-out sessions. MSE is averaged equally across animals. MSE reductions measure the improvement with TRAIL over BAMS, with paired animal-bootstrap 95% confidence intervals. The comparison uses 183,884 windows, except proficiency, which has 181,639 complete cases. BAMS uses local coordinates and maze context. The final column counts animals with lower MSE under TRAIL.

Readout	Day	Proficiency
Animal-specific mean	2.012	.02571
14 kinematic features	1.950	.02541
TRAIL	1.756	.02046

Appendix

Table 14: Day and proficiency readouts on held-out sessions. MSE is averaged equally across six animals. The lower block reports the reduction in MSE achieved by TRAIL relative to each baseline, with paired animal-bootstrap 95% confidence intervals. Day uses 190,814 test windows, and Proficiency uses 188,261 complete cases.

Representation	Balanced acc. (%)	Gain (pp)	Confidence interval
Chance	16.67%	58.21 pp	$[44.08,\,72.33]$
Kinematic features	25.90%	48.98 pp	$[33.91,\,62.35]$
BAMS (local + maze)	38.89%	35.99 pp	$[20.80,\,55.76]$
TRAIL	74.87%	–	–

Appendix

Table 15: Identity and rewarded-count prediction on a held-out day. Days 1–3 train the encoders and probes; day 4 selects regularization. Results use 139,875 day-5 windows. TRAIL improves identity accuracy in all six animals; gains are measured against each listed baseline in percentage points. Rewarded-count MSE is averaged equally across animals; five of six animals improve. Intervals are paired animal-bootstrap 95% confidence intervals.

Representation	Aligned	Permuted	Reduction	Confidence interval
BAMS (local + maze)	1.593	1.661	$0.068$	$[0.0031,\,0.1604]$
TRAIL	0.948	0.980	$0.032$	$[0.0029,\,0.0683]$

Appendix

Table 16: Within-recording time alignment. Errors are mean per-animal rewarded-count MSE. The reduction compares correctly aligned features with permuted features; intervals are animal-bootstrap 95% confidence intervals. Each method improves in five of six animals (one-sided permutation

p=0.001

Target	TRAIL	Kinematics	Reduction	Confidence interval
Day	1.621	1.861	$0.241$	$[0.182,\,0.284]$
Proficiency	.00944	.01112	$.00167$	$[.00073,\,.00261]$
Reward	0.860	1.025	$0.165$	$[0.082,\,0.249]$
Success Rate	0.05	0.07	$0.02$	$[0.01,\,0.02]$

Appendix

Table 17: Transfer with calibration on four new animals. MSE is averaged equally across the four animals using 74,768 test windows. MSE reductions measure the improvement with TRAIL over kinematic features, with paired animal-bootstrap 95% confidence intervals.

Table 18: Ablation studies on loss function and multi-frequency modeling . Bold : best based on unrounded values.

Figure 9: Latent representation trade-off. The default representation has 16 dimensions.

View	Identity	Day	Proficiency
Local Pose	77.1	1.6047	.0189
	$[66.2,\,87.4]$	$[1.3303,\,1.9455]$	$[.0124,\,.0277]$
Global Trajectory	72.1	1.4733	.0172
	$[61.4,\,82.3]$	$[1.3063,\,1.6259]$	$[.0128,\,.0231]$
Full Movement	84.0	1.2579	.0156
	$[76.2,\,91.1]$	$[0.9569,\,1.5937]$	$[.0109,\,.0223]$

Appendix

Table 21: Prediction from local pose and global trajectory. Pooled estimates use the same 105,882 test windows as figure 5 . Intervals below the estimates are animal-bootstrap 95% confidence intervals. Identity is percentage accuracy; Day and Proficiency are MSE.

Figure 10: Movement inputs and reconstruction. Points show pooled MSE reductions relative to each ablated model; horizontal bars show animal-bootstrap 95% confidence intervals. All methods use the same 105,882 test windows. The comparisons use the runs in figures 5 and 18(a) .

Group	Rat A	Rat B	Rat C	Rat D	Rat E	Rat F	Proficiency SD
Arm segment	63	97	101	122	96	79	.174
Arm endpoint	320	449	396	376	404	432	.155
Junction	42	40	72	52	77	43	.157
Day 1	82	158	110	114	120	116	.105
Day 2	84	107	114	109	113	108	.165
Day 3	88	97	115	109	116	111	.111

Appendix

Table 23: Composition and target spread in the primary subgroup analysis. Counts include only scored windows. Rat labels follow table 22 . The final column gives the standard deviation of Proficiency across these windows.

Maze region
Group	$n$	Identity	Day	Proficiency
Arm segment	558	81.7	1.59	.0211
		$[72.4,\,90.1]$	$[1.04,\,2.11]$	$[.0109,\,.0310]$
Arm endpoint	2377	83.0	1.20	.0149
		$[74.1,\,91.0]$	$[0.92,\,1.50]$	$[.0113,\,.0203]$
Junction	326	88.7	1.20	.0135

Appendix

Table 24: Prediction across maze regions, recording days, and movement motifs. Values are pooled estimates, with animal-bootstrap 95% confidence intervals directly below. Identity is percentage accuracy; Day and Proficiency are MSE. Day prediction is omitted when day defines the group.

Classifier	Rat A	Rat B	Rat C	Rat D	Rat E	Rat F
kNN	17	50	21	40	81	32
SVM-RBF	6	43	11	44	85	49
Decision tree	12	49	24	37	78	34
Random forest	11	49	21	42	84	35
TabPFN v2.6	11	49	16	44	84	42

Appendix

Table 28: Per-animal recall from mean segment lengths. Values are percentages on the default split in table 27 . Rat labels follow table 22 .

Panel of Figure 11 — Figure 11: t-SNE visualizations of 3-second representations for each baseline. Left: A joint projection colored by rat. Right: Six per-rat projections colored by day. The figure continues on the next page.

Panel of Figure 12 — Figure 12: t-SNE visualizations of 3-second representations (continued). The final panel repeats TRAIL from figure 6 for direct comparison. Colors follow the same rat and day conventions as above.

Figure 13: Cross-day transformation of turning in local pose. Skeleton sequences and body ratio for the same transformation as the left panel of figure 7 .

Figure 14: Cross-individual transformation of turning in body trajectory. Body-keypoint trajectories and body speed for the same transformation as the right panel of figure 7 .

Figure 15: Cross-day transformation of going straight. Left: Body Trajectory with body speed. Right: Local Pose with body ratio.

Figure 16: Cross-individual transformation of going straight. Left: Body Trajectory with body speed. Right: Local Pose with body ratio.

Animal Behavior Analysis

Animal behavior analysis uses computational tools to measure how animals move, interact, communicate, and respond to their environments.

See all 37 papers

IBAD: Interpretable Behavioral Anomaly Detection on Human Mobility Data
Jun 14, 2026Bita Azarijoo, John Krumm, Cyrus ShahabiInterpretable Anomaly Detection Anomaly Detection
University of Southern California Los Angeles, California, USA
Modeling Fly Social Behavior with Agent-Centric Pose Forecasting
Jul 21, 2026Eyrun Eyjolfsdottir, Kristin BransonMulti-Agent Trajectory Prediction Animal Behavior Analysis
HHMI Janelia Research Campus, Ashburn, VA 20147
REFINE: Trajectory Representation Learning via Closed-Loop Transcription -- Extended Version
Sep 7, 2026Sean Bin Yang, Ying Sun, Jilin Hu +5Contrastive Learning Trajectory Reconstruction
Aalborg University Aalborg, Denmark · Chongqing Universtiy of Posts and Telecommunications Chongqing, China · East China Normal University Shanghai, China +1

Jun 14, 2026J/K move · Enter open · S save

IBAD: Interpretable Behavioral Anomaly Detection on Human Mobility Data

Bita Azarijoo, John Krumm, Cyrus Shahabi

University of Southern California Los Angeles, California, USA

Interpretable Anomaly Detection Anomaly Detection

Open paper AskPDF arXiv

Human mobility appears highly diverse, yet much of a person's daily mobility can be explained by a small set of recurring behavioral templates, such as commuting, school-centered activities, caregiving, nightlife, or errand patterns. We present IBAD (Interpretable Behavioral Anomaly Detection), a framework that learns interpretable daily mobility templates and represents each individual as a distribution over mixtures of these templates. Rather than focusing on specific locations, IBAD characterizes activities that individuals perform across locations. This approach first discovers global behavioral templates using Latent Dirichlet Allocation (LDA), then employs a hierarchical self-supervised model to learn normal behavior of individuals from their soft behavioral templates. We also introduce a splicing benchmark that creates controlled behavioral mismatches between an individual's historical profile and injected mobility patterns. Experiments on real-world and synthetic datasets show that daily behavior can be effectively decomposed into a small number of interpretable templates. Crucially, we show that the learned behavioral archetypes transfer across distinct geographic and demographic contexts. Furthermore, IBAD maintains a robust competitive performance across all settings. For reproducibility purposes, the code is accessible at ~https://github.com/USC-InfoLab/IBAD.

Abstract

Animal Behavior Analysis

Explore similar work

IBAD: Interpretable Behavioral Anomaly Detection on Human Mobility Data

Modeling Fly Social Behavior with Agent-Centric Pose Forecasting

REFINE: Trajectory Representation Learning via Closed-Loop Transcription -- Extended Version