Hypergraph-State Collaborative Reasoning for Multi-Object Tracking

Zikai Song; Junqing Yu; Yi-Ping Phoebe Chen; Wei Yang; Xinchao Wang

cs.CVApr 14, 2026

Hypergraph-State Collaborative Reasoning for Multi-Object Tracking

Authors: Zikai Song, Junqing Yu, Yi-Ping Phoebe Chen, Wei Yang, Xinchao Wang

Organizations: Huazhong University of Science and Technology · National University of Singapore · La Trobe Uniersity

Abstract

Motion reasoning serves as the cornerstone of multi-object tracking (MOT), as it enables consistent association of targets across frames. However, existing motion estimation approaches face two major limitations: (1) instability caused by noisy or probabilistic predictions, and (2) vulnerability under occlusion, where trajectories often fragment once visual cues disappear. To overcome these issues, we propose a collaborative reasoning framework that enhances motion estimation through joint inference among multiple correlated objects. By allowing objects with similar motion states to mutually constrain and refine each other, our framework stabilizes noisy trajectories and infers plausible motion continuity even when target is occluded. To realize this concept, we design HyperSSM, an architecture that integrates Hypergraph computation and a State Space Model (SSM) for unified spatial-temporal reasoning. The Hypergraph module captures spatial motion correlations through dynamic hyperedges, while the SSM enforces temporal smoothness via structured state transitions. This synergistic design enables simultaneous optimization of spatial consensus and temporal coherence, resulting in robust and stable motion estimation. Extensive experiments on four mainstream and diverse benchmarks(MOT17, MOT20, DanceTrack, and SportsMOT) covering various motion patterns and scene complexities, demonstrate that our approach achieves state-of-the-art performance across a wide range of tracking scenarios.

Figures & tables

Figure 1: Hypergraph-based collaborative motion estimation . We first construct a motion-aware hypergraph, represented by an incidence matrix, the hyperedge ( e ) selectively associating strongly correlated targets based on motion states ( gathering ), the gathered group is then dispersed back to individual nodes for fine-grained motion refinement ( scattering ), achieving both collaborative inference and precise object-specific estimation.

Figure 2: HyperSSM block and Hyper Convolution. The HyperSSM block integrates hypergraph computation into the SSM. Given the input motion feature $X$ of all objects $N$ within $L$ frames. First, collaborative reasoning is performed via hyper convolution (HConv), involving a vertex-to-edge (V2E) gathering from vertices to hyperedges and followed by a edge-to-vertex (E2V) scattering back to vertices. The block then computes hidden states $h$ from the HConv-processed input $X$ using discretized parameters $\overline{A}$ and $\overline{B}$ . Finally, outputs $Y$ are generated through trainable parameters $C$ and an additional HConv block.

Figure 3: The architecture of the motion estimation model. The motion estimation module comprises multiple cascaded HyperSSM layers. Given the multi-frame multi-object position information $\mathbf{P}_{L_{t}}$ , we encode it into trajectory embeddings to guide each layer. The resulting motion features are processed through a feed-forward network (FFN), generating the position information $\mathbf{P}_{L_{t+1}}$ for the next sliding window at one-frame intervals.

Methods	HOTA	IDF1	MOTA	AssA	DetA
query-based methods:
TransTrack [ 55 ]	48.9	59.4	65.0	45.2	53.3
TransCenter [ 60 ]	54.5	62.2	73.2	49.7	60.1
MOTR [ 66 ]	57.2	68.4	71.9	55.8	/
MeMOT [ 4 ]	56.9	69.0	72.5	55.2	/
MeMOTR [ 17 ]	58.8	71.5	72.8	58.4	59.6

Table 1: Performance comparison on MOT17 test sets, following the private protocol. The * denotes Kalman Filter, others adopt learnable motion estimation.

Methods	HOTA	IDF1	MOTA	AssA	DetA
query-based methods:
TransCenter [ 60 ]	/	58.7	67.7	/	/
MeMOT [ 4 ]	54.1	66.1	63.7	55.0	/
motion-based methods:
Bytetrack [ 68 ] *	61.3	75.2	77.8	59.6	63.4
StrongSORT [ 11 ] *	62.6	77.0	73.8	64.0	61.3

Table 2: Performance comparison on MOT20 test sets, following the private protocol. The * denotes Kalman Filter, others adopt learnable motion estimation.

Methods	HOTA	IDF1	MOTA	AssA	DetA
query-based methods:
TransTrack [ 55 ]	45.5	45.2	88.4	27.5	75.9
MOTR [ 66 ]	54.2	51.5	79.7	40.2	73.5
MeMOTR [ 17 ]	68.5	71.2	89.9	58.4	80.5
MOTRv2 [ 69 ]	73.4	76.0	92.1	64.4	83.7
MOTRv3 [ 63 ]	70.4	72.3	92.9	59.3	83.8

Table 3: Performance comparison on DanceTrack test sets. The * denotes Kalman Filter, others adopt learnable motion estimation.

Methods	HOTA	IDF1	MOTA	AssA	DetA
query-based methods:
TransTrack [ 55 ]	68.9	72.5	92.6	57.5	82.7
MeMOTR [ 17 ]	70.0	71.4	91.5	59.1	83.1
MOTIP [ 16 ]	72.6	77.1	92.4	63.2	83.5
SAMBA [ 42 ]	69.8	71.9	90.3	59.4	82.2
DualMOT [ 30 ]	73.9	78.7	91.5	66.6	82.2

Table 4: Performance comparison on SportsMOT test sets. The * denotes Kalman Filter, others adopt learnable motion estimation.

Motion methods	MOT17			DanceTrack
Motion methods	HOTA	IDF1	MOTA	HOTA	IDF1	MOTA
Kalman	64.5	74.7	79.1	47.0	52.2	87.3
OC-Kalman	64.6	74.9	79.0	52.5	54.3	89.1
SSM	63.4	72.1	78.6	53.7	53.5	88.9
HyperSSM	65.5	75.9	79.7	59.7	60.6	89.5

Table 5: Comparison with other motion models. The detection and association strategies are fixed across all methods.

Panel of Figure 4 — Figure 4: Ablation study on the number of HyperSSM layers(left) and the length of trajectory segments(right).

$\theta$	0.5	0.7	0.8	0.9	0.95	0.98
HOTA	47.7	62.1	65.5	65.3	64.8	64.7
IDF1	47.3	68.8	75.9	75.5	73.4	72.9
MOTA	72.1	76.2	79.7	79.5	79.4	77.8

Table 6: Ablation analysis of parameter

\theta

, it measures motion similarity between two objects to establish hyperedges.

Model	SSM	HyperSSM
YOLOX	X	X	L	M	S
HOTA	71.9	74.1	72.4	68.1	57.3
IDF1	77.0	81.9	78.8	73.7	67.0
Params(M)	104.3	135.3	76.5	41.1	19.0
fps	20.3	17.5	22.7	26.8	28.8

Table 7: Efficiency comparison on various YOLOX models.

Panel of Figure 5 — Figure 5: Visualizations under the two key challenge scenarios: a motion-smoothness–dependent scene (top) and a crowded scene with severe occlusion (bottom).

Multi-Object Tracking

Multi-object tracking (MOT) links detections across video frames to maintain object identities through motion, occlusion, and changing viewpoints.

See all 193 papers

SAMOFT: Robust Multi-Object Tracking via Region and Flow
May 10, 2026Yanchao Wang, Dawei Zhang, Chengzhuan Yang +5Query Re-Identification Multi-Object Tracking
School of Computer Science and Technology, Zhejiang Normal University, Jinhua 321004, China · School of Automation and Intelligent Sensing, the Institute of Image Processing and Pattern Recognition, and the Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai 200240, China · Institute for Sustainable Industries and Liveable Cities, College of Engineering and Science, Victoria University, Melbourne, VIC 8001, Australia +1
Time-series Meets Complex Motion Modeling: Robust and Computational-effective Motion Predictor for Multi-object Tracking
May 1, 2026Nhat-Tan Do, Le-Huy Tu, Nhi Ngoc-Yen Nguyen +2Temporal Convolutional Networks Multi-Object Tracking
Faculty of Information Science and Engineering, University of Information Technology, Ho Chi Minh City, Vietnam. · Vietnam National University, Ho Chi Minh City, Vietnam.
Polycepta: Object-Centric Appearance Estimation for Multi-Object Tracking
Jun 22, 2026Mohamed Nagy, Naoufel Werghi, Jorge Dias +1Visual Object Tracking Object-Centric Representation Learning
Khalifa University, Abu Dhabi, UAE

May 10, 2026J/K move · Enter open · S save

SAMOFT: Robust Multi-Object Tracking via Region and Flow

Yanchao Wang, Dawei Zhang, Chengzhuan Yang, Wei Liu+4

School of Computer Science and Technology, Zhejiang Normal University, Jinhua 321004, China · School of Automation and Intelligent Sensing, the Institute of Image Processing and Pattern Recognition, and the Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai 200240, China · Institute for Sustainable Industries and Liveable Cities, College of Engineering and Science, Victoria University, Melbourne, VIC 8001, Australia+1

Query Re-Identification Multi-Object Tracking

Open paper AskPDF arXiv

Multi-object tracking (MOT) is a fundamental task in computer vision that requires continuously tracking multiple targets while maintaining consistent identities across frames. However, most existing approaches primarily rely on instance-level object features for trajectory association, which often leads to degraded performance under challenging conditions such as object deformation, nonlinear motion, and occlusion. In this work, we propose SAMOFT, a robust tracker that leverages pixel-level cues to improve robustness under complex motion scenarios. Specifically, we introduce a Pixel Motion Matching (PMM) module that integrates the Segment Anything Model (SAM) with dense optical flow to refine Kalman filter-based motion prediction using instantaneous foreground pixel motion. To further enhance robustness under unreliable detections, we design a Centroid Distance Matching (CDM) module that performs flexible mask-based centroid matching for low-confidence or partially occluded observations. Moreover, a Distribution-Based Correction (DBC) module models long-tailed motion patterns in a training-free manner using historical optical flow statistics and dynamically corrects trajectory states online. We also incorporate a Cluster-Aware ReID (CA-ReID) strategy to improve the stability and discriminative power of trajectory appearance features. Extensive experiments on the DanceTrack and MOTChallenge benchmarks demonstrate that SAMOFT consistently improves baseline trackers and achieves competitive performance compared with recent state-of-the-art methods, validating the effectiveness of leveraging pixel-level cues for robust multi-object tracking.

Abstract

Multi-Object Tracking

Explore similar work

SAMOFT: Robust Multi-Object Tracking via Region and Flow

Time-series Meets Complex Motion Modeling: Robust and Computational-effective Motion Predictor for Multi-object Tracking

Polycepta: Object-Centric Appearance Estimation for Multi-Object Tracking