Point Tracking

Follow this topic Ask about this topic

Point tracking estimates where points move across video frames, including when they are occluded, leave the view, or reappear. Methods range from optical-flow and feature-matching systems to recurrent transformers, event-camera trackers, and models that combine image trajectories with depth or 3D geometry. The task supports applications such as robotics, medical imaging, video editing, and motion understanding, and is evaluated on benchmarks including TAP-Vid and 3D multi-view tracking sets. Current work seeks reliable long-term tracks with less annotation and computation, while accurate geometry and robustness to large motion and occlusion remain difficult.

Latest papers

36 papers2 papers in the last four weeks, down 75% on the four weeks before. 0.0% of all new papers.

M2P: Improving Visual Foundation Models with Mask-to-Point Weakly-Supervised Learning for Dense Point Tracking
Mar 18, 2026Qiangqiang Wu, Tianyu Yang, Bo Fang +4Point Tracking Vision Foundation Model Adaptation
Department of Computer Science, City University of Hong Kong, China · Department of Electrical and Computer Engineering, Princeton University, Princeton, NJ 08544 USA · Meituan, Shenzhen, China +2
Real-World Point Tracking with Verifier-Guided Pseudo-Labeling
Mar 12, 2026Görkay Aydemir, Fatma Güney, Weidi XiePoint Tracking Pseudo-Labeling
Department of Computer Engineering, Koc¸ University · KUIS AI Center · School of Artificial Intelligence, Shanghai Jiao Tong University
TAPFormer: Robust Arbitrary Point Tracking via Transient Asynchronous Fusion of Frames and Events
Mar 5, 2026Jiaxiong Liu, Zhen Tan, Jinpu Zhang +4Point Tracking Multimodal Sensor Fusion
National University of Defense Technology · Hunan University
Geometry OR Tracker: Universal Geometric Operating Room Tracking
Feb 28, 2026Yihua Shao, Kang Chen, Feng Xue +6Point Tracking Multi-View Geometry
Centre for Artificial Intelligence and Robotics Hong Kong Institute of Science and Innovation Chinese Academy of Sciences · State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS) Institute of Automation Chinese Academy of Sciences (CASIA) · Peking University +4
Probing and Leveraging Video Diffusion Transformer Features for Robust Point Tracking
Dec 23, 2025Soowon Son, Honggyu An, Jisu Nam +7Point Tracking Video Diffusion Models
KAIST AI · Google DeepMind
Track and Caption Any Motion: Open-Vocabulary Spatiotemporal Captioning via Trajectory-Conditioned Generation
Dec 11, 2025Bishoy Galoaa, Sarah OstadabbasPoint Tracking Temporal Video Grounding
Northeastern University

Mar 18, 2026J/K move · Enter open · S save

M2P: Improving Visual Foundation Models with Mask-to-Point Weakly-Supervised Learning for Dense Point Tracking

Qiangqiang Wu, Tianyu Yang, Bo Fang, Jia Wan+3

Department of Computer Science, City University of Hong Kong, China · Department of Electrical and Computer Engineering, Princeton University, Princeton, NJ 08544 USA · Meituan, Shenzhen, China+2

Point Tracking Vision Foundation Model Adaptation Visual Representation Learning Video Representation Learning

Open paper AskPDF arXiv

Tracking Any Point (TAP) has emerged as a fundamental tool for video understanding. Current approaches adapt Vision Foundation Models (VFMs) like DINOv2 via offline finetuning or test-time optimization. However, these VFMs rely on static image pre-training, which is inherently sub-optimal for capturing dense temporal correspondence in videos. To address this, we propose Mask-to-Point (M2P) learning, which leverages rich video object segmentation (VOS) mask annotations to improve VFMs for dense point tracking. Our M2P introduces three new mask-based constraints for weakly-supervised representation learning. First, we propose a local structure consistency loss, which leverages Procrustes analysis to model the cohesive motion of points lying within a local structure, achieving more reliable point-to-point matching learning. Second, we propose a mask label consistency (MLC) loss, which enforces that sampled foreground points strictly match foreground regions across frames. The proposed MLC loss can be regarded as a regularization, which stabilizes training and prevents convergence to trivial solutions. Finally, mask boundary constrain is applied to explicitly supervise boundary points. We show that our weaklysupervised M2P models significantly outperform baseline VFMs with efficient training by using only 3.6K VOS training videos. Notably, M2P achieves 12.8% and 14.6% performance gains over DINOv2-B/14 and DINOv3-B/16 on the TAP-Vid-DAVIS benchmark, respectively. Moreover, the proposed M2P models are used as pre-trained backbones for both test-time optimized and offline fine-tuned TAP tasks, demonstrating its potential to serve as general pre-trained models for point tracking. Code will be made publicly available upon acceptance.

Latest papers

M2P: Improving Visual Foundation Models with Mask-to-Point Weakly-Supervised Learning for Dense Point Tracking

Real-World Point Tracking with Verifier-Guided Pseudo-Labeling

TAPFormer: Robust Arbitrary Point Tracking via Transient Asynchronous Fusion of Frames and Events

Geometry OR Tracker: Universal Geometric Operating Room Tracking

Probing and Leveraging Video Diffusion Transformer Features for Robust Point Tracking

Track and Caption Any Motion: Open-Vocabulary Spatiotemporal Captioning via Trajectory-Conditioned Generation