Self-Supervised Visual Representation Learning

Follow this topic Ask about this topic

Momentum

20 papers in the last four weeks, up 100% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 151

All topics

BEAST3D: Animal behavioral analysis and neural encoding from multi-view video via Gaussian splatting
Jun 1, 2026Yanchen Wang, Lenny Aharon, Wangshu Zhu +7Sparse-View 3D Reconstruction 3D Representation Learning
Columbia University · Cold Spring Harbor · Stanford University
VISReg: Variance-Invariance-Sketching Regularization for JEPA training
Jun 1, 2026Haiyu Wu, Randall Balestriero, Morgan LevineRepresentation Collapse Joint-Embedding Predictive Architecture
Altos Labs · Brown University
Multi-modal Video Representation Alignment for Robust Self-supervised Driver Distraction Detection
Jun 1, 2026David J. Lerch, Livien Majer, Zeyun Zhong +3Multimodal Robustness Cross-Modal Alignment
Fraunhofer IOSB, Karlsruhe, Germany · Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany
Paving the Way for Point Cloud Video Representation Learning Using A PDE Model
Jun 1, 2026Zhuoxu Huang, Zhenkun Fan, Jungong Han +1Point Cloud Learning Video Representation Learning
Department of Computer Science, Aberystwyth University, Aberystwyth SY23 3DB, U.K. · Department of Automation, Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing, 100084, China · Department of Electrical Engineering, Surrey University, Surrey GU2 7XH, U.K.
UR-JEPA: Uniform Rectifiability as a Regularizer for Joint-Embedding Predictive Architectures
May 31, 2026Triet M. LeJoint-Embedding Predictive Architecture Geometric Representation Learning
Generate in Reconstruction Space, Match in Semantic Space: Transport Geometry for One-Step Generation
May 30, 2026Hugues Van Assel, Edward De Brouwer, Saeed Saremi +2One-Step Generative Modeling Distribution Matching
Genentech
NTR: Neural Token Reconstruction for Scene Token Bottleneck in End-to-End Driving
May 29, 2026Jiahui Li, Jiawei Sun, Zixiang Ren +7End-to-End Autonomous Driving Self-Supervised Visual Representation Learning
National University of Singapore · Black Sesame Technologies
HQ-JEPA: Hybrid Quantum Joint-Embedding Predictive Architecture for Cross-Modal Remote Sensing Representation Learning
May 29, 2026Md Aminur Hossain, Ayush V. Patel, Sanjay K. Singh +1Representation Learning Cross-Modal Representation Learning
Space Applications Centre, Indian Space Research Organisation, Ahmedabad, India · Centre of Studies in Resources Engineering, Indian Institute of Technology Bombay, Mumbai, India
Unsupervised Semantic Segmentation Facilitates Model Understanding
May 28, 2026Xiaoyan Yu, Lisa Mais, Jannik Franzen +4Transformer Interpretability Vision Transformer
Max-Delbrück-Center (MDC), Berlin, Germany · Helmholtz Imaging · Humboldt-Universität zu Berlin, Berlin, Germany +2
Bayesian Gated Non-Negative Contrastive Learning
May 27, 2026Peng Cui, Jiahao Zhang, Lijie HuContrastive Learning Disentangled Representation Learning
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI).
A self-supervised learning approach to deep filter banks for texture recognition
May 27, 2026Joao B. Florindo, Lucas O. Lyra, Antonio E. FabrisAutoencoders Convolutional Autoencoder
Institute of Mathematics, Statistics and Scientific Computing of the University of Campinas, Rua Sergio Buarque de Holanda, 651, Campinas, 13083-859, Sao Paulo, Brazil · Institute of Mathematics and Statistics of the University of Sao Paulo, Rua do Matao, 1010, Sao Paulo, 05508-090, Sao Paulo, Brazil
Anatomy-Anchored Self-Supervision: Distilling Vision Foundation Models for Invariant Ultrasound Representation
May 25, 2026Chunzheng Zhu, Yijun Wang, Jianxin Lin +5Invariant Representation Learning Medical Image Representation Learning
Hunan University, Changsha, China · Shenzhen Maternity and Child Healthcare Hospital, Shenzhen, China
Learning from Semantic Dictionaries: Discriminative Codebook Contrastive Learning for Unified Visual Representation and Generation
May 24, 2026Imanol G. Estepa, Jesús M Rodríguez-de-Vera, Bhalaji Nagarajan +1Contrastive Learning Image Generation
1Universitat de Barcelona, Barcelona, Spain · 2Barcelona Supercomputing Center (BSC), Barcelona, Spain
The TIME Machine: On The Power of Motion for Efficient Perception
May 21, 2026Mantas Skackauskas, Xinyue Hao, Laura Sevilla-LaraRepresentation Learning Video Representation Learning
School of Informatics University of Edinburgh
Beyond Isotropy in JEPAs: Hamiltonian Geometry and Symplectic Prediction
May 19, 2026Robert Jenkinson AlvarezJoint-Embedding Predictive Architecture Geometric Representation Learning
Harnessing Self-Supervised Features for Art Classification
May 18, 2026Federico Melis, Davide Bilardello, Emanuele Prato +2Vision Foundation Models Self-Supervised Visual Representation Learning
University of Modena and Reggio Emilia, Modena, Italy
Collision-Resistant Single-Pass Method for Unsupervised Fine-Grained Image Hashing
May 18, 2026Anh-Kiet Duong, Petra Gomez-Krämer, Jean-Michel CarozzaFine-Grained Image Retrieval Image Retrieval
L3i Laboratory, La Rochelle University, 17042 La Rochelle Cedex 1, France · LIENSs Laboratory, La Rochelle University, 17042 La Rochelle Cedex 1, France
Self-supervised Hierarchical Visual Reasoning with World Model
May 17, 2026Yuanfei Xu, Lin Liu, Wengang Zhou +2World Model Learning World Models
Department of Electronic Engineering and Information Science, University of Science and Technology of China, Hefei, Anhui, China
HyperVision: A Channel-Adaptive Ground-Based Hyperspectral Vision Pre-trained Backbone
May 17, 2026Guanyiman Fu, Jingtao Li, Zihang Cheng +8Vision Foundation Model Adaptation Hyperspectral Imaging
Griffith University, Australia · Wuhan University, China · Nanjing University of Science and Technology, China +2
LACE: Latent Visual Representation for Cross-Embodiment Learning
May 16, 2026Yoo Sung Jang, Kanchana Ranasinghe, Cristina Mata +3Cross-Embodiment Robot Learning Robot Imitation Learning
Stony Brook University · Salesforce AI Research
Latent Video Prediction for World Modeling: An Evaluation Uncovering Intriguing Favorable Evidence
May 15, 2026Ali J Alrasheed, Aryan Yazdan Parast, Basim Azam +2Video Representation Learning Video Prediction
The University of Melbourne · Monash University
Pretraining Objective Matters in Extreme Low-Data FGVC: A Backbone-Controlled Study
May 15, 2026Alexander Hackett, Srikanth Thudumu, Ginny Fisher +1Contrastive Learning Masked Autoencoders
Santa Clara University · IAAIR
Masked Next-Scale Prediction for Self-supervised Scene Text Recognition
May 14, 2026Zhuohao Chen, Zeng Li, Yifei Zhang +2Scene Text Recognition Self-Supervised Visual Representation Learning
1Nankai University · Institute of Information Engineering, Chinese Academy of Sciences · Department of Automation and BNRist, Tsinghua University
Learning to Perceive "Where": Spatial Pretext Tasks for Robust Self-Supervised Learning
May 11, 2026Yang Shen, Yusen Cai, Weronika Hryniewska-Guzik +2Visual Spatial Reasoning Self-Supervised Visual Representation Learning
Nanyang Technological University, Singapore · Warsaw University of Technology, Poland
CalibFree: Self-Supervised View Feature Separation for Calibration-Free Multi-Camera Multi-Object Tracking
May 10, 2026Ruiqi Xian, Deep Patel, Iain Melvin +3Multi-View Learning Invariant Representation Learning
University of Maryland, College Park, MD, USA · NEC Laboratories America, Princeton, NJ, USA · University of North Carolina, Greenboro, NC, USA
Boosting Self-Supervised Tracking with Contextual Prompts and Noise Learning
May 7, 2026Yaozong Zheng, Qihua Liang, Bineng Zhong +4Representation Learning Self-Supervised Visual Representation Learning
Key Laboratory of Education Blockchain and Intelligent Technology, Ministry of Education Guangxi Normal University, Guilin 541004, China · University Engineering Research Center of Educational Intelligent Technology Guangxi Normal University, Guilin 541004, China · University of Southampton, Southampton, SO17 1BJ, United Kingdom +1
MTL-MAD: Multi-Task Learners are Effective Medical Anomaly Detectors
May 7, 2026Bogdan Alexandru Bercean, Florinel Alin Croitoru, Vlad Hondru +3Medical Image Anomaly Detection Multi-Task Learning
1Rayscape · 3Polytechnic University of Timi¸soara · University of Bucharest +2
TinySSL: Distilled Self-Supervised Pretraining for Sub-Megabyte MCU Models
May 7, 2026Bibin WilsonSelf-Supervised Learning Self-Supervised Pre-Training
Chaotic Contrastive Learning for Robust Texture Classification
May 6, 2026Joao B FlorindoContrastive Learning Synthetic Data Augmentation
Institute of Mathematics, Statistics and Scientific Computing of the University of Campinas, Rua Sergio Buarque de Holanda, 651, Campinas, 13083-859, Sao Paulo, Brazil
Text-Conditional JEPA for Learning Semantically Rich Visual Representations
May 5, 2026Chen Huang, Xianhang Li, Vimal Thilak +2Joint-Embedding Predictive Architecture Predictive Representation Learning
Apple, Cupertino, United States

Jun 1, 2026, q-bio.NCJ/K move · Enter open · S save

BEAST3D: Animal behavioral analysis and neural encoding from multi-view video via Gaussian splatting

Yanchen Wang, Lenny Aharon, Wangshu Zhu, Kyle Daruwalla+6

Columbia University · Cold Spring Harbor · Stanford University

Sparse-View 3D Reconstruction 3D Representation Learning Animal Behavior Analysis Self-Supervised Visual Representation Learning

Open paper AskPDF arXiv

Multi-view video recordings are increasingly used to capture the 3D movements of animals in experimental settings, yet extracting rich 3D representations from these recordings remains challenging. Supervised pose estimation requires extensive manual annotation, while general-purpose 3D reconstruction models trained on generic scene datasets fail on the specialized imagery and sparse-view setting of laboratory experiments. We address these limitations with BEAST3D, a self-supervised pretraining framework that learns 3D visual representations from unlabeled, calibrated multi-view video. BEAST3D uses a vision transformer to predict 3D Gaussian splats that reconstruct held-out views through differentiable rendering, while simultaneously segmenting the animal from the background. BEAST3D reconstructs 3D structure with as few as four views by conditioning directly on known camera parameters--unlike general-purpose models, which must estimate camera geometry from dense overlapping viewpoints that are seldom available in lab settings. Through comprehensive evaluation across four species, we demonstrate that BEAST3D produces rich, viewpoint-invariant features that transfer effectively to three downstream tasks: novel view synthesis, which validates the quality of the learned 3D representations; multi-view pose estimation, which provides the sparse keypoint trajectories widely used in behavioral analysis; and neural encoding, which relates 3D behavioral features to simultaneously recorded neural activity. BEAST3D thus establishes a versatile framework for behavioral analysis that leverages 3D structure in modern multi-view laboratory recordings.

Momentum

Related topics

Latest papers 151