Vision Foundation Model Adaptation

Follow this topic Ask about this topic

This field studies how to reuse pretrained vision models when target tasks, data, or sensors differ from their original training conditions. Common approaches include full or partial fine-tuning, parameter-efficient methods such as LoRA and prompt or adapter tuning, feature alignment, and distillation into smaller models. Applications range from medical and remote-sensing segmentation to video understanding, robotics, and 3D geometry, often with limited labels or shifted test data. A central tension is that adaptation can improve target-task accuracy while weakening robustness or transfer to unfamiliar settings, so efficiency and out-of-domain performance remain important evaluation criteria.

Latest papers

328 papers33 papers in the last four weeks, up 22% on the four weeks before. 0.2% of all new papers.

Compact Task-Aligned Imitation Learning for Laboratory Automation
Mar 1, 2026Kanata Suzuki, Hanon Nakamura, Kana Miyamoto +1Robot Imitation Learning Laboratory Automation
Spatial Robotics Research Center, Fujitsu Limited., Kanagawa 211-8588, Japan. · Faculty of Science and Engineering, Waseda University, Tokyo 169-8050, Japan. · National Institute of Advanced Industrial Science and Technology, Tokyo 100-8921, Japan.
TAP-SLF: Parameter-Efficient Adaptation of Vision Foundation Models for Multi-Task Ultrasound Image Analysis
Feb 28, 2026Hui Wan, Libin LanVision Foundation Model Adaptation Medical Image Analysis
College of Computer Science and Engineering, Chongqing University of Technology, Chongqing, China
A Mixed Diet Makes DINO An Omnivorous Vision Encoder
Feb 27, 2026Rishabh Kabra, Maks Ovsjanikov, Drew A. Hudson +5Cross-Modal Representation Learning Cross-Modal Alignment
Google DeepMind · University College London
VesselBridge3D: A Foundation Model Adaptation Framework for Label-Efficient 3D Vessel Segmentation
Feb 27, 2026Kirato Yoshihara, Yohei Sugawara, Yuta Tokuoka +13D Medical Image Segmentation Few-Shot Medical Image Segmentation
The University of Osaka · Preferred Networks, Inc.
pMoE: Prompting Diverse Experts Together Wins More in Visual Adaptation
Feb 26, 2026Shentong Mo, Xufang Luo, Dongsheng LiVisual Prompt Learning Vision Foundation Model Adaptation
Carnegie Mellon University · Microsoft Research
SegMoTE: Token-Level Mixture of Experts for Medical Image Segmentation
Feb 22, 2026Yujie Lu, Jingwen Li, Sibo Ju +5Medical Image Segmentation Vision Foundation Model Adaptation
Sichuan University · Xinjiang University · Fuzhou University +1
Frame2Freq: Spectral Adapters for Fine-Grained Video Understanding
Feb 21, 2026Thinesh Thiyakesan Ponbagavathi, Constantin Seibold, Alina RoitbergVideo Action Recognition Vision Foundation Model Adaptation
Institute for Artificial Intelligence, University of Stuttgart, Germany · Intelligent Assistive Systems Lab, University of Hildesheim · University Hospital Heidelberg, Diagnostic and Interventional Radiology
EasyControlEdge: A Foundation-Model Fine-Tuning for Edge Detection
Feb 18, 2026Hiroki Nakamura, Hiroto Iino, Masashi Okada +1Edge Detection Vision Foundation Model Adaptation
Panasonic Holdings Corporation · Kyoto University
Spanning the Visual Analogy Space with a Weight Basis of LoRAs
Feb 17, 2026Hila Manor, Rinon Gal, Haggai Maron +2Vision Foundation Model Adaptation Low-Rank Adaptation
2NVIDIA, Israel · 1Technion, Israel · 3Bar-Ilan University, Israel
Depth Completion as Parameter-Efficient Test-Time Adaptation
Feb 16, 2026Bingxin Ke, Qunjie Zhou, Jiahui Huang +5Depth Completion Test-Time Adaptation
NVIDIA · ETH Zürich
STORM: Slot-based Task-aware Object-centric Representation for robotic Manipulation
Jan 28, 2026Alexandre Chapin, Emmanuel Dellandréa, Liming ChenVision Foundation Model Adaptation Generalization in Robotic Manipulation
LIRIS · Ecole Centrale de Lyon, LIRIS, 69130, Ecully, France
Attentive multilayer fusion for vision transformers
Jan 14, 2026Laure Ciernik, Marco Morik, Lukas Thede +4Linear Probing Vision Transformer
Machine Learning Group, Technische Universität Berlin, Berlin, Germany · Hector Fellow Academy, Karlsruhe, Germany · Berlin Institute for the Foundations of Learning and Data (BIFOLD), Berlin, Germany +7
Depth Anything in $360^\circ$ : Towards Scale Invariance in the Wild
Dec 28, 2025Hualie Jiang, Ziyang Song, Zhiqiang Lou +2Depth Estimation Monocular Depth Estimation
Insta360 Research
Probing and Leveraging Video Diffusion Transformer Features for Robust Point Tracking
Dec 23, 2025Soowon Son, Honggyu An, Jisu Nam +7Point Tracking Video Diffusion Models
KAIST AI · Google DeepMind
Multi-View Foundation Models
Dec 17, 2025Leo Segre, Or Hirschorn, Shai AvidanMulti-View Consistency Vision Foundation Model Adaptation
Tel Aviv University
ColonAdapter: Geometry Estimation Through Foundation Model Adaptation for Colonoscopy
Nov 27, 2025Zhiyi Jiang, Yifu Wang, Xuelian Cheng +1Endoscopic 3D Reconstruction Vision Foundation Model Adaptation
School of Computer Science and Engineering, Southeast University, China · Vetex Lab, Shanghai, China · Southeast University - Monash University Joint Graduate School, Suzhou 215123, China +3
D $^{2}$ -VPR: A Parameter-efficient Visual-foundation-model-based Visual Place Recognition Method via Knowledge Distillation and Deformable Aggregation
Nov 16, 2025Zheyuan Zhang, Jiwei Zhang, Boyu Zhou +2Visual Place Recognition Vision Foundation Model Adaptation
School of Computer Science, Beijing University of Posts and Telecommunications · Key Laboratory of Interactive Technology and Experience System, Ministry of Culture and Tourism, Beijing University of Posts and Telecommunications
AD-SAM: Adapting the Segment Anything Model for Semantic Segmentation in Autonomous Driving
Oct 30, 2025Het Patel, Mario Camarena, Fatemeh Nazari +3Vision Foundation Model Adaptation Autonomous Driving Perception
Department of Computer Science, The University of Texas Rio Grande Valley, Edinburg, TX 78539 USA · Department of Civil Engineering, The University of Texas Rio Grande Valley, Edinburg, TX 78539 USA · Department of Computer Science and Engineering, University of California Riverside, Riverside, CA 92521 USA
Evaluating Fundus-Specific Foundation Models for Diabetic Macular Edema Detection
Oct 8, 2025Franco Javier Arellano, José Ignacio OrlandoMedical Imaging Foundation Models Vision Foundation Model Adaptation
Yatiris Group, PLADEMA Institute, UNICEN, Tandil, Argentina · CONICET, Tandil, Argentina
Robust and Efficient Finetuning of Vision Foundation Models via Implicit Ensembling
Oct 8, 2025Masih Aminbeidokhti, Heitor Rapela Medeiros, Eric Granger +1Vision Foundation Model Adaptation OOD Generalization
Phantom-Insight: Adaptive Multi-cue Fusion for Video Camouflaged Object Detection with Multimodal LLM
Sep 8, 2025Hua Zhang, Changjiang LuoVideo Object Segmentation Vision Foundation Model Adaptation
Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences
EndoUFM: Utilizing Foundation Models for Monocular depth estimation of endoscopic images
Aug 25, 2025Xinning Yao, Bo Liu, Bojian Li +3Medical Imaging Monocular Depth Estimation
Image Processing Center, Beihang University, Beijing, 100191, China. · State Key Laboratory of High-Efficiency Reusable Aerospace Transportation Technology, Beijing 102206, China.
Extending Foundational Monocular Depth Estimators to Fisheye Cameras with Calibration Tokens
Aug 6, 2025Rit Gangopadhyay, Jung-Hee Kim, Xien Chen +3Domain Adaptation Depth Estimation
Yale University1 · Michigan State University2
WaRA: Wavelet Low-Rank Adaptation for Medical Image Classification
Jun 25, 2025Moein Heidari, Yijin Huang, Yasamin Medghalchi +3Vision Foundation Model Adaptation Low-Rank Adaptation
School of Biomedical Engineering, The University of British Columbia, Vancouver, BC, Canada · Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China · Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, BC, Canada +2
Unlocking Pretrained Vision Transformers for Time Series Classification
Jun 10, 2025Simon Roschmann, Quentin Bouniot, Vasilii Feofanov +2Time Series Classification Vision Transformer
Helmholtz Munich · Technical University of Munich · Munich Center for Machine Learning +2
SARTM: Segment Any RGB Thermal Model with Language aided Distillation
May 4, 2025Dong Xing, Jinhe Zhang, Hang Yang +1Vision Foundation Model Adaptation Semantic Segmentation
Changchun Institute of Optics, Fine Mechanicsand Physics, Chinese Academy of Sciences University
C-LoRA: Continual Low-Rank Adaptation for Pre-trained Visual Models
Feb 25, 2025Xin Zhang, Liang Bai, Xian YangContinual LoRA Low-Rank Adaptation
Institute of Intelligent Information Processing, Shanxi University, Taiyuan, 030006, China · Alliance Manchester Business School, The University of Manchester, Manchester, M13 9PL, UK
Large Pretraining Datasets Don't Guarantee Robustness after Fine-Tuning in Image Classification
Oct 28, 2024Jaedong Hwang, Brian Cheung, Zhang-Wei Hong +3Neural Network Robustness OOD Generalization
Massachusetts Institute of Technology

Mar 1, 2026J/K move · Enter open · S save

Compact Task-Aligned Imitation Learning for Laboratory Automation

Kanata Suzuki, Hanon Nakamura, Kana Miyamoto, Tetsuya Ogata

Spatial Robotics Research Center, Fujitsu Limited., Kanagawa 211-8588, Japan. · Faculty of Science and Engineering, Waseda University, Tokyo 169-8050, Japan. · National Institute of Advanced Industrial Science and Technology, Tokyo 100-8921, Japan.

Robot Imitation Learning Laboratory Automation Efficient VLA Models Vision Foundation Model Adaptation Diffusion Policy

Open paper AskPDF arXiv

Robotic laboratory automation has traditionally relied on carefully engineered motion pipelines and task-specific hardware interfaces, resulting in high design cost and limited flexibility. While recent imitation learning techniques can generate general robot behaviors, their large model sizes often require high-performance computational resources, limiting applicability in practical laboratory environments. In this study, we propose a compact imitation learning framework for laboratory automation using small foundation models. The proposed method, TVF-DiT, aligns a self-supervised vision foundation model with a vision-language model through a compact adapter, and integrates them with a Diffusion Transformer-based action expert. The entire model consists of fewer than 500M parameters, enabling inference on low-VRAM GPUs. Experiments on three real-world laboratory tasks - test tube cleaning, test tube arrangement, and powder transfer - demonstrate an average success rate of 86.6%, significantly outperforming alternative lightweight baselines. Furthermore, detailed task prompts improve vision-language alignment and task performance. These results indicate that small foundation models, when properly aligned and integrated with diffusion-based policy learning, can effectively support practical laboratory automation with limited computational resources.