Vision Foundation Model Adaptation

Ask about this topic

Overview

This field studies how to reuse pretrained vision models when target tasks, data, or sensors differ from their original training conditions. Common approaches include full or partial fine-tuning, parameter-efficient methods such as LoRA and prompt or adapter tuning, feature alignment, and distillation into smaller models. Applications range from medical and remote-sensing segmentation to video understanding, robotics, and 3D geometry, often with limited labels or shifted test data. A central tension is that adaptation can improve target-task accuracy while weakening robustness or transfer to unfamiliar settings, so efficiency and out-of-domain performance remain important evaluation criteria.

Latest papers

CardsList

328 papers33 papers in the last four weeks, up 22% on the four weeks before. 0.2% of all new papers.

  1. M2P: Improving Visual Foundation Models with Mask-to-Point Weakly-Supervised Learning for Dense Point Tracking

    Mar 18, 2026Qiangqiang Wu, Tianyu Yang, Bo Fang +4Point TrackingVision Foundation Model Adaptation

    Department of Computer Science, City University of Hong Kong, China · Department of Electrical and Computer Engineering, Princeton University, Princeton, NJ 08544 USA · Meituan, Shenzhen, China +2

  2. Exploring parameter-efficient fine-tuning (PEFT) of billion-parameter vision models with QLoRA and DoRA: insights into generalization for limited-data image classification under a 98:1 test-to-train regime

    Mar 18, 2026Haiyu Yang, Sumit Sharma, Enhong Liu +1Precision Livestock FarmingParameter-Efficient Fine-Tuning

    Cornell University, College of Agriculture and Life Sciences, Ithaca, NY 14853

  3. Parameter-Efficient Modality-Balanced Symmetric Fusion for Multimodal Remote Sensing Semantic Segmentation

    Mar 18, 2026Haocheng Li, Juepeng Zheng, Shuangxi Miao +4Remote Sensing Image SegmentationMultimodal Fusion

    the College of Land Science and Technology, China Agricultural University, and also with the Key Laboratory of Remote Sensing for Agri-Hazards, Ministry of Agriculture and Rural Affairs, Beijing 100083, China · the School of Artificial Intelligence, Sun Yat-Sen University, Zhuhai, China · Henan Polytechnic University, and also with the Key Laboratory of Spatio-Temporal Information and Ecological Restoration of Mines, Ministry of Natural Resources of the People’s Republic of China, Jiaozuo 454003, China +4

  4. Early Intervention for VFM-based Multimodal Medical Image Classification

    Mar 18, 2026Qijie Wei, Hailan Lin, Xirong LiCross-Modal Representation LearningVision Foundation Model Adaptation

    Renmin University of China, Beijing 100872, China

  5. FMMC: Harnessing the Power of Foundation Models for Accurate Material Classification

    Mar 18, 2026Qingran Lin, Fengwei Yang, Chaolun ZhuVision-Language ModelsSynthetic Data Generation

    Georgia Institute of Technology · Duke University · Waseda University

  6. What DINO saw: ALiBi positional encoding reduces positional bias in Vision Transformers

    Mar 17, 2026Moritz Pawlowsky, Antonis Vamvakeros, Alexander Weiss +3Vision TransformerPositional Encoding

    Center for Materials Research, Justus Liebig University, 35392 Giessen · Dyson School of Design Engineering, Imperial College London, London · Finden ltd, Building R71, Rutherford Appleton Laboratory, Harwell Science and Innovation Campus +1

  7. ViT-AdaLA: Adapting Vision Transformers with Linear Attention

    Mar 17, 2026Yifan Li, Seunghyun Yoon, Viet Dac Lai +4Efficient ViTsVision Foundation Model Adaptation

    Michigan State University, East Lansing, MI, USA · Adobe Research

  8. Spectral Rectification for Parameter-Efficient Adaptation of Foundation Models in Colonoscopy Depth Estimation

    Mar 16, 2026Xiaoxian Zhang, Minghai Shi, Lei LiMonocular Depth EstimationVision Foundation Model Adaptation

    Department of Biomedical Engineering, National University of Singapore, Singapore · Department of Radiotherapy, The First Affiliated Hospital of Xi’an Jiaotong University, Xi’an, China

  9. Edit2Interp: Adapting Image Foundation Models from Spatial Editing to Video Frame Interpolation with Few-Shot Learning

    Mar 16, 2026Nasrin Rahimi, Mısra Yavuz, Burak Can Biner +6Video Frame InterpolationVision Foundation Model Adaptation

    Codeway AI Research · Dept. of Electrical & Electronics Engineering, Koç University, İstanbul, Türkiye

  10. EviATTA: Evidential Active Test-Time Adaptation for Medical Segment Anything Models

    Mar 15, 2026Jiayi Chen, Yasmeen George, Winston Chong +1Test-Time AdaptationVision Foundation Model Adaptation

    Department of Data Science & AI, Faculty of Information Technology, Monash University, VIC 3800, Australia · Alfred Health, VIC 3004, Australia

  11. Adaptation of Foundation Models in Medical Image Analysis: Mechanisms, Evidence, and Deployment Considerations

    Mar 15, 2026Karma Phuntsho, Abdullah, Kyungmi Lee +2Medical Imaging Foundation ModelsVision Foundation Model Adaptation

    James Cook University, 14-88 McGregor Rd, Smithfield, 4878, Queensland, Australia

  12. DiveUp: Learning Feature Upsampling from Diverse Vision Foundation Models

    Mar 13, 2026Xiaoqiong Liu, Heng FanVision Foundation Model Adaptation

    Department of Computer Science and Engineering, University of North Texas

  13. VFM-Recon: Unlocking Cross-Domain Scene-Level Neural Reconstruction with Scale-Aligned Foundation Priors

    Mar 13, 2026Yuhang Ming, Tingkang Xi, Xingrui Yang +43D ReconstructionMonocular 3D Reconstruction

    Shanghai Jiao Tong University, Shanghai, China · Hangzhou Dianzi University, Hangzhou, China · CARDC, Mianyang, China

  14. Prompt-Driven Lightweight Foundation Model for Instance Segmentation-Based Fault Detection in Freight Trains

    Mar 13, 2026Guodong Sun, Qihang Liang, Xingyu Pan +2Fault DetectionInstance Segmentation

    the School of Mechanical Engineering, Hubei University of Technology, Wuhan 430068, China · The Hubei Key Laboratory of Modern Manufacturing Quality Engineering, Hubei University of Technology, Wuhan 430068, China · the School of Mechanical Science and Engineering, Huazhong University of Science and Technology, Wuhan 430074, China

  15. ABRA: Teleporting Fine-Tuned Knowledge Across Domains for Open-Vocabulary Object Detection

    Mar 12, 2026Mattia Bernardi, Chiara Cappellino, Matteo Mosconi +3Open-Vocabulary Object DetectionVision Foundation Model Adaptation

    University of Modena and Reggio Emilia

  16. Diabetic Retinopathy Grading with CLIP-based Ranking-Aware Adaptation:A Comparative Study on Fundus Image

    Mar 12, 2026Sungjun ChoDiabetic Retinopathy GradingRetinal Disease Classification

    Hong Kong University of Science and Technology (HKUST), Hong Kong, China

  17. MIL-PF: Multiple Instance Learning on Precomputed Features for Mammography Classification

    Mar 10, 2026Nikola Jovišić, Milica Škipina, Nicola Dall'Asen +1Mammography Image ClassificationMedical Image Classification

    Institute for AI R&D of Serbia, Serbia · Faculty of Technical Sciences, University of Novi Sad, Serbia · University of Trento, Italy +2

  18. VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer

    Mar 9, 2026Yanning Hou, Peiyuan Li, Zirui Liu +4Zero-Shot Anomaly DetectionVision Foundation Model Adaptation

    College of Intelligence Science and Technology, National University of Defense Technology, Changsha, China · School of Artificial Intelligence, Anhui University, Hefei, China · State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, Anhui University, Hefei, China

  19. MINT: Molecularly Informed Training with Spatial Transcriptomics Supervision for Pathology Foundation Models

    Mar 9, 2026Minsoo Lee, Jonghyun Kim, Juseung Yun +2Pathology Foundation ModelsGene Expression Prediction from Histopathology Images

    LG AI Research

  20. Fusion Complexity Is Not Fusion Quality: Backbone, Cross View Fusion and Metadata as Separate Factors in Pasture Biomass Regression

    Mar 8, 2026Mridankan MandalMulti-View Feature FusionAgricultural Image Analysis

    Department of Information Technology, Indian Institute of Information Technology, Allahabad, Prayagraj, India

  21. FedEU: Evidential Uncertainty-Driven Federated Fine-Tuning of Vision Foundation Models for Remote Sensing Image Segmentation

    Mar 8, 2026Xiaokang Zhang, Xuran Xiong, Jianzhong Huang +1Federated Fine-TuningRemote Sensing Image Segmentation

    School of Artificial Intelligence, Wuhan University, Wuhan 430072, China · School of Information Science and Engineering, Wuhan University of Science and Technology, Wuhan 430081, China · School of Computer Science, Wuhan University, Wuhan 430072, China

  22. Local Precise Refinement: A Dual-Gated Mixture-of-Experts for Enhancing Foundation Model Generalization against Spectral Shifts

    Mar 8, 2026Xi Chen, Maojun Zhang, Yu Liu +1Domain Generalization for Semantic SegmentationRemote Sensing Image Segmentation

    National University of Defense Technology

  23. RePer-360: Releasing Perspective Priors for 360∘^\circ Depth Estimation via Self-Modulation

    Mar 6, 2026Cheng Guan, Chunyu Lin, Zhijie Shen +2Domain AdaptationDepth Estimation

    Institute of Information Science, Beijing Jiaotong University, Beijing 100044, China. · Visual Intelligence +X International Cooperation Joint Laboratory of MOE, Beijing 100044, China.

  24. Unify the Views: View-Consistent Prototype Learning for Few-Shot Segmentation

    Mar 6, 2026Hongli Liu, Yu Wang, Shengjie ZhaoFew-Shot Semantic SegmentationPromptable Image Segmentation

    School of Computer Science and Technology, Tongji University

  25. Tell2Adapt: A Unified Framework for Source Free Unsupervised Domain Adaptation via Vision Foundation Model

    Mar 5, 2026Yulong Shi, Shijie Li, Ziyi Li +1Source-Free Domain AdaptationMedical Image Segmentation

    College of Medicine and Biological Information Engineering, Northeastern University · Engineering Research Center of Medical Imaging and Intelligent Analysis, Ministry of Education · Key Laboratory of Medical Image Computing, Ministry of Education, Northeastern University

  26. HE-VPR: Height Estimation Enabled Aerial Visual Place Recognition Against Scale Variance

    Mar 4, 2026Mengfan He, Xingyu Shao, Chunyu Li +4Visual Place RecognitionVision Foundation Model Adaptation

    Department of Precision Instrument, Tsinghua University, Beijing 100084, China · School of Aerospace Engineering, Beijing Institute of Technology, Beijing 100081, China · School of Instrumentation Science and Opto-electronics Engineering, Beijing Information Science and Technology University, Beijing 100192, China +1

  27. Spectral Gaps and Spatial Priors: Studying Hyperspectral Downstream Adaptation Using TerraMind

    Mar 4, 2026Julia Anna Leonardi, Johannes Jakubik, Paolo Fraccaro +1Geospatial Foundation ModelsVision Foundation Model Adaptation

    Department of Civil and Environmental Engineering, Politecnico di Milano, Milan, Italy · IBM Research Europe, Rüschlikon, Switzerland

  28. MoECLIP: Patch-Specialized Experts for Zero-shot Anomaly Detection

    Mar 3, 2026Jun Yeong Park, JunYoung Seo, Minji Kang +1Zero-Shot Anomaly DetectionVision Foundation Model Adaptation

    Yonsei University, Seoul, Korea

  29. Meta-Learning Hyperparameters for Parameter Efficient Fine-Tuning

    Mar 2, 2026Zichen Tian, Yaoyao Liu, Qianru SunParameter-Efficient Fine-TuningVision Foundation Model Adaptation

    Singapore Management University · University of Illinois Urbana-Champaign

  30. LUMOS: Latent Universal Medical Priors for Segmentation

    Mar 1, 2026Zhuonan Liang, Wei Guo, Jie Gan +4Medical Image SegmentationTransfer Learning

    The University of Sydney, Sydney, NSW 2006, Australia. · Biological System & Engineering Division, Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA. · Berkeley Biomedical Data Science Center, Lawrence Berkeley National Laboratory, Berkeley, CA 94720, USA.