Multimodal Foundation Models

Follow this topic Ask about this topic

Momentum

13 papers in the last four weeks, up 117% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 303

All topics

MedGPT-oss: Training a General-Purpose Vision-Language Model for Biomedicine
Mar 1, 2026Kai Zhang, Zhengqing Yuan, Cheng Peng +10Healthcare Multimodal Pretraining
Department of Computer Science and Engineering, Lehigh University · Department of Computer Science and Engineering, University of Notre Dame · Department of Health Outcomes & Biomedical Informatics, University of Florida +3
MMLoP: Multi-Modal Low-Rank Prompting for Efficient Vision-Language Adaptation
Feb 24, 2026Sajjad Ghiasvand, Haniyeh Ehsani Oskouie, Mahnoosh Alizadeh +1Cross-Modal Learning Prompt Learning
University of California, Santa Barbara · University of California, Los Angeles
UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models
Feb 9, 2026Cheng Yang, Chufan Shi, Bo Shui +7Unified Multimodal Models Image Generation
University of California San Diego · University of Southern California · University of Illinois Urbana-Champaign +1
HeartMuLa: A Family of Open Sourced Music Foundation Models
Jan 15, 2026Dongchao Yang, Yuxin Xie, Yuguo Yin +26Text-to-Music Generation Neural Audio Codecs
CASHEW: Stabilizing Multimodal Reasoning via Iterative Trajectory Aggregation
Jan 12, 2026Chaoyu Li, Fei Tao, Pooyan FazliTest-Time Scaling for VLMs Test-Time Scaling
Arizona State University · NewsBreak
CLIMP: Contrastive Language-Image Mamba Pretraining
Jan 11, 2026Nimrod Shabtay, Itamar Zimerman, Eli Schwartz +1Cross-Modal Learning VLM Robustness
Tel-Aviv University · IBM Research
BabyVision: Visual Reasoning Beyond Language
Jan 10, 2026Liang Chen, Weichu Xie, Yiyan Liang +27VLM Evaluation Visual Reasoning
1UniPat AI · 6Peking University · 3Alibaba Group +8
Parameter Efficient Multimodal Instruction Tuning for Romanian Vision Language Models
Dec 16, 2025George-Andrei Dima, Răzvan-Alexandru Smădu, Dumitru-Clementin CercelVisual Question Answering Image Captioning
National University of Science and Technology POLITEHNICA Bucharest, Bucharest, Romania
OpenPhone: Mobile Agentic Foundation Models
Oct 24, 2025Yangqin Jiang, Chao HuangOn-Device Language Model Inference GUI Agents
The University of Hong Kong
From Charts to Code: A Hierarchical Benchmark for Multimodal Models
Oct 20, 2025Jiahao Tang, Henry Hengyuan Zhao, Lijian Wu +8Data Visualization Code Generation
CSU-JPG, Central South University · National University of Singapore · Nanyang Technological University
SRUM: Fine-Grained Self-Rewarding for Unified Multimodal Models
Oct 14, 2025Weiyang Jin, Yuwei Niu, Jiaqi Liao +4Unified Multimodal Models T2I Generation
HKU MMLab · The University of Hong Kong · Shenzhen Loop Area Institute +2
HunyuanImage 3.0 Technical Report
Sep 28, 2025Tencent Hunyuan Foundation Model TeamUnified Multimodal Models Autoregressive Image Generation
Tencent Hunyuan Foundation Model Team
LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models
Sep 28, 2025Shubhang Bhatnagar, Andy Xu, Kar-Han Tan +1Mixed-Precision Quantization LLM Quantization
University of Illinois Urbana-Champaign · Work done as an intern at HP Inc. · University of California, Los Angeles +1
Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation
Sep 23, 2025Shufan Li, Jiuxiang Gu, Kangning Liu +4Masked Diffusion Models Unified Multimodal Models
Adobe · UCLA
Reconstruction Alignment Improves Unified Multimodal Models
Sep 8, 2025Ji Xie, Trevor Darrell, Luke Zettlemoyer +1Unified Multimodal Models Cross-Modal Alignment
1UC Berkeley · University of Washington · 3Duke University
VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark
Aug 19, 2025Vy Tuong Dang, An Vo, Emilio Villa-Cueva +4VLM Evaluation Multimodal Reasoning
KAIST · MBZUAI
Inverse-LLaVA: Rethinking Multimodal Alignment via Text-to-Vision Mapping
Aug 17, 2025Xuhui Zhan, Tyler DerrCross-Modal Learning Cross-Modal Representation Learning
Data Science Institute, Vanderbilt University, 2201 West End Ave, Nashville, 37235, Tennessee, United States. · Computer Science Department, Vanderbilt University, 2201 West End Ave, Nashville, 37235, Tennessee, United States.
Position: Reasoning After Perception Means Reasoning Without Vision
Jul 21, 2025Hongcheng Gao, Zihao Huang, Jingyi Tang +12Visual Reasoning VLM Reasoning
Tsinghua University · BIT · Peking University +6
Can Argus Judge Them All? Comparing VLMs Across Domains
Jun 23, 2025Harsh Joshi, Gautam Siddharth Kashyap, Rafiq Ali +5VLM Evaluation VLM Robustness
Bharati Vidyapeeth, New Delhi, India · Macquarie University, Sydney, Australia · DSEU-Okhla, New Delhi, India +3
Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning
Jun 8, 2025Weiwen Xu, Hou Pong Chan, Long Li +15Medical Report Generation Multimodal Pretraining
LASA Team · DAMO Academy, Alibaba Group
Dimple: Discrete Diffusion Multimodal Large Language Model with Parallel Decoding
May 22, 2025Runpeng Yu, Xinyin Ma, Xinchao WangMultimodal Large Language Models LLM Inference Acceleration
National University of Singapore
LaViDa: A Large Diffusion Language Model for Multimodal Understanding
May 22, 2025Shufan Li, Konstantinos Kallidromitis, Hritik Bansal +7Vision-Language Models Efficient VLM Inference
1UCLA · 2Panasonic AI Research · 4Salesforce Research +1
HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluation
May 16, 2025Shaina Raza, Aravind Narayanan, Vahid Reza Khazaie +6Algorithmic Fairness Multimodal Large Language Models
Vector Institute for Artificial Intelligence, Toronto, Canada · University of Central Florida, Orlando, USA
ChronoSteer: Bridging Large Language Model and Time Series Foundation Model via Synthetic Cross-Modal Alignment Dataset
May 15, 2025Chengsen Wang, Qi Qi, Zhongwen Rao +2Cross-Modal Learning Multivariate Time Series Forecasting
Beijing University of Posts and Telecommunications · Huawei Noah’s Ark Lab
ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models
Feb 13, 2025Jonathan Roberts, Mohammad Reza Taesiri, Ansh Sharma +31Visual Reasoning VLM Reasoning
University of Cambridge
Histopathology Multi-modal Embedding for Pathology Composed Retrieval
Feb 11, 2025Qifeng Zhou, Wenliang Zhong, Thao M. Dang +4Computational Pathology Healthcare
The University of Texas at Arlington, Arlington TX 76010, USA
MultiViewDx: Evidence-Linked Multi-View Clinical Diagnosis
Oct 19, 2024Junda Wang, Zonghai Yao, Yujan Ting +5Healthcare Medical Image Analysis
United Imaging Intelligence, MA, USA · Manning College of Information and Computer Sciences, University of Massachusetts Amherst, MA, USA · Department of Medicine, University of Massachusetts Medical School, Worcester, MA, USA +1
VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models
Jul 16, 2024Haodong Duan, Xinyu Fang, Junming Yang +41VLM Evaluation LLM Evaluation
ABACUS: Adapting Unified Foundation Model for Bridging Image Count Understanding and Generation
Date pendingAnindya Mondal, Sauradip Nag, Anjan DuttaVLM Adaptation Crowd Counting
University of Surrey, United Kingdom & Simon Fraser University, Canada · University of Surrey, United Kingdom
Retrieved Images as Visual Thought: Training-Free Multimodal In-Context Learning for the Open-vs-Closed Gap
Date pendingBingchen Huang, Zhiling Wang, Yifu Chen +1Multimodal RAG Multimodal ICL
Meituan

Mar 1, 2026, cs.CLJ/K move · Enter open · S save

MedGPT-oss: Training a General-Purpose Vision-Language Model for Biomedicine

Kai Zhang, Zhengqing Yuan, Cheng Peng, Songlin Zhao+9

Department of Computer Science and Engineering, Lehigh University · Department of Computer Science and Engineering, University of Notre Dame · Department of Health Outcomes & Biomedical Informatics, University of Florida+3

Healthcare Multimodal Pretraining Medical VLMs Clinical Reasoning Multimodal Foundation Models

Open paper AskPDF arXiv

Biomedical multimodal assistants have the potential to unify radiology, pathology, and clinical-text reasoning, yet a critical deployment gap remains: top-performing systems are either closed-source or computationally prohibitive, precluding the on-premises deployment required for patient privacy and PHI compliance. We introduce MEDGPT-OSS, an open-weight, 20B-parameter generalist vision-language model designed to facilitate open research in clinical AI. Rather than relying on architectural complexity, MEDGPT-OSS pairs the GPT-oss language backbone with a visual front-end via a optimized, three-stage training curriculum. By progressively domain-adapting these modules through rigorous data curation and long-context multimodal alignment, we demonstrate that a 20B model can bridge the capacity gap. It successfully outperforms larger open medical models on out-of-distribution (OOD) multimodal reasoning and complex text-only clinical tasks. By unifying diverse modalities under a single instruction-following interface, MEDGPT-OSS maintains a parameter-efficient footprint fully compatible with commodity GPUs. We release the complete training recipe, open-weight checkpoints, and a rigorous evaluation harness to serve as a verifiable foundation for privacy-preserving, institution-specific clinical AI research.

Momentum

Related topics

Latest papers 303