Large Multimodal Models

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 133% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 75

All topics

Latent Denoising Improves Visual Alignment in Large Multimodal Models
Apr 23, 2026Dhruv Parikh, Jacob Fein-Ashley, Rajgopal Kannan +1Large Multimodal Models Visual Representations
University of Southern California Los Angeles, CA, USA · DEVCOM ARL Army Research Office USA
Unlocking Multi-Spectral Data for Multi-Modal Models with Guided Inputs and Chain-of-Thought Reasoning
Apr 22, 2026Dahun Kim, Ganesh Satish Mallya, Anelia AngelovaHyperspectral Image Remote Sensing
Google DeepMind
Revisiting Change VQA in Remote Sensing with Structured and Native Multimodal Qwen Models
Apr 20, 2026Yakoub Bazi, Mohamad M. Al Rahhal, Mansour Zuair +1Remote Sensing Large Multimodal Models
Computer Engineering Department, College of Computer and Information Sciences, King Saud University, Riyadh 11543, Saudi Arabia · Applied Computer Science Department, College of Applied Computer Science, King Saud University, Riyadh 11543, Saudi Arabia
UniCon: Unified Framework for Efficient Contrastive Alignment via Kernels
Apr 17, 2026Hangke Sui, Yuqing Wang, Minh N DoContrastive Alignment Large Multimodal Models
Department of Electrical & Computer Engineering, The Grainger College of Engineering, UIUC · Coordinated Science Laboratory, University of Illinois Urbana-Champaign (UIUC) · Siebel School of Computing and Data Science, The Grainger College of Engineering, UIUC +1
Find, Fix, Reason: Context Repair for Video Reasoning
Apr 17, 2026Haojian Huang, Chuanyu Qin, Yinchuan Li +1Reasoning Skills Large Multimodal Models
The Hong Kong University of Science and Technology (Guangzhou) · Knowin AI · Institute of Information Engineering, Chinese Academy of Sciences
Robust Multispectral Semantic Segmentation under Missing or Full Modalities via Structured Latent Projection
Apr 17, 2026Irem Ulku, Erdem Akagündüz, Ömer Özgür TanrıöverAmodal Segmentation Complementary Modalities
Department of Computer Engineering, Ankara University, Ankara, Türkiye · Department of Modeling and Simulation, Graduate School of Informatics, METU, Ankara, Türkiye
GeoHeight-Bench: Towards Height-Aware Multimodal Reasoning in Remote Sensing
Mar 26, 2026Xuran Hu, Zhitong Xiong, Zhongcheng Hong +3Geospatial Reasoning Remote Sensing
KTH Royal Institute of Technology · Technical University of Munich · The Hong Kong University of Science and Technology (Guangzhou)
HART: High-Resolution Annotation-Free Reasoning Technique through a Closed-loop Framework
Feb 27, 2026Jiacheng Yang, Anqi Chen, Yunkai Dang +5Large Multimodal Models Recent Vision-Language Models
State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China · Institute of Brain-inspired Intelligence, Nanjing University, Nanjing, China · Shenzhen Research Institute of Nanjing University, Shenzhen, China
Visual Species Recognition with Large Multimodal Models as Post-Hoc Correctors
Dec 10, 2025Tian Liu, Anwesha Basu, James Caverlee +1Species Identification Large Multimodal Models
Department of Computer Science and Engineering, Texas A&M University · Department of AI, University of Macau
From Charts to Code: A Hierarchical Benchmark for Multimodal Models
Oct 20, 2025Jiahao Tang, Henry Hengyuan Zhao, Lijian Wu +8Image-To-Code Generation Chart
CSU-JPG, Central South University · National University of Singapore · Nanyang Technological University
LM Fight Arena: Benchmarking Large Multimodal Models via Game Competition
Oct 10, 2025Yushuo Zheng, Tongrui Ye, Zicheng Zhang +3Chatbot Arena Large Multimodal Models
Shanghai Jiao Tong University, Shanghai, China · Shanghai AI Lab, Shanghai, China · Tongji University, Shanghai, China
Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning
Jun 8, 2025Weiwen Xu, Hou Pong Chan, Long Li +15Multimodal Clinical Data Multimodal Large Language Models
LASA Team · DAMO Academy, Alibaba Group
Q-SiT: Teaching LMMs for Image Quality Scoring and Interpreting
Mar 12, 2025Zicheng Zhang, Haoning Wu, Ziheng Jia +2Image Quality Assessment Large Multimodal Models
Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University, China · S-Lab, Nanyang Technological University, Singapore · School of Computer Science and Engineering, Nanyang Technological University, Singapore
ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models
Feb 13, 2025Jonathan Roberts, Mohammad Reza Taesiri, Ansh Sharma +31Large Multimodal Models
University of Cambridge
VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models
Jul 16, 2024Haodong Duan, Xinyu Fang, Junming Yang +41Large Multimodal Models Multimodal Benchmarks

Apr 23, 2026, cs.CVJ/K move · Enter open · S save

Latent Denoising Improves Visual Alignment in Large Multimodal Models

Dhruv Parikh, Jacob Fein-Ashley, Rajgopal Kannan, Viktor Prasanna

University of Southern California Los Angeles, CA, USA · DEVCOM ARL Army Research Office USA

Large Multimodal Models Visual Representations Iterative Denoising Methods

Open paper AskPDF arXiv

Large Multimodal Models (LMMs) such as LLaVA are typically trained with an autoregressive language modeling objective, providing only indirect supervision to visual tokens. This often yields weak internal visual representations and brittle behavior under distribution shift. Inspired by recent progress on latent denoising for learning high-quality visual tokenizers, we show that the same principle provides an effective form of visual supervision for improving internal visual feature alignment and multimodal understanding in LMMs. We propose a latent denoising framework that corrupts projected visual tokens using a saliency-aware mixture of masking and Gaussian noising. The LMM is trained to denoise these corrupted tokens by recovering clean teacher patch features from hidden states at a selected intermediate LLM layer using a decoder. To prevent representation collapse, our framework also preserves the teacher's intra-image similarity structure and applies intra-image contrastive patch distillation. During inference, corruption and auxiliary heads are disabled, introducing no additional inference-time overhead. Across a broad suite of standard multimodal benchmarks, our method consistently improves visual understanding and reasoning over strong baselines, and yields clear gains on compositional robustness benchmarks (e.g., NaturalBench). Moreover, under ImageNet-C-style non-adversarial common corruptions applied to benchmark images, our method maintains higher accuracy and exhibits reduced degradation at both moderate and severe corruption levels. Our code is available at https://github.com/dhruvashp/latent-denoising-for-lmms.

Momentum

Related topics

Latest papers 75