Multimodal Understanding

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, up 233% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 413

All topics

Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models
May 15, 2026Yujun Tong, Dongliang Chang, Zijin Yin +3Unified Multimodal Models Multimodal Reasoning
Beijing University of Posts and Telecommunications · Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance
Attribute-Grounded Selective Reasoning for Artwork Emotion Understanding with Multimodal Large Language Models
May 15, 2026Cheng Zhang, Yuer Liu, Zhiyu Zhou +2Feature Attribution Multimodal Large Language Models
Department of Computer Science and Technology, Jilin University, Changchun, China · Department of Computer Science, National Taiwan University, Taipei, Taiwan
UAM: A Dual-Stream Perspective on Forgetting in VLA Training
May 15, 2026Jianke Zhang, Yuanfei Luo, Yucheng Hu +6Robotic Manipulation Vision-Language-Action Models
Tsinghua University · ByteDance Seed
GOMA: Toward Structure-Driven Multimodal Alignment from a Graph Signal Smoothing Perspective
May 15, 2026Xu Wang, Xunkai Li, Yinlin Zhu +2Cross-Modal Learning Multimodal IR
School of Airspace Science and Engineering, Shandong University, WeiHai, China · Department of Computer Science, Beijing Institute of Technology, Beijing, China · School of Computer Science and Engineering, Sun Yat-sen University, GuangZhou, China
Pelican-Unify 1.0: A Unified Embodied Intelligence Model for Understanding, Reasoning, Imagination and Action
May 14, 2026Yi Zhang, Yinda Chen, Che Liu +26Action-Conditioned Video Generation Unified Multimodal Models
Beijing Innovation Center of Humanoid Robotics (X-Humanoid) WFM System Group
From Scenes to Elements: Multi-Granularity Evidence Retrieval for Verifiable Multimodal RAG
May 14, 2026Guanhua Chen, Chuyue Huang, Yutong Yao +4Multimodal RAG Multimodal Grounding
NLP2CT Lab, Department of Computer and Information Science, University of Macau
SceneParser: Hierarchical Scene Parsing for Visual Semantics Understanding
May 14, 2026Pengxin Xu, Xincheng Lin, Luping Xiao +5Fine-Grained Visual Perception Multimodal Understanding
Harbin Institute of Technology, Shenzhen · Zhongguancun Academy · Huazhong University of Science and Technology +4
MemLens: Benchmarking Multimodal Long-Term Memory in Large Vision-Language Models
May 14, 2026Xiyu Ren, Zhaowei Wang, Yiming Du +11VLM Evaluation Memory-Augmented VLMs
CSE Deparment, HKUST · CUHK · OmniMemory (Shenzhen) Intelligent Technology Co., Ltd. +1
Do We Really Need External Tools to Mitigate Hallucinations? SIRA: Shared-Prefix Internal Reconstruction of Attribution
May 14, 2026Tian Qin, Junzhe Chen, Yuqing Shi +3Efficient VLM Inference Hallucination in Language Models
Tsinghua University · The University of Sydney · Stanford University +1
ViMU: Benchmarking Video Metaphorical Understanding
May 14, 2026Qi Li, Xinchao WangVideo Understanding Video QA
National University of Singapore
Physics-R1: An Audited Olympiad Corpus and Recipe for Visual Physics Reasoning
May 13, 2026Shan YangVisual Reasoning RL for Language Model Reasoning
Independent Researcher
Senses Wide Shut: A Representation-Action Gap in Omnimodal LLMs
May 13, 2026Trung Nguyen Quang, Yiming Gao, Fanyi Pu +3Multimodal Grounding Multimodal Large Language Models
Nanyang Technological University · LMMs-Lab Team · Johns Hopkins University
Multilingual OCR-Aware Fine-Tuning and Prompt-Guided Chain-of-Thought Reasoning for Multimodal Large Language Models
May 13, 2026Qinwu Xu, Yifan Jiang, Haoyu RenMultimodal Grounding CoT Reasoning
Meta AI · UT Austin · Current independent researcher
MMSkills: Towards Multimodal Skills for General Visual Agents
May 13, 2026Kangning Zhang, Shuai Shao, Qingyao Li +8GUI Agents LLM Agent Skill Learning
1Shanghai Jiao Tong University · 2Xiaohongshu Inc. · 3Southeast University
GeoFlowVLM: Geometry-Aware Joint Uncertainty for Frozen Vision-Language Embedding
May 13, 2026Mayank Nautiyal, Li Ju, Andreas Hellander +2Cross-Modal Learning Flow Matching
Department of Information Technology, Uppsala University, Uppsala, Sweden · SciLifeLab, Uppsala University, Uppsala, Sweden
Learning to See What You Need: Gaze Attention for Multimodal Large Language Models
May 13, 2026Junha Song, Byeongho Heo, Geonmo Gu +3Efficient Multimodal Inference Visual Attention
1KAIST · 2NAVER AI Lab
When Vision Speaks for Sound
May 13, 2026Xiaofei Wen, Wenjie Jacky Mo, Xingyu Fu +6Audio-Language Model Evaluation Audio-Visual Understanding
dUniversity of California, Davis · pPrinceton University · wUniversity of Wisconsin–Madison +1
DiM\textsuperscript{3}: Bridging Multilingual and Multimodal Models via Direction- and Magnitude-Aware Merging
May 13, 2026Zijing Wang, Mingyang Wang, Ercong Nie +6Multilingual Language Models Multimodal Large Language Models
Northeastern University, China · CIS, LMU Munich, Germany · Munich Center for Machine Learning (MCML), Germany +1
SMA: Submodular Modality Aligner For Data Efficient Multimodal Learning
May 13, 2026Truong Pham, Anay Majee, Rishabh IyerCross-Modal Learning Cross-Modal Alignment
The University of Texas at Dallas
Multimodal Hidden Markov Models for Persistent Emotional State Tracking
May 13, 2026Anamika Ragu, Aneesh JonelagaddaEmotion Recognition in Conversations Valence-Arousal Modeling
Kaliber AI, San Mateo, California, USA.
MMCL-Bench: Multimodal Context Learning from Visual Rules, Procedures, and Evidence
May 12, 2026Yifan Chen, Fei Yin, Qingyan Bai +2Multimodal ICL Multimodal Reasoning
University of Cambridge · HKUST · Tsinghua University
SynIB: Informational Bottleneck for Maximizing Synergy in Multimodal Learning
May 12, 2026Konstantinos Kontras, Teodora Gagaleska, Thomas Strypsteen +4Cross-Modal Learning Multimodal Learning
1KU Leuven · 2MIT
SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture
May 12, 2026Haiwen Diao, Penghao Wu, Hanming Deng +55Vision-Language Models Unified Multimodal Models
Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models
May 12, 2026Yanting Miao, Yutao Sun, Dexin Wang +8Multimodal Large Language Models Latent Visual Reasoning
1Qwen Large Model Application Team, Alibaba · 3Zhejiang University · University of Waterloo +1
G $^2$ TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Models
May 12, 2026Junxian Li, Kai Liu, Zizhong Ding +4Efficient Multimodal Inference Unified Multimodal Models
Shanghai Jiao Tong University · Huawei Technologies Ltd
OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Models
May 12, 2026Yuchen Deng, Zidang Cai, Hai-Tao Zheng +3Audio-Visual Understanding Efficient Multimodal Inference
Tsinghua Shenzhen International Graduate School, Tsinghua University · Pengcheng Laboratory
Boosting Omni-Modal Language Models: Staged Post-Training with Visually Debiased Evaluation
May 12, 2026Che Liu, Lichao Ma, Xiangyu Tony Zhang +4Cross-Modal Learning Multimodal Large Language Models
StepFun-Audio Team
UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs
May 12, 2026Houcheng Jiang, Jiajun Fu, Junfeng Fang +4Efficient Multimodal Inference Multimodal Large Language Models
University of Science and Technology of China · Zhongguancun Academy · National University of Singapore +1
Human-Grounded Multimodal Benchmark with 900K-Scale Aggregated Student Response Distributions from Japan's National Assessment of Academic Ability
May 12, 2026Kyosuke Takami, Yuka Tateisi, Satoshi Sekine +1Educational Assessment Multimodal Reasoning
Osaka Kyoiku University · University of Tokyo · NII LLMC
OmniThoughtVis: A Scalable Distillation Pipeline for Deployable Multimodal Reasoning Models
May 12, 2026Yuanhao Yue, Chengyu Wang, Yuanjie Lyu +2VLM Distillation Multimodal Reasoning
Alibaba Group, Hangzhou, China

May 15, 2026, cs.CVJ/K move · Enter open · S save

Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models

Yujun Tong, Dongliang Chang, Zijin Yin, Xintong Liu+2

Beijing University of Posts and Telecommunications · Beijing Key Laboratory of Multimodal Data Intelligent Perception and Governance

Unified Multimodal Models Multimodal Reasoning Multimodal Generation Multimodal Foundation Models Multimodal Learning Multimodal Understanding

Open paper AskPDF arXiv

The long-standing goal of multimodal AI is to build unified models in which visual understanding and visual generation mutually enhance one another. Despite recent works such as BAGEL, BLIP3o achieves remarkable progress; In practice, however, this unification remains one-directional: understanding routinely guides generation, yet how and why generation can support understanding is rarely investigated. We revisit this asymmetry and propose Generation-to-Understanding (G2U) synergy, where visual generation becomes an explicit intermediate reasoning step. Our framework enables a model to perform controlled generative acts, such as detail enhancement, context expansion or structural visualisation, to produce self-generated visual thoughts, which are then fed back into the model to refine perception without retraining or external tools. Through a comprehensive evaluation on twelve benchmarks, this reversed information flow consistently improves multimodal understanding. We show that generative fidelity bounds perceptual gain and that distinct families of edit prompts govern transfer efficiency. We further analyse whether models can decide what to imagine. While they can produce plausible edits, these self-generated visual thoughts lack stable task alignment, revealing that current large multimodal models fall short of true self-reflection. This work exposes a missing mechanism in unified cognition and suggests that imagination is not the end of understanding but its beginning.

Momentum

Related topics

Latest papers 413