Multimodal Understanding

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, up 233% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 413

All topics

Infinity-Parser2 Technical Report
Jul 8, 2026Zuming Huang, Jun Huang, Kexuan Ren +12Synthetic Data Generation Multimodal Large Language Models
Do You Remember? Toward Memory-Centric Multimodal AI
Jul 7, 2026Xuguang Yu, Weigang Zheng, Minyue YuMemory-Augmented VLMs Multimodal Memory
Vision as Unified Multimodal Generation
Jul 7, 2026Xiaoyang Han, Jianhua Li, Kewang Deng +14Unified Multimodal Models Multimodal Generation
SenseTime Research · Peking University · Shanghai Jiao Tong University +3
CMDR: Contextual Multimodal Document Retrieval
Jul 7, 2026Ryota Tanaka, Taku Hasegawa, Kyosuke NishidaMultimodal IR Multimodal Embedding
Human Informatics Labs., NTT, Inc., Tokyo, Japan
Audio Sentiment Analysis via Distillation and Cross-Modal Integration of Generated Multilingual Transcripts
Jul 7, 2026Andrei-George Durdun, Victor Constantinescu, Radu Tudor IonescuCross-Modal Learning Sentiment Analysis
Department of Computer Science, University of Bucharest, 15G Iuliu Maniu, Bucharest 061075, Romania · Department of Data Science, PPC Romania, 30 Mircea Voda, Bucharest 030667, Romania
Light-Omni: Reflex over Reasoning in Agentic Video Understanding with Long-Term Memory
Jul 6, 2026Chang Nie, Jiaju Wei, Junlan Feng +2Multimodal Agents Memory-Augmented Agents
Nanjing University
Unified Audio Intelligence Without Regressing on Text Intelligence
Jul 6, 2026Zhifeng Kong, Sang-gil Lee, Jaehyeon Kim +17TTS Synthesis Unified Multimodal Models
NVIDIA
Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesis
Jul 6, 2026Zhipeng Xu, Zulong Chen, Qing Liu +6Synthetic Data Generation Document Information Extraction
Alibaba Group · Taobao and Tmall Group, Alibaba · Qwen Team, Alibaba Group +1
Hierarchical Evidence-Driven Reasoning for Long Document Understanding
Jul 6, 2026Junyu Xiong, Yonghui Wang, Rongjian Gu +4Multi-Hop QA Document Understanding
University of Science and Technology of China · iFLYTEK Research
Transferability Between Understanding and Generation in Unified Multimodal Models
Jul 5, 2026Jiwon Kang, Heeji Yoon, Jaewoo Jung +5Unified Multimodal Models Multimodal Generation
KAIST AI, South Korea · Blynx, South Korea · Trillionlabs, South Korea
Q-TriM: Question-Guided Tri-Modal Attention for Audio-Visual Question Answering
Jul 4, 2026SungHun Kim, SeungJun BaekMultimodal QA Cross-Modal Attention
Dept. of Computer Science and Engineering Korea University Seoul, South Korea
Bridging Interleaved Multi-Modal Reasoning as a Unified Decision Process
Jul 4, 2026Zican Hu, Xuyang Hu, Yiming Liu +10Unified Multimodal Models Multimodal Reasoning
Nanjing University · Shanghai AI Laboratory · Tsinghua University +5
Attending to Multimodal Generation One Token at a Time
Jul 4, 2026Varun Gupta, Vineet Gandhi, Makarand TapaswiMultimodal Large Language Models Multimodal Generation
CVIT, IIIT Hyderabad
Modality Relevance is not Modality Utility: Post-hoc Selective Modality Escalation for Cost-Aware Multimodal RAG
Jul 3, 2026Xue Li, Yiming GaiMultimodal RAG Efficient Multimodal Inference
Hangzhou International Innovation Institute, Beihang University Hangzhou, China
OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models
Jul 3, 2026Shijie Cao, Qingyu Zhang, Boxi Yu +6Efficient Multimodal Inference Multimodal Large Language Models
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Limerick +1
Gemma 4 Technical Report
Jul 2, 2026Gemma Team, Sherif El Abd, Vaibhav Aggarwal +298Efficient Multimodal Inference Unified Multimodal Models
Show Me Examples: Inferring Visual Concepts from Image Sets
Jul 2, 2026Nick Stracke, Kolja Bauer, Stefan Andreas Baumann +3Vision-Language Models Conditional Image Generation
Munich Center for Machine Learning · Apple
Hidden Forgetting in Continual Multimodal Learning: When Accuracy Survives but Grounding Fails
Jul 2, 2026Qianyu Chen, Canran Xiao, Runxuan TangMultimodal Grounding Multimodal Foundation Models
1Nanyang Technological University · 2Shenzhen Campus of Sun Yat-sen University
Towards High-Resolution Visual Perception via Hierarchical Entity Exploration
Jul 1, 2026Ziyu Ma, Shidong Yang, Yuxiang Ji +5Large Vision-Language Models Fine-Grained Visual Perception
AMAP, Alibaba Group, China · Monash University, Australia
M2Note: Continual Evolution of Vision Language Models via Mistake Notebook Learning
Jul 1, 2026Haiwen Li, Jing Tang, Rui Chen +2Memory-Augmented VLMs VLM Reasoning
AMAP, Alibaba Group
H-OPD: Confidence Aware Heterogeneous Multi-Teacher Multimodal On-policy Distillation
Jul 1, 2026Qixiang Yin, Huanjin Yao, Yuchen Cai +5Cross-Modal Knowledge Distillation VLM Distillation
Beijing University of Posts and Telecommunications · Zhongguancun Academy, Beijing, China · ByteDance +3
Homer: Understanding Long-form Videos with Hierarchical Memory and Agentic Reasoning
Jul 1, 2026Yixin Ji, Fanghua Ye, Juntao Li +5Long-Video Understanding Memory-Augmented Video Understanding
1Soochow University · 2Tencent Hunyuan Multimodal Department
Multimodal Continuous Reasoning via Asymmetric Mutual Variational Learning
Jul 1, 2026Shijie Li, Yilin Gao, Siyuan Yang +7Multimodal Large Language Models Latent Visual Reasoning
Shanghai Jiao Tong University · Ant Group
Wake up for Touch! Mask-isolated Tactile Alignment Learning in MLLMs
Jul 1, 2026Yoonhyung Park, Minji Kim, Sungwon Moon +1Visuo-Tactile Representation Learning Multimodal Large Language Models
Division of Artificial Intelligence & Software, Ewha Womans University
Rosetta: Composable Native Multimodal Pretraining
Jul 1, 2026Xiangyue Liu, Zijian Zhang, Miles Yang +3Unified Multimodal Models Multimodal Pretraining
HKUST · Tencent Hunyuan
CoLT: Teaching Multi-Modal Models to Think with Chain of Latent Thoughts
Jun 30, 2026Lianyu Hu, Shengqian Qin, Zeqin Liao +4Efficient Multimodal Inference Efficient Language Model Reasoning
Nanyang Technological University, Singapore · Shanghai Jiao Tong University, China · NKIARI, Shenzhen Futian, China +2
Attend, Transform, or Silence: Operator-Level Visual Skipping for Efficient Multimodal LLM Inference
Jun 30, 2026Zhaoyang Luo, Runmin Dong, Miao Yang +4Efficient VLM Inference Multimodal Large Language Models
Tsinghua Shenzhen International Graduate School, Shenzhen, China · Sun Yat-sen University, Zhuhai, China · Tsinghua University, Beijing, China +1
Token-Sparse Medical Multimodal Reasoning via Dual-Stream Reinforcement Learning
Jun 30, 2026Kaitao Chen, Weiqian Zhao, Jiamin Wu +6Efficient Multimodal Inference Medical VQA
1Fudan University · 2Shanghai Artificial Intelligence Laboratory · 3Shanghai Jiao Tong University +1
MS-Resampler: Multi-Scope Visual Resampling for Efficient Multimodal LLMs
Jun 30, 2026Zhongyang Li, Yaqian Li, Faming Fang +6Multimodal Large Language Models Multimodal Fusion
Li Auto Inc. · East China Normal University.
Learning to Deny: Action Denial in Multimodal Large Language Models
Jun 30, 2026Raiyaan Abdullah, Shehreen Azad, Yogesh Singh RawatMultimodal Large Language Models Causal Reasoning in Videos
Institute of Artificial Intelligence, University of Central Florida, USA

Jul 8, 2026, cs.AIJ/K move · Enter open · S save

Infinity-Parser2 Technical Report

Zuming Huang, Jun Huang, Kexuan Ren, Baode Wang+11

Synthetic Data Generation Multimodal Large Language Models Document Parsing Multi-Task RL Multimodal Foundation Models Multimodal Understanding

Open paper AskPDF arXiv

We present Infinity-Parser2, a large multimodal model that couples a controllable data-synthesis pipeline with multi-task reinforcement learning for end-to-end document parsing, addressing the persistent scarcity of faithfully annotated parsing corpora. Our contributions are threefold. First, we build a scalable synthesis engine, pairing a controllable rendering framework with an iterative refinement loop, and use it to construct and open-source Infinity-Doc2-5M: a 5-million-sample bilingual (Chinese/English) corpus spanning diverse document types, annotated with element bounding boxes, canonical content forms (Markdown, HTML, LaTeX, SMILES, structured charts), and full-page reading order. Second, we introduce a verifiable, multi-task reward system that enables Joint Reinforcement Learning across eight co-trained objectives (document parsing, layout analysis, table parsing, math formula parsing, chart parsing, chemical formula parsing, document VQA, and general multimodal understanding), unifying perception, structure, and reasoning in a single optimization signal. Third, we release two variants under a shared architecture: Infinity-Parser2-Flash, optimized for low-latency inference with a 3.68x throughput gain over Infinity-Parser-7B, and Infinity-Parser2-Pro, engineered for precision-critical settings. Infinity-Parser2-Pro reaches state-of-the-art 87.6% on olmOCR-Bench and 74.3% on ParseBench, surpassing DeepSeek-OCR-2, PaddleOCR-VL-1.5, and MinerU2.5, with strong generalization to charts, chemical formulas, and document VQA.

Momentum

Related topics

Latest papers 413