Multimodal Foundation Models

Follow this topic Ask about this topic

Momentum

13 papers in the last four weeks, up 117% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 303

All topics

Infinity-Parser2 Technical Report
Jul 8, 2026Zuming Huang, Jun Huang, Kexuan Ren +12Synthetic Data Generation Multimodal Large Language Models
MedPMC: A Systematic Framework for Scaling High-Fidelity Medical Multimodal Data for Foundation Models
Jul 8, 2026Hyunjae Kim, Dain Kim, Pan Xiao +25Medical Imaging Foundation Models Training Data Curation
Yale University, New Haven, CT, USA · Korea University, Seoul, South Korea · Washington University in St. Louis, St. Louis, MO, USA +4
Do You Remember? Toward Memory-Centric Multimodal AI
Jul 7, 2026Xuguang Yu, Weigang Zheng, Minyue YuMemory-Augmented VLMs Multimodal Memory
ELSA3D: Elastic Semantic Anchoring for Unified 3D Understanding and Generation
Jul 7, 2026Tianjiao Yu, Xinzhuo Li, Yifan Shen +43D Foundation Models Unified Multimodal Models
University of Illinois Urbana-Champaign
Vision as Unified Multimodal Generation
Jul 7, 2026Xiaoyang Han, Jianhua Li, Kewang Deng +14Unified Multimodal Models Multimodal Generation
SenseTime Research · Peking University · Shanghai Jiao Tong University +3
Unified Audio Intelligence Without Regressing on Text Intelligence
Jul 6, 2026Zhifeng Kong, Sang-gil Lee, Jaehyeon Kim +17TTS Synthesis Unified Multimodal Models
NVIDIA
Bridging Interleaved Multi-Modal Reasoning as a Unified Decision Process
Jul 4, 2026Zican Hu, Xuyang Hu, Yiming Liu +10Unified Multimodal Models Multimodal Reasoning
Nanjing University · Shanghai AI Laboratory · Tsinghua University +5
Attending to Multimodal Generation One Token at a Time
Jul 4, 2026Varun Gupta, Vineet Gandhi, Makarand TapaswiMultimodal Large Language Models Multimodal Generation
CVIT, IIIT Hyderabad
WorldBagel: Uncovering the Power of Unified Multimodal Models for Vision-Language-Action-World Modeling
Jul 3, 2026Zelin Zhao, Min Shi, Bo Yuan +5World Models for Robotics Unified Multimodal Models
Georgia Institute of Technology
OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models
Jul 3, 2026Shijie Cao, Qingyu Zhang, Boxi Yu +6Efficient Multimodal Inference Multimodal Large Language Models
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Limerick +1
Gemma 4 Technical Report
Jul 2, 2026Gemma Team, Sherif El Abd, Vaibhav Aggarwal +298Efficient Multimodal Inference Unified Multimodal Models
Hidden Forgetting in Continual Multimodal Learning: When Accuracy Survives but Grounding Fails
Jul 2, 2026Qianyu Chen, Canran Xiao, Runxuan TangMultimodal Grounding Multimodal Foundation Models
1Nanyang Technological University · 2Shenzhen Campus of Sun Yat-sen University
InduceKV: Fixed-Footprint Continual Adaptation of Multimodal LLMs via Inducing KV Memories
Jul 2, 2026Qianyu Chen, Ziteng Feng, Canran Xiao +1Continual Learning for LLMs Memory-Efficient Fine-Tuning
1Nanyang Technological University · University of Science and Technology of China · 3Shenzhen Campus of Sun Yat-sen University
Multimodal Continuous Reasoning via Asymmetric Mutual Variational Learning
Jul 1, 2026Shijie Li, Yilin Gao, Siyuan Yang +7Multimodal Large Language Models Latent Visual Reasoning
Shanghai Jiao Tong University · Ant Group
Wake up for Touch! Mask-isolated Tactile Alignment Learning in MLLMs
Jul 1, 2026Yoonhyung Park, Minji Kim, Sungwon Moon +1Visuo-Tactile Representation Learning Multimodal Large Language Models
Division of Artificial Intelligence & Software, Ewha Womans University
Rosetta: Composable Native Multimodal Pretraining
Jul 1, 2026Xiangyue Liu, Zijian Zhang, Miles Yang +3Unified Multimodal Models Multimodal Pretraining
HKUST · Tencent Hunyuan
Decompose, Compare, and Decide: Multimodal LLMs are Implicit Few-Shot Learners
Jun 30, 2026Yunhan Wang, Eshika Khandelwal, Edson Araujo +3Multimodal Large Language Models Few-Shot Prompting
Tuebingen AI Center, University of Tuebingen, Germany
FedLAB: Traceable Semantic Codebooks for Federated Multimodal Graph Foundation Learning
Jun 30, 2026Zekai Chen, Kairui Yang, Xuaner Chen +4Multimodal Federated Learning Representation Learning
Beijing Institute of Technology, Beijing, China
CoLT: Teaching Multi-Modal Models to Think with Chain of Latent Thoughts
Jun 30, 2026Lianyu Hu, Shengqian Qin, Zeqin Liao +4Efficient Multimodal Inference Efficient Language Model Reasoning
Nanyang Technological University, Singapore · Shanghai Jiao Tong University, China · NKIARI, Shenzhen Futian, China +2
Rethinking Foundation Model Collaboration: Enhancing Specialized Models through Proxy Task Reasoning
Jun 30, 2026Hongyi Lin, Yang Liu, Jinhua Zhao +1VLM Reasoning Multimodal Foundation Models
School of Vehicle and Mobility, Tsinghua University, Beijing, China · Department of Urban Studies and Planning, Massachusetts Institute of Technology, Cambridge, MA, USA
Orca: The World is in Your Mind
Jun 29, 2026Yihao Wang, Yuheng Ji, Mingyu Cao +54World Model Learning Multimodal Pretraining
Consistency as Inductive Bias: Learning Cross-View Invariance for Robust Multimodal Reasoning
Jun 29, 2026Xin Zou, Haolin Deng, Yibo Yan +6Multi-View Consistency Multimodal Robustness
HKUST (GZ) · HKUST · UIC +1
Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation
Jun 29, 2026Xin Zou, Haolin Deng, Yibo Yan +5Multimodal Large Language Models Multimodal Hallucination
HKUST (GZ) · HKUST · OPPO AI Center
MIRROR: Aligning Semantic Relations from Language to Image via Gromov--Wasserstein
Jun 28, 2026Hong-Han Wang, Yuntao Wang, Hu DingLarge Vision-Language Models VLM Reasoning
University of Science and Technology of China, Hefei, China
COMPASS: Grounding Composition-Intent Guidance in Unified Multimodal Models
Jun 27, 2026Ziqi Zhou, Weize Quan, Mining Tan +6Multimodal Grounding Unified Multimodal Models
University of Edinburgh, United Kingdom · State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences, China · University of Chinese Academy of Sciences, China +1
ZooClaw-FashionSigLIP2: Distilled Fine-tuning for Robust Fashion Retrieval
Jun 26, 2026Siqiao Xue, Chunxue XuVLM Robustness VLM Adaptation
Ask, Solve, Generate: Self-Evolving Unified Multimodal Understanding and Generation via Self-Consistency Rewards
Jun 25, 2026Ritesh Thawkar, Shravan Venkatraman, Omkar Thawakar +5Unified Multimodal Models Multimodal Generation
1Mohamed bin Zayed University of Artificial Intelligence · 4Linköping University · 3Australian National University +1
Unison: Benchmarking Unified Multimodal Models via Synergistic Understanding and Generation
Jun 25, 2026Jinyu Liu, Xincheng Shuai, Henghui Ding +1Unified Multimodal Models Multimodal Foundation Models
Institute of Trustworthy Embodied AI, Fudan University, Shanghai, China · Shanghai Key Laboratory of Multimodal Embodied AI, Shanghai, China · Institute of Big Data, College of Computer Science and Artificial Intelligence, Fudan University, Shanghai, China
ReasonCLIP-58M: Visually Grounded Commonsense Reasoning Supervision for CLIP
Jun 25, 2026Sicheng Zhang, Muzammal Naseer, Binzhu Xie +5Compositional Reasoning Multimodal Reasoning
Khalifa University · University of Western Australia · The Chinese University of Hong Kong +2
iFLYTEK-Embodied-Omni Technical Report
Jun 24, 2026Yuan Zhang, Jingfei Ni, Guanchen Lu +12Unified Multimodal Models Robot Motion Generation
1iFLYTEK · University of Science and Technology of China · 2LindenBot

Jul 8, 2026, cs.AIJ/K move · Enter open · S save

Infinity-Parser2 Technical Report

Zuming Huang, Jun Huang, Kexuan Ren, Baode Wang+11

Synthetic Data Generation Multimodal Large Language Models Document Parsing Multi-Task RL Multimodal Foundation Models Multimodal Understanding

Open paper AskPDF arXiv

We present Infinity-Parser2, a large multimodal model that couples a controllable data-synthesis pipeline with multi-task reinforcement learning for end-to-end document parsing, addressing the persistent scarcity of faithfully annotated parsing corpora. Our contributions are threefold. First, we build a scalable synthesis engine, pairing a controllable rendering framework with an iterative refinement loop, and use it to construct and open-source Infinity-Doc2-5M: a 5-million-sample bilingual (Chinese/English) corpus spanning diverse document types, annotated with element bounding boxes, canonical content forms (Markdown, HTML, LaTeX, SMILES, structured charts), and full-page reading order. Second, we introduce a verifiable, multi-task reward system that enables Joint Reinforcement Learning across eight co-trained objectives (document parsing, layout analysis, table parsing, math formula parsing, chart parsing, chemical formula parsing, document VQA, and general multimodal understanding), unifying perception, structure, and reasoning in a single optimization signal. Third, we release two variants under a shared architecture: Infinity-Parser2-Flash, optimized for low-latency inference with a 3.68x throughput gain over Infinity-Parser-7B, and Infinity-Parser2-Pro, engineered for precision-critical settings. Infinity-Parser2-Pro reaches state-of-the-art 87.6% on olmOCR-Bench and 74.3% on ParseBench, surpassing DeepSeek-OCR-2, PaddleOCR-VL-1.5, and MinerU2.5, with strong generalization to charts, chemical formulas, and document VQA.

Momentum

Related topics

Latest papers 303