Multimodal Large Language Models

Also known as MLLM

Follow this topic Ask about this topic

Momentum

67 papers in the last four weeks, up 91% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 676

All topics

ReGraph: Learning to Generate Recipe Graphs from Food Images
Aug 7, 2026Guoshan Liu, Bin Zhu, Pengkun Jiao +3Graph Generation Graph Representation Learning
Institute of Trustworthy Embodied AI, Fudan University, China and Shanghai Key Laboratory of Multimodal Embodied AI, China · Singapore Management University, Singapore
Stockmark-Nemotron-3-Nano-Omni-JapanDocReader: Structured Document Parsing via Capability Injection and Forgetting Control
Aug 7, 2026Shi Chen, Hayato Aida, Makoto Morinaga +2Document Understanding Multimodal Large Language Models
Stockmark Inc.
AgentPatch: Coarse-to-Fine Weak-Task Repair for Merging Agentic Multimodal Large Language Models
Aug 7, 2026Zibo Shao, Baochen Xiong, Chengdong Xu +6Multimodal Large Language Models Model Merging
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · Pengcheng Laboratory, Shenzhen, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China +1
Can MLLMs Decode the Creative Leap? Introducing C4 for Cross-Concept Understanding
Aug 6, 2026Ming Wang, Yuqing Zhang, Tingna Xie +5Creativity Assessment Multimodal Large Language Models
School of Computer Science and Engineering, Northeastern University, Liaoning, China · School of Computing and Information Systems, Singapore Management University, Singapore · School of Computer Science and Engineering, Shandong University of Science and Technology, Shandong, China
MoCA: Implicit Social Context Analysis
Aug 6, 2026Wenhao Xu, Kaiwen Zhang, Hao Li +10Abductive Reasoning Multimodal Large Language Models
National University of Singapore, Singapore, Singapore. · Wuhan University, Wuhan, China. · University of Oxford, Oxford, United Kingdom.
ChronoVision: Temporal Reasoning via Latent State Reconstruction
Aug 6, 2026Yifan Shen, Jian Xu, Boyi Li +6Multimodal Large Language Models Latent Visual Reasoning
University of Illinois Urbana-Champaign · PediaMed AI · University of Pennsylvania
Beyond Frame Selection: Rethinking Long-Video Understanding with MLLMs
Aug 6, 2026Ziling Huang, Shin'ichi SatohMultimodal Large Language Models Long-Video Understanding
National Institute of Informatics, Japan
SmartMage: Dynamic Modality Orchestration for 3D Scene Understanding
Aug 5, 2026Yue Zhang, Yingzhao Jian, Yunqiu Xu +2Mixture of Experts Multimodal Large Language Models
Zhejiang University Hangzhou, Zhejiang, China · Stanford University Stanford, California, USA
OPD-V: Visual On-Policy Self-Distillation with Modality Balance
Aug 5, 2026Aniri, Jinhe Bi, Peng Liao +5Modality Imbalance On-Policy Self-Distillation
2Ludwig Maximilian University of Munich · 3Munich Center for Machine Learning · 1National University of Singapore +1
Trace, Verify, and Correct: A Training-Free Framework for Spatial Reasoning in Multimodal LLMs
Aug 5, 2026Yang Yang, Jiawei Chen, Tairan Chen +1Multimodal Grounding Visual Spatial Reasoning
Shanghai Key Laboratory of Multidimensional Information Processing, East China Normal University · Zhongguancun Academy · Stevens Institute of Technology
A Model Merging Approach for Continual MLLM Unlearning
Aug 5, 2026Yuhang Wang, Linlin Zhang, Haoxuan Ji +3Multimodal Large Language Models MLLM Unlearning
1Xidian University, Xi’an, China · 2Xi’an Jiaotong University, Xi’an, China
Multi-Branch Policy Optimization for Multimodal Large Language Models
Aug 5, 2026Shuai Lyu, Yuning Gong, Ruiling Gao +7Credit Assignment in RL Multimodal Large Language Models
Beijing University of Posts and Telecommunications Beijing, China · Sichuan University Chengdu, China · Shanghai University Shanghai, China +1
SIGNPOST-Bench: Benchmarking Text-Vision Conflict Resolution in Multimodal Large Language Models
Aug 4, 2026Sirun Li, Minghao Liu, Ling Dai +4VLM Robustness Multimodal Large Language Models
Peking University, Beijing, China · 22077AI · The Hong Kong University of Science and Technology, Hong Kong, China
ParVL: Parallel Scaling and Expandable Compute Allocation for Multimodal LLMs
Aug 4, 2026Yang Yang, Qinyu Zhao, Mouxiang Chen +5Vision-Language Models Multimodal Large Language Models
1The Australian National University · 2Shanghai AI Laboratory · 3Zhejiang University +2
OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models
Aug 4, 2026Wanshun Su, Yang Shi, Feihu Liu +10Efficient Multimodal Inference Audio Token Compression
Northwestern Polytechnical University · Peking University · Alibaba Group +1
Failure-Informed Image Self-Augmentation for Multimodal Large Language Model Self-Improvement
Aug 4, 2026Chunyang Jiang, Pingping Zhang, Yuzhi Zhao +9Synthetic Data Augmentation Data Augmentation
Hong Kong University of Science and Technology · City University of Hong Kong · Chinese University of Hong Kong
Pattern over Pixels: Measuring Pattern Completion Bias in Multimodal Code Generation
Aug 4, 2026Khai-Nguyen Nguyen, Oscar Chaparro, Antonio MastropaoloCode Generation Multimodal Large Language Models
William & Mary Williamsburg, USA · University of Virginia
Taming the Implicit: Dual-Channel Risk-Aware Reinforcement Fine-Tuning for Continual Multimodal Post-Training
Aug 4, 2026Yibei Liu, Jiajun Chen, Qianle Zhang +4Multimodal Large Language Models Risk-Sensitive RL
University of Electronic Science and Technology of China · Zhejiang University · South China University of Technology +1
ChartAnno: Benchmarking Multimodal Large Language Models for Chart Annotation Generation
Aug 4, 2026Zhenghan Chen, Zekai Shao, Lidan Tan +10Data Visualization Multimodal Large Language Models
Fudan University · Sun Yat-sen University · The Hong Kong University of Science and Technology (Guangzhou) +1
Balancing Efficiency and Efficacy: Training-Free Attention-Guided Switching Between Explicit and Latent Thoughts for MLLMs
Aug 4, 2026Haoqian Kang, Liupeng Li, Kuofeng Gao +5Efficient Multimodal Inference Multimodal Large Language Models
Harbin Institute of Technology, Shenzhen Shenzhen, China · Tsinghua Shenzhen International Graduate School, Tsinghua University Shenzhen, China · Peng Cheng Laboratory Shenzhen, China +1
ArtECulture: Benchmarking Culture-Conditioned Visual Emotion Understanding in Multimodal Large Language Models
Aug 4, 2026Xiaolin Chen, Xuemeng Song, Wenhao Shi +3Multimodal Large Language Models Multimodal Emotion Recognition
National University of Singapore · Southern University of Science and Technology · Nanyang Technological University
DocTrace: Towards Traceable Long Document VQA via Hierarchical Evidence Graph Reasoning
Aug 4, 2026Le Xiang, Zhicheng Guan, Hong Chen +5Visual Question Answering Multimodal Large Language Models
Baidu Basic Model Research and Development Department, Baidu Inc. · Tsinghua Shenzhen International Graduate School, Tsinghua University
NeuroMosaic: Anatomically Grounded Multimodal Large Language Modeling for Molecularly Aware Glioma Reasoning from 3D MRI and Clinical Narratives
Aug 4, 2026Yantong Liu, Zheyu Zhang, Runpeng Liu +3Multimodal Large Language Models Medical VLMs
Department of Computer Information Engineering, Kunsan National University, Gunsan 54150, Republic of Korea · School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China
LDU-Bench: Multimodal LLM Evaluation for Lithography Defect Understanding under Layout-Varying Circuit Backgrounds
Aug 4, 2026Huanglong Ji, Botong Zhao, Shujing Lv +1Semiconductor Manufacturing Multimodal Large Language Models
ET-Prune: Evidence-Aware Dynamic Budgeting for Visual Token Pruning in Text-Rich MLLMs
Aug 3, 2026Zizhong Ding, Junxian Li, Kai Liu +4Efficient Multimodal Inference Multimodal Large Language Models
Shanghai Jiao Tong University · Xidian University
Exploring and Bridging Knowledge Holes in Unlearned Multimodal Large Language Models
Aug 3, 2026Junxiang You, Junkai Chen, Yuhao He +3Multimodal Large Language Models MLLM Unlearning
University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · 3Cloudspace Technology
DeepVoyager-VL: Incentivizing Vision-in-the-Loop Search for Long-Horizon Multimodal Agents
Aug 3, 2026Huanyao Zhang, Jiepeng Zhou, Runhao Zhao +12Multimodal IR Multimodal Large Language Models
1PKU · 2HKUST(GZ) · 3NUDT +3
Mitigating Visual Degradation in MLLMs via Spatial-Spectral Visual Anchor Learning
Aug 3, 2026Qianlong Yang, Bowen Ye, Xianda Guo +4VLM Robustness Multimodal Large Language Models
China University of Petroleum (East China) Qingdao, Shandong, China · Great Wall Motor · Shanghai Jiao Tong University Shanghai, China +3
PixVL: Self-Supervised Training of Pixel-Level MLLMs via a Unified Mask--Text Consistency Cycle
Aug 2, 2026Yicheng Xiao, Haoxuan Ma, Caorui Li +7Self-Supervised Learning VLM Adaptation
Institute of Automation, Chinese Academy of Sciences · Nanjing University · Southeast University +3
CodeShrink: Adaptive Visual Compression for Efficient Multimodal Code Understanding
Jul 31, 2026Wenxin Tang, Jingyu Xiao, Zhenyu Liu +6Efficient Multimodal Inference Multimodal Large Language Models
Tsinghua University · The Chinese University of Hong Kong · Northwestern Polytechnical University +4

Aug 7, 2026, cs.AIJ/K move · Enter open · S save

ReGraph: Learning to Generate Recipe Graphs from Food Images

Guoshan Liu, Bin Zhu, Pengkun Jiao, Jingjing Chen+2

Institute of Trustworthy Embodied AI, Fudan University, China and Shanghai Key Laboratory of Multimodal Embodied AI, China · Singapore Management University, Singapore

Graph Generation Graph Representation Learning Multimodal Large Language Models

Open paper AskPDF arXiv

Recent Large Multimodal Models (LMMs) have achieved impressive performance in recipe generation from food images.However, cooking is a structured transformation process in which ingredients undergo state changes through ordered actions,while free-form recipe language leaves the corresponding entities, intermediate states, and dependencies largely implicit and entangled.A graph representation makes this procedural knowledge explicit and compositional, providing a structured basis for assessing whether model outputs encode process-level knowledge rather than merely presenting plausible textual descriptions. To address this limitation, we present ReGraph, a large-scale recipe graph dataset that represents ingredients, cooking actions, and tools as entities, uses entity attributes to describe ingredient state changes, and employs typed relations to encode manipulation targets, destinations, and procedural ordering. ReGraph further incorporates explicit Recipe Reasoning Chain-of-Thought (RR-CoT) traces, providing auxiliary supervision for procedural decomposition and structured graph generation. Building on ReGraph, we propose Recipe Graph Learning (RGL), a two-stage framework that enables LMMs to generate a plausible fine-grained cooking workflow from a food image in the form of a structured recipe graph. Under a deterministic, schema-aware matching protocol, our experiments reveal a substantial gap between text-generation quality and recoverable procedural structure: recipes produced by existing approaches achieve competitive text-generation scores yet yield limited reference-aligned entity and relation structure under the ReGraph schema. In contrast, across two representative LMM backbones, RGL consistently improves the generation of cooking entities and procedural relations, while our analysis further shows that fine-grained ingredient-state capture remains the most challenging dimension.

Momentum

Related topics

Latest papers 676