Multimodal Agents

Follow this topic Ask about this topic

Latest papers 364

49 papers in the last four weeks, up 20% on the four weeks before. 0.3% of all new papers.

All topics

VIEWCRAFT: Evolving What Multimodal Agents See and How They Read It
Oct 11, 2026Binxiao Xu, Ruichuan An, Bocheng Zou +7Multimodal Agents LLM Agent Skill Learning
PKU · UW–Madison · CMU +4
How Should a Camera-First AI Agent Work?
Oct 10, 2026Xiaofan Wu, Xi Zeng, Miaoxia Chen +6Multimodal Agents Memory-Augmented Agents
Chance AI
JoyInside-VLM-Agent: Product-Conditioned Multimodal Policy Learning for Embodied Agents
Oct 9, 2026Gangyi Li, LuYao Wang, Jingfu Sun +7Multimodal Agents Robot Policy Learning
What Should Embodied Agents Remember? Benchmarking and Evaluating Memory Modality for Embodied Agents
Oct 8, 2026Junxiang Wang, Yuchen Liu, Yi Wu +6Agent Memory Memory Agent Benchmarks
Dexmate · Carnegie Mellon University · Purdue University
OA-MAP: Evidence-Grounded Multi-Agent Multimodal Framework for Interpretable Knee Osteoarthritis Progression
Oct 8, 2026Sixu Chen, Mingrui Yang, Qiang Guan +1Clinical Outcome Prediction Multimodal Agents
Cleveland Clinic, Cleveland, OH, USA
ActiveMedAgent: Cost-Aware Trajectory Learning for Multimodal Medical Diagnosis
Oct 8, 2026Weiwei Ma, Xiaobing Yu, Peijie Qiu +5Medical Diagnosis Active Information Gathering
Washington University in St. Louis · Yale University · Icahn School of Medicine at Mount Sinai +1
RobotWorld: Benchmarking Multimodal Agents for Robot Use Across Diverse Tasks and Embodiments
Oct 7, 2026Zhiqin Yang, Chenxin Li, Xiaomeng Hu +30Robot Manipulation Benchmarks Multimodal Agents
RoboQuest: Generalist Physical Agents that Search, Inspect and Test
Oct 7, 2026Liu Renhang, Navonil Majumder, Tej Deep Pala +1Robot Manipulation Benchmarks Multimodal Agents
Nanyang Technological University
DUDA-Bench: Benchmarking LLM Agents on Multimodal Data-Driven Urban Diagnosis
Oct 7, 2026Yizhi Song, Hang Ni, Weijia Zhang +1Data Analysis Agents AI Agent Benchmarks
The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, China
Visual Orchestration Tax in Agentic VLM Pipelines: Auditing and Certifying Visual Evidence Reuse
Oct 6, 2026Lingteng ZengVLM Evaluation AI Agent Auditing
Faculty of Engineering, The Chinese University of Hong Kong Hong Kong, China
MedPrune: Topology-Efficient Multimodal Multi-Agent Communication Evolution for Medical VQA Tasks
Oct 5, 2026Jiuheng Wan, Runze Li, Chen Chen +5Medical VQA Multi-Agent Collaboration
School of Computer Science and Information Engineering, Hefei University of Technology · Nanjing University · Guangdong University of Finance and Economics +2
From Social Reasoning to Embodied Interaction: An Agentic Framework for Social Robots
Oct 5, 2026Ziyu Cheng, Yuewen Guo, Zhirui Liu +5Agentic Reasoning Human-Robot Interaction
ShanghaiTech University · InstAdapt
Long-MDR: Long-Context Reinforcement Learning for Multimodal Deep-Research Agents
Oct 4, 2026On Tai TangReinforcement Learning Deep Research Agents
The Hong Kong University of Science and Technology (HKUST)
VoCa: Designing Speech-Canvas Interaction for Voice-Based Conversational Agents
Oct 3, 2026Yate Ge, Run Yuan, Yueran Qi +7Human-AI Interaction Conversational Agents
Tongji University Shanghai, China · Grinnell College Grinnell, Iowa, United States · Hunan University Changsha, Hunan, China +1
VisAudit: Evaluating Multimodal Agents for Visual Diagnosis and Repair
Oct 1, 2026Shicheng Liu, Adam Kahirov, Qi Zhang +5Multimodal Model Evaluation Multimodal Agents
VISTA: A Visual Harness for Reasoning in an Interactive World
Oct 1, 2026Qiushi Han, Keya Hu, Linlu Qiu +2Memory-Augmented VLMs Visual Reasoning
Massachusetts Institute of Technology
Form and Void: Entangled Composition through an Autonomous AI Agent
Oct 1, 2026Shiwen Wang, Jian Yang, Xu Wang +2Image Generation Compositional Image Generation
School of AI, University of Chinese Academy of Sciences · Renmin University of China · Shanghai Theatre Academy +1
WorldAuditBench: Interactive 3D World Auditing with Multimodal Agents
Sep 30, 2026Ziyan Jiang, Jingbo Yang, Jiabao Ji +5Multimodal Anomaly Detection Multimodal Agents
UC Santa Barbara · MIT CSAIL · MIT-IBM Watson AI Lab
AIMS: An Agentic AI Framework for Sim-to-Real Multi-Modal ISAC
Sep 30, 2026Yijie Bian, Kai Zhang, Wei Guo +4Integrated Sensing and Communication Sim-to-Real Transfer
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Clear Water Bay, Hong Kong
NavHarness: Adaptive Goals for Agentic Vision-Language Navigation
Sep 30, 2026Haoxiang Shi, Zaijing Li, Muhe Ding +3Vision-Language Navigation Multimodal Agents
Harbin Institute of Technology (Shenzhen) · Pengcheng Laboratory
VideoLoop: Looped Working Memory Against Semantic Thrashing in Long-Form Video Agents
Sep 29, 2026Jianming Xu, Jinfa Huang, Jingyang Lin +2Memory-Augmented VLMs Long-Video Understanding
University of Rochester · Microsoft
VISTA: Internalizing Collective Visual Experience via On-Policy Distillation for Active Multimodal Agents
Sep 29, 2026Zheng Jiang, Houde Qian, Yiming Chen +5On-Policy Distillation Multimodal Agents
Tsinghua University
EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memory
Sep 29, 2026Ziyun Zeng, Hang Hua, Shaden Alshammari +3LLM Agent Memory Multi-Agent Systems
MIT-IBM Computing Research Lab · University of Rochester · Massachusetts Institute of Technology
MatToolBench: Benchmarking Multimodal Agents in Real-World Materials Science Workflows
Sep 29, 2026Mei Wu, Rui Xie, Runyu Zhang +6GUI Agents AI Agent Benchmarks
X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · Suzhou Laboratory, Suzhou, China · State Key Laboratory for General Artificial Intelligence, BIGAI, Beijing, China +3
MERID: Multimodal Exploration via Recursive Self-Improvement Agents for Major Depression Analysis
Sep 28, 2026Lei Liu, Zhaokang Liang, Qingcheng Zeng +4Depression Detection Multimodal Classification
Yale University · Zhejiang University · Northwestern University +3
FlowAct-R2: Beyond Talking Avatar via Streaming Multimodal References and Proactive Agent Planning
Sep 28, 2026Ziyao Huang, Zhengkun Rong, Shiyang Qin +5Action-Conditioned Video Generation Real-Time Interactive Video Generation
Bytedance Intelligent Creation
A.D.A.M.O. (Agent for language-Driven Actions with Multimodal Observations): A Visual-Symbolic Framework for Virtual Humans
Sep 28, 2026Alessandro Emmanuel Pecora, Stefano Calzolari, Francesco Strada +1Language Model-Based Control World Models
DAUIN, Politecnico di Torino, Torino, Italy
Recent Advances in Agentic Agri-Robotic Phenotyping: A Perspective Review from Fragmented Multimodal Sensing to Unified PhenoAgent Intelligence
Sep 28, 2026Muhammad Owais, Ehtesham Iqbal, Samee Ullah Khan +3Plant Phenotyping Agricultural Robotics
Khalifa University Center for Autonomous Robotic Systems (KU-CARS), Khalifa University of Science and Technology, Abu Dhabi, United Arab Emirates. · Advanced Research and Innovation Center (ARIC), Khalifa University of Science and Technology, Abu Dhabi, United Arab Emirates. · Department of Computer Science and Engineering, Chung-Ang University, Seoul, South Korea. +1
SkillRubric: Co-Evolving Actor Guidance and Evaluator Rubrics for Multimodal Agents
Sep 28, 2026Bingqing Jiang, Guoxi Zhang, Jasper Wang +7Multimodal Reward Modeling Rubric-Based RL
The University of Hong Kong · WeChat, Tencent · Peking University +2
MMSkillRisk: Can Agents Stay Safe When Multimodal Skills Become Traps?
Sep 28, 2026Lingqi Jiang, Jialuo Chen, Jianan Ma +8Prompt Injection Attacks on AI Agents Agent Skill Security Auditing
Zhejiang University · Ant Group · Hangzhou Dianzi University +1

Oct 11, 2026J/K move · Enter open · S save

VIEWCRAFT: Evolving What Multimodal Agents See and How They Read It

Binxiao Xu, Ruichuan An, Bocheng Zou, Weihua Du+6

PKU · UW–Madison · CMU+4

Multimodal Agents LLM Agent Skill Learning Multimodal Tool Use

Open paper AskPDF arXiv

Reusable skills improve multimodal agents without updating model weights, but their effectiveness depends on both the evidence presented and the procedure used to read it. We introduce ViewCraft, which jointly evolves an executable evidence interface and its reading procedure. The interface selects, transforms, and organizes observations; the reader interprets the resulting views. Failure traces guide revisions, replay and paired evaluation retain useful changes, and interface projection recovers visual edits from eligible rejected joint proposals. Across eight benchmarks, ViewCraft improves over direct execution by an average of

6.1

points in task score, with gains on every benchmark. The advantage persists at a matched input-token budget. On all five multi-source tasks, the learned interface uses 8%-15% fewer input tokens than Initial skill. The learned skills also transfer across executors and outperform direct execution in every evaluated transfer, supporting evidence construction and reading as a coupled, reusable skill.