Multimodal Agents

Follow this topic Ask about this topic

Multimodal agents combine language models with visual, audio, and video perception to answer questions, use tools, and interact with apps, websites, robots, or other environments. Common approaches include planning and tool orchestration, specialized collaborating agents, active evidence gathering, and memory or reusable skills for long tasks; reinforcement learning and execution feedback are increasingly used to improve these behaviors. Benchmarks now test browsing, GUI use, video reasoning, and embodied tasks, often evaluating action traces and process reliability as well as final answers. Despite progress, agents remain brittle over long horizons, and more modalities or more agents do not automatically improve grounding, safety, efficiency, or task success.

Latest papers 367

49 papers in the last four weeks, up 20% on the four weeks before. 0.3% of all new papers.

All topics

CaM-Wolf: Causal-Aware Multimodal Agents for Social Deduction Games
Jul 29, 2026Zheng Zhang, Nanjie Yao, Jiarui He +3Game-Playing Agents Social Deduction Games
The Hong Kong University of Science and Technology (Guangzhou) · Nanyang Technological University · Shanghai Jiao Tong University
VetClaw: An Edge-Cloud Multimodal Agentic System for Veterinary Disease Screening
Jul 28, 2026Syed Mhamudul Hasan, Anas AlSobeh, Hussein Zangoti +1Multimodal Classification Tool-Using Vision-Language Agents
School of Computing, Southern Illinois University, Carbondale, 62901, IL, USA. · 2Information Systems and Technology, Utah Valley University, Orem, Orem, 84058, UT, USA. · College of Engineering and Computer Science, Jazan University, Saudi Arabia.
Agentic AI in medicine: architectures, applications, evaluation, and challenges for clinical translation
Jul 28, 2026Zheng Tong, Yang Liu, Wanshu Fan +6AI Agent Evaluation Agentic Workflows
School of Software Engineering, Dalian University, Dalian, China · Department of Surgery, Affiliated Zhongshan Hospital of Dalian University, Dalian, China · The Chinese University of Hong Kong, Shenzhen, China +3
JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agents
Jul 26, 2026Yunlong Lin, Zixu Lin, Zhaohu Xing +23Multimodal Generation Multimodal Agents
ETPDesigner: Multi-Agent Orchestration for Interactive Multimodal Electronic Theater Program
Jul 22, 2026Mengtian Li, Xinru Guo, Xiaoru Lin +3Multi-Agent Orchestration Compositional T2I Generation
Shanghai University Shanghai Engineering Research Center of Motion Picture Special Effects Shanghai, China · Institute for Math and AI, Wuhan School of Artificial Intelligence, Wuhan University Wuhan, Hubei, China
MAGE: Human-Like Macro Placement via Agentic Multimodal Reasoning
Jul 20, 2026Andrew B. Kahng, Sayak Kundu, Bodhisatta PramanikElectronic Design Automation Multimodal Agents
Sparse Evidence Can Suffice: Agentic Evidence Seeking for Multimodal Video Misinformation Detection
Jul 20, 2026Haochen Zhao, Yongxiu Xu, Xinkui Lin +6Automated Fact-Checking Multimodal Agents
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Baidu, Beijing, China
SciForge: An AI-Native, Multimodal Workbench for Scientific Discovery
Jul 17, 2026SciForge Team, Zhangyang Gao, Minghao Fang +10AI Agents for Scientific Discovery AI Agent Governance
SciForge Team, Shanghai Artificial Intelligence Laboratory
Do Agents Dream of False Memories? Black-box Visual Attacks on Long-term Memory in Multimodal AI Agents
Jul 17, 2026Halima Bouzidi, Mboutidem Ekemini Mkpong, Mohammad Abdullah Al FaruqueMultimodal Memory Agent Memory Poisoning
University of California, Irvine
An Intelligent-Cloud Edge Multimodal Interaction System for Robots
Jul 16, 2026Zihan Guo, Xiaoqi LiRobotics Gesture Recognition
Hainan University, China
Unleashing Multimodal Large Language Models for Training-free HOI Detection in the Wild
Jul 15, 2026Ting Lei, Jialin Liu, Zhu Xu +2Multimodal Large Language Models Multimodal Agents
Wangxuan Institute of Computer Technology, Peking University, Beijing, China
SPyCE: Skill-Policy Co-evolution for Multimodal Agents
Jul 15, 2026Ru Zhang, Weijie QiuMultimodal Reasoning Agent Skill Learning
Zhejiang University, Hangzhou, China · Beijing University of Posts and Telecommunications, Beijing, China
Multi-Agent Collaborative Reasoning with Tool-Augmented Evidence for Urban Region Profiling
Jul 15, 2026Xixuan Hao, Yutian Jiang, Jiabo Liu +4Multi-Agent Reasoning Multimodal Agents
The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, China · University of Washington Seattle, USA · Chang’an University Xi’an, China +1
LightMem-Ego: Your AI Memory for Everyday Life
Jul 13, 2026Yijun Chen, Boyi Xiao, Yixian Zhao +10Multimodal Memory Episodic Memory
1Zhejiang University · 2South China University of Technology · 3Central China Normal University +1
Omni-Decision: Evidence-Ledger Planning for Omni-Modal Agents
Jul 13, 2026Ming Ma, Yi Zhu, Yiran Zhong +6Multimodal Agents Multimodal Tool Use
Institute of Neuroscience, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Tongyi Lab, Alibaba Group +2
A Glimpse into Long-term Physical Coexistence with Intelligent Robots
Jul 13, 2026Weiqi Jin, Peijun Tang, Kuncheng Luo +5Multi-Robot Systems Assistive Robotics
Astribot Team
UNIBROWSE: A Data-to-Agent Framework for Multimodal BrowseComp
Jul 12, 2026Xiyu Wei, Qingwei Zong, Zhuocheng Yu +1Multimodal QA AI Agent Benchmarks
Key Laboratory of Computational Linguistics, MOE, Peking University · School of Software and Microelectronics, Peking University · School of Computer Science, Peking University
Task-Specific Multimodal Question Answering Agents via Confidence Calibration and Incremental Reasoning for QANTA 2026
Jul 10, 2026Nirjhar Das, Md. Al-Mamun ProvathQuestion Answering Multimodal QA
Department of Computer Science and Engineering, Chittagong University of Engineering & Technology, Chattogram, Bangladesh.
SAGEAgent: A Self-Evolving Agent for Cost-Aware Modality Acquisition in Multimodal Survival Prediction
Jul 10, 2026Chongyu Qu, Can Cui, Zhengyi Lu +8Cancer Survival Prediction Clinical Decision Support
Vanderbilt University, Nashville TN 37235, USA · Vanderbilt University Medical Center, Nashville TN, 37232 USA
UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks
Jul 9, 2026Zhekai Chen, Chengqi Duan, Kaiyue Sun +4AI Agent Benchmarks Multimodal Agents
HKU MMLab · Meituan
Cognitive-structured Multimodal Agent for Multimodal Understanding, Generation, and Editing
Jul 9, 2026Feng Wang, Canmiao Fu, Zhipeng Huang +3Multimodal Memory Episodic Memory
Peking University · WeChat Vision, Tencent Inc.
VaseMuseum: Digital Intelligent Museum for Ancient Greek Pottery
Jul 7, 2026Jiazi Wang, Nonghai Zhang, Qiushi Xie +5Multimodal RAG Cultural Heritage
School of Computer Science and Technology, Beijing Jiaotong University, Beijing 100044, China · School of Computer Science, Peking University, Beijing 100871, China · Department of Computer Science and Information Technology, La Trobe University, Melbourne VIC 3000, Australia +1
SearchEyes: Towards Frontier Multimodal Deep Search Intelligence via Search World Simulation
Jul 7, 2026Zhengbo Jiao, Yiming Cheng, Yilei Jiang +15Multi-Hop QA Multimodal Reasoning
1MMLab, CUHK · 2THU · 3HKU +6
Light-Omni: Reflex over Reasoning in Agentic Video Understanding with Long-Term Memory
Jul 6, 2026Chang Nie, Jiaju Wei, Junlan Feng +2Multimodal Agents Memory-Augmented Agents
Nanjing University
ReflectWorld-MM: An Entity-Oriented Multimodal Memory System for Open-Ended Video Streams
Jul 6, 2026Xiaokang Ma, Yifan Sun, Zhihong Jin +6Streaming Video Understanding Multimodal Memory
1Rightly Robotics · 2Hangzhou Institute for Advanced Study, UCAS · 3Zhejiang University
CogRad: A Cognitively-Inspired Multi-Agent Framework for Radiology Report Generation
Jul 4, 2026Saif Ur Rehman Khan, Hasaan Maqsood, Sebastian Vollmer +2Radiology Report Generation Medical VLMs
Department of Computer Science, Rhineland-Palatinate Technical University of Kaiserslautern-Landau, Kaiserslautern, 67663, Germany. · 2German Research Center for Artificial Intelligence, Kaiserslautern, 67663, Germany. · 3Intelligentx GmbH (intelligentx.com), Kaiserslautern, Germany. +1
BOUNDARY_SYNC: Measuring Communication-Induced Representational Coupling in Multi-Agent LLM Systems
Jul 2, 2026Zewen LiuMulti-Agent LLM Systems Multimodal Agents
Qilu Institute of Technology, Tai’an, Shandong, China
MECoBench: A Systematic Study of Multimodal Agent Collaboration in Embodied Environments
Jun 30, 2026Qingyun Liu, Jiwen Zhang, Jingyi Hu +2Multi-Agent Collaboration AI Agent Benchmarks
1Fudan University · 3The Chinese University of Hong Kong · 2Shanghai Innovation Institute
Xiaomi-GUI-0 Technical Report
Jun 30, 2026Wanxia Cao, Chengzhen Duan, Pei Fu +29Mobile GUI Automation RL for GUI Agents
ManimAgent: Self-Evolving Multimodal Agents for Visual Education
Jun 29, 2026Wenjia Jiang, Zongyuan Cai, Yuanhang Shao +7Continual Learning for LLM Agents LLM Agent Self-Improvement
University of Alberta · Southeast University · Virginia Tech +2

Jul 29, 2026J/K move · Enter open · S save

CaM-Wolf: Causal-Aware Multimodal Agents for Social Deduction Games

Zheng Zhang, Nanjie Yao, Jiarui He, Deheng Ye+2

The Hong Kong University of Science and Technology (Guangzhou) · Nanyang Technological University · Shanghai Jiao Tong University

Game-Playing Agents Social Deduction Games LLM Agents Multimodal Agents

Open paper AskPDF arXiv

Social deduction games (SDGs) such as Werewolf have become challenging testbeds for AI agents. These games require complex social skills such as reasoning, deception, and collaboration. While recent advances in large language models (LLMs) have driven significant progress in SDG agents, current approaches are predominantly text-based, overlooking the multimodal nature that is fundamental to human social interaction. To bridge this gap, we introduce CaM-Wolf, the first SDG agent that integrates multimodal perception and generation. CaM-Wolf processes video inputs from other players, employs a causal-aware Reasoner trained via reinforcement learning to establish logical chains between observable behaviors and hidden roles, and presents itself through an animated avatar. Our experiments and user study show that CaM-Wolf achieves superior agent gameplay performance and enhances the quality of human-AI interaction. This work represents a significant advancement towards creating more human-like AI agents capable of participating in nuanced social dynamics. Our code is available at https://3dagentworld.github.io/avatar_wolf.