Multimodal Agent Evaluation

Latest papers

CardsList

42 papers24 papers in the last four weeks, against 1 the four weeks before. 0.2% of all new papers.

  1. WebArena-Pro: A Heterogeneous, Multimodal, Reproducible Benchmark for Web Agents

    Nov 13, 2026Imene Kerboua, Fatemeh Pesaran Zadeh, Xing Han Lù +19Web Agent BenchmarksWeb Agents

  2. SituRecBench: Evaluating Frontier Models as Situated Recommendation Assistants in 3D Environments

    Nov 7, 2026Jingtong Yang, Dongding Lin, Jian Wang +1Recommender SystemsMultimodal Recommendation

  3. ABHBench: Evaluating Moral Decision-Making of Foundation Models from an Agentic Perspective

    Nov 6, 2026Chaoran Li, Yukun Li, Zeyuan Zhao +3Moral Reasoning in Language ModelsAI Agent Safety Benchmarks

  4. EarthRecord: From Pixels to Land Change with Multimodal Agents over Four Decades of Satellite Imagery

    Nov 6, 2026Haoran Zhang, Huizhen Sun, Hanbin Wen +3Remote Sensing BenchmarksGeospatial Reasoning Benchmarks

  5. Interactive Visual Gym: Evaluating Visual Interaction Models with Simulated Users in Interactive Worlds

    Nov 5, 2026Chenhao Zheng, Zixian Ma, Kai Li +11VLM EvaluationMultimodal Agent Evaluation

  6. Multimodal-SEA: Benchmarking AI Agent Security under Multimodal Social Engineering Attacks

    Nov 5, 2026Zhitao He, Chi Him Zhang, Chengru Han +1AI Agent SecurityAI Agent Security Benchmarks

  7. ModalSearcher: Weaving Visual Evidence into Autonomous Cross-Modal Deep Search

    Nov 4, 2026Zhihan Yin, Ruyi Li, Chengyu Fang +6Multimodal Search AgentsMultimodal QA

  8. VisSearch-Bench: Evaluating Long Horizon Visual Search in Multimodal Agents

    Nov 3, 2026ZhengXian Wu, Qianhao Nie, Zhuohong Chen +10Multimodal Search AgentsVisual Search

  9. DualWorldBench: Can Agents Plan Deliveries across Symbolic and Grounded Worlds?

    Nov 3, 2026Yang Chen, Bin Wen, Danyang Peng +3Temporal PlanningEmbodied Agent Benchmarks

  10. Omni-Life: Benchmarking Omnimodal Real-Time AI Assistants in People's Daily Lives

    Nov 2, 2026Mengze Hong, Ziwei Shang, Di Liang +9Multimodal Agent EvaluationMultimodal Agents

  11. MAGAR: Multimodal Agents for GUI-Based 3D Mechanical Assembly and Minimal Rework

    Nov 2, 2026Jingxuan Wei, Jiahao Fu, Fuqiang Wang +5Multimodal AgentsAI Agent Reliability

  12. Efficient Agentic Audio-Visual Question Answering through Toolset Pruning and Modality Consolidation

    Oct 28, 2026Yi Wang, Tianzi Wang, Peter BellAudio-Visual QAMultimodal Agents

  13. Gym-Anything-RT: Scaling Real-Time Environments for Vision-Language Agents

    Oct 27, 2026Pranjal Aggarwal, Sean WelleckAI Agent BenchmarksAgentic AI

  14. CogArena: Benchmarking Multimodal Agents on Interactive Behavioral Experiments

    Oct 27, 2026Kianté Fernandez, Caitlin Chen, Jialu Zhou +3Multimodal Agent EvaluationHuman Behavior Simulation

  15. Profiling a Frontier Agent with EBench: From Direct Control to Harness with Policy

    Oct 26, 2026Yifei Yao, Yude Zou, Guanbo Jiao +12Language Model-Based ControlRobotic Policy Evaluation

  16. Omni-InteractScope: Benchmarking Proactive and Agentic Interaction in Continuous Omnimodal Streams

    Oct 26, 2026Yinsong Yan, Ling Wang, Yuxuan Wang +11Multimodal Agent EvaluationMLLM Evaluation

  17. Co-Design Bench: Multimodal Collaborative Coding Evaluation for Agentic Models

    Oct 26, 2026Xilin Wei, Shengyuan Ding, Xinyu Fang +13Coding Agent BenchmarksMultimodal Agent Evaluation

  18. Multimodal Socially Aware Framework for Human-Robot Interaction in Virtual Reality

    Oct 25, 2026Richard Xu, Dakarai Crowder, Alexis E. Block +1Human-Robot InteractionSocial Robotics

  19. AndroidFlux: Evaluating and Reward Modeling Failure Recovery Capabilities in Mobile Use Multimodal Agents

    Oct 25, 2026Gyubeum Lim, Yemo Koo, Ghunwoo Jang +1Multimodal Agent EvaluationReward Model Evaluation

  20. OmniStepBench: Deployment-Aware Agent Evaluation across Modalities and Task Complexity

    Sep 5, 2026Yu Liu, Zhilin Liu, Zhiwei Yang +7Multimodal Agent EvaluationLLM Evaluation

  21. Paint What You See: Benchmarking Dexterous Visual Tool Use in Multimodal Agents

    Aug 26, 2026Shudong Liu, Dongyang Chen, Enci Zhang +3AI Agent BenchmarksTool Use

  22. M3^3Exam: Benchmarking Multimodal Memory for Realistic User-Agent Interactions

    Jun 5, 2026Zhengjun Huang, Wenxuan Liu, Zhoujin Tian +6Multimodal UnderstandingMultimodal Grounding

  23. WorldMemArena: Evaluating Multimodal Agent Memory Through Action-World Interaction

    May 28, 2026Chengzhi Liu, Yuzhe Yang, Sophia Xiao Pu +14Multimodal Agent EvaluationMultimodal Memory