LLM Agent Reliability

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

90 papers in the last four weeks, up 109% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 558

All topics

VeriTrip: A Verifiable Benchmark for Travel Planning Agents over Unstructured Web Corpora
May 27, 2026Yuting Xu, Jiayi Tian, Jian Liang +4Multimodal Reasoning AI Agent Benchmarks
Institute of Information Engineering, CAS · School of Cyber Security, UCAS · Amap, Alibaba Group +2
Mobile-Aptus: Confidence-Driven Proactive and Robust Interaction in MLLM-based Mobile-Using Agents
May 27, 2026Zheng Wu, Pengzhou Cheng, Zongru Wu +5Probability Calibration GUI Agents
School of Computer Science, Shanghai Jiao Tong University · GenAI, Meta
OR-Space: A Full-Lifecycle Workspace Benchmark for Industrial Optimization Agents
May 27, 2026Chenyu Zhou, Xinyun Lu, Jiangyue Zhao +3LLM Agent Evaluation AI Agent Benchmarks
Shanghai Jiao Tong University · Shanghai University of Finance and Economics · Stanford University
Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows
May 27, 2026Yilun Yao, Xinyu Tan, Chao-Hsuan Liu +9Computer-Use Agent Benchmarks AI Agent Reliability
1Peking University · 2Qiyuan Tech
Diagnosing Live Within-Policy Instruction Conflicts in LLM Agents with Witnessed Resolution Profiles
May 27, 2026Lu Yan, Xuan Chen, Xiangyu ZhangLLM Agent Evaluation LLM Agents
Purdue University
Detection Without Correction: A Two-Parameter Decomposition of Multi-Stage LLM Pipelines
May 26, 2026Prashanti Nilayam, Kiran Ramanna, Prashil TumbadeLLM Self-Correction LLM Evaluation
Servicenow CA, USA
Learning to Act under Noise: Enhancing Agent Robustness via Noisy Environments
May 26, 2026Yuxin Chen, Xiaodong Cai, Junfeng Fang +9LLM Agents LLM Agent Training
National University of Singapore · Meituan · Tsinghua University +2
It's Not the Capability: Harness Sensitivity Is Non-Monotone Across LLM Agent Tiers
May 26, 2026Yong-eun ChoAI Agent Reliability LLM Agent Evaluation
KailosLab · Seoul, Republic of Korea
Your Agents Are Aging Too: Agent Lifespan Engineering for Deployed Systems
May 25, 2026Jianing Zhu, Yeonju Ro, John Robertson +5AI Agent Reliability AI Agent Benchmarks
The University of Texas at Austin
VeriTrace: Evolving Mental Models for Deep Research Agents
May 25, 2026Haolang Zhao, Yunbo Long, Lukas Beckenbauer +1Cognitive Modeling Deep Research Agents
Department of Engineering, University of Cambridge · TUM School of Management, Technical University of Munich · The Alan Turing Institute
When Do LLM Agents Treat Surface Noise Differently from Semantic Noise? A 68-Cell Measurement Study with a Held-Out Trace-Level Validation
May 25, 2026Liyun Zhang, Jiayi GuoCoT Reasoning LLM Agent Evaluation
School of Information and Software Engineering, UESTC · Jacobs School of Engineering, UC San Diego
Meta-Engineering Harnesses for AI-Native Software Production: A Contract-Driven Adversarial Verification Architecture with Early Deployment Report
May 25, 2026Satadru Sengupta, Tamunokorite Briggs, Ivan MyshakivskyiSoftware Engineering Agents Automated Software Engineering
HireNimbus
A Universal Cliff and a Design Fingerprint: Cross-Section Defect Detection Under LLM Orchestration
May 25, 2026Hiroki FukuiMulti-Agent LLM Systems LLM Evaluation
Research Institute of Criminal Psychiatry · Department of Neuropsychiatry, Graduate School of Medicine, Kyoto University
CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures
May 25, 2026Akash Bonagiri, Devang Borkar, Gerard Janno Anderias +2Agent Failure Analysis Causal Interventions in Language Models
Department of Computer Science University of California, Davis Davis, CA 95616
Meta-Agent: From Task Descriptions to Verified Multi-Agent Systems
May 24, 2026Andy Xu, Yu-Wing TaiMulti-Agent Orchestration Multi-Agent Systems
Dartmouth College
Memory-Induced Tool-Drift in LLM Agents
May 24, 2026Mahavir Dabas, Jihyun Jeong, Ming Jin +1AI Agent Benchmarks LLM Tool Use
Virginia Tech
PRIMA: Operational Patterns for Resilient Multi-Agent Research with Verifiable Identity and Convergent Feedback
May 23, 2026Sasank AnnapureddyMulti-Agent Coordination Multi-Agent Orchestration
CP-Agent: A Calibrated Risk-Controlled Agent for Feedback-Driven Competitive Programming
May 23, 2026Peisong Wang, Bowen Liu, Zehua Li +4Code Generation LLM-Based Program Synthesis
The Hong Kong University of Science and Technology (Guangzhou) Guangzhou, China
Push Your Agent: Measuring and Enforcing Quantitative Goal Persistence in Long-Horizon LLM Agents
May 22, 2026Yuandao Cai, Yuzhang Zhu, Liyou Gao +2Long-Horizon Planning Long-Horizon Agent Evaluation
Independent Researcher · Xidian University
When Planning Fails Despite Correct Execution: On Epistemic Calibration for LLM-Based Multi-Agent Systems
May 22, 2026Zehao Wang, Shilong Jin, Zhao Cao +1Multi-Agent LLM Systems Multi-Agent Planning
College of Intelligence and Computing, Tianjin University, Tianjin, China · School of New Media and Communication, Tianjin University, Tianjin, China · Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China
DART: Semantic Recoverability for Structured Tool Agents
May 22, 2026Ke Yang, Panpan Li, Zonghan Wu +3Agent Reliability Runtime Assurance
1MOS Intelligent Connectivity Technology Co. Ltd. · 2Sichuan Vocational College of Post and Telecom · 3East China Normal University +2
More Skills, Worse Agents? Skill Shadowing Degrades Performance When Expanding Skill Libraries
May 21, 2026Hongwen Song, Song WeiLLM Agent Evaluation LLM Agent Skill Learning
Databricks Inc.
Evaluating Large Language Models as Live Strategic Agents: Provider Performance, Hybrid Decomposition, and Operational Gaps in Timed Risk Play
May 21, 2026H. C. EkneLLM Agent Evaluation AI Agent Benchmarks
Diagnosis Is Not Prescription: Linguistic Co-Adaptation Explains Patching Hazards in LLM Pipelines
May 21, 2026Yoon Jeonghun, Kim DongchanLLM Agents Causal Interventions in Language Models
KAIST (Korea Advanced Institute of Science and Technology), Republic of Korea · NAVER Corp., Seongnam, Republic of Korea · NAVER Corp., Bellevue, WA, USA
Agent JIT Compilation for Latency-Optimizing Web Agent Planning and Scheduling
May 20, 2026Caleb Winston, Ron Yifeng Wang, Azalia Mirhoseini +1Computer-Use Agents LLM Agent Orchestration
Stanford University, Stanford, CA, USA.
Insights Generator: Systematic Corpus-Level Trace Diagnostics for LLM Agents
May 20, 2026Akshay Manglik, Vijay S. Kalmath, Jason Qin +6Multi-Agent LLM Systems LLM Agent Evaluation
Scale AI
What Do Agents Communicate? Characterizing Information Exchange in Multi-Agent Systems
May 19, 2026Yong Jin Chun, Iftekhar AhmedMulti-Agent Collaboration Multi-Agent Systems
Department of Informatics University of California, Irvine Irvine, CA, 92618
AgentAtlas: Beyond Outcome Leaderboards for LLM Agents
May 19, 2026Parsa Mazaheri, Kasra MazaheriAgent Failure Analysis Computer-Use Agent Benchmarks
University of California, Santa Cruz · Massachusetts Institute of Technology
Modeling Emotional Dynamics in Agent-to-Agent Interactions on Moltbook
May 19, 2026Syed Mhamudul Hasan, Abdur R. ShahidMulti-Agent LLM Systems LLM Agent Evaluation
Southern Illinois University Carbondale, IL, USA
A Methodology for Selecting and Composing Runtime Architecture Patterns for Production LLM Agents
May 19, 2026Vasundra SrinivasanLLM Agent Orchestration LLM Agents
AI Architect, Independent Researcher · Stanford School of Engineering

May 27, 2026, cs.AIJ/K move · Enter open · S save

VeriTrip: A Verifiable Benchmark for Travel Planning Agents over Unstructured Web Corpora

Yuting Xu, Jiayi Tian, Jian Liang, Xin Xiong+3

Institute of Information Engineering, CAS · School of Cyber Security, UCAS · Amap, Alibaba Group+2

Multimodal Reasoning AI Agent Benchmarks Multimodal Agents LLM Agent Reliability

Open paper AskPDF arXiv

Existing benchmarks have laid the foundation for travel planning agents by establishing API-centric paradigms. However, as the capabilities of Autonomous Agents continue to advance, their evaluation must evolve beyond simple tool execution toward handling the inherent complexities of the open web. Current benchmarks bypass core cognitive hurdles: they fail to account for information noise, ignore multi-source factual contradictions, and overlook the necessity of grounding visual perception into logical planning. We introduce VeriTrip, a verifiable benchmark designed to meet the increasing demands for agent robustness and reliability. VeriTrip shifts the evaluation focus to evidence-grounded reasoning over unstructured multimodal web corpora. It establishes a Multimodal Retrieval Base (MRB) derived from real-world sources, forcing agents to autonomously orchestrate queries across heterogeneous data. A synchronized Verifiable Knowledge Base (VKB) enables a cell-wise verification protocol that precisely quantifies factual reliability, distinguishing systematic reasoning failures from parametric hallucinations. Our evaluations across leading MLLMs reveal a critical \textit{retrieval-reasoning trade-off}: the cognitive load of autonomous retrieval significantly erodes instruction retention. VeriTrip provides the rigorous foundation necessary for the next generation of planning agents capable of operating in unconstrained, multimodal environments.

Momentum

Related topics

Latest papers 558