LLM Agent Reliability

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

90 papers in the last four weeks, up 109% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 558

All topics

From Accuracy to Auditability: A Survey of Determinism in Financial AI Systems
May 11, 2026Ruizhe Zhou, Xiaoyang Liu, Gaoyuan Du +4Algorithmic Auditing ML Reproducibility
Amazon.com USA
Engineering Robustness into Personal Agents with the AI Workflow Store
May 11, 2026Roxana Geambasu, Mariana Raykova, Pierre Tholoniat +3Agentic Workflows LLM Agent Security
Columbia University and Google · Google
ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox
May 11, 2026Yuanyang Li, Xue Yang, Longyue Wang +2LLM Agent Evaluation AI Agent Benchmarks
1Zhejiang University · 2Zhejiang Lab · 3Alibaba Group
The Agent Use of Agent Beings: Agent Cybernetics Is the Missing Science of Foundation Agents
May 11, 2026Xinrun Wang, Chang Yang, He Zhao +2Self-Improving Agents Long-Horizon LLM Agents
Singapore Management University · The Hong Kong Polytechnic University · Nanyang Technological University +2
Agent-First Tool API: A Semantic Interface Paradigm for Enterprise AI Agent Systems
May 11, 2026Kai PanTool-Using Agents AI Agent Governance
A2A Lab
Beyond Autonomy: A Dynamic Tiered AgentRunner Framework for Governable and Resilient Enterprise AI Execution
May 11, 2026Kai Pan, Rong HouLLM Agent Orchestration AI Agent Safety
A2A Lab
Instruction Adherence in Coding Agent Configuration Files: A Factorial Study of Four File-Structure Variables
May 11, 2026Damon McMillanCoding Agents LLM Agent Evaluation
HxAI Australia
Nautilus Compass: Black-box Persona Drift Detection for Production LLM Agents
May 11, 2026Chunxiao WangAgent Memory AI Agent Reliability
Yiluo Technology Co., Ltd.
RubricRefine: Improving Tool-Use Agent Reliability with Training-Free Pre-Execution Refinement
May 10, 2026Will LeVine, Brendan Evers, Sam Saltwick +1LLM Self-Refinement Rubric-Based Evaluation
Anduril Industries
The Trap of Trajectory: Towards Understanding and Mitigating Spurious Correlations in Agentic Memory
May 10, 2026Luoxi Tang, Rupali Rajendra Vaje, Yuqiao Meng +5AI Agent Benchmarks LLM Agent Reliability
Binghamton University, State University of New York, Binghamton, NY, USA · Oakland University, Rochester, MI, USA · University of Pennsylvania, Philadelphia, PA, USA
A Prompt-Aware Structuring Framework for Reliable Reuse of AI-Generated Content in the Agentic Web
May 10, 2026Shusaku Egami, Masahiro HamasakiAI Agent Safety LLM Agent Reliability
National Institute of Advanced Industrial Sciencen and Technology (AIST) · Koto, Tokyo, Japan
CIVeX: Causal Intervention Verification for Language Agents
May 9, 2026Fabio RovaiStructural Causal Models Tool-Using Agents
The Tesseract Academy
A Communication-Theoretic Framework for LLM Agents: Cost-Aware Adaptive Reliability
May 9, 2026Hamed Omidvar, Vahideh AkhlaghiLLM Inference Efficiency AI Agent Reliability
INTELLERCE LLC San Jose, CA, USA
Trustworthy AI: Ensuring Reliability and Accountability from Models to Agents
May 9, 2026Carol Xuan LongMulti-Agent LLM Systems AI Accountability
John A. Paulson School Of Engineering And Applied Sciences, Harvard University, Cambridge, Massachusetts
When Agents Overtrust Environmental Evidence: An Extensible Agentic Framework for Benchmarking Evidence-Grounding Defects in LLM Agents
May 9, 2026Strick Sheng, Ziyue Wang, Liyi ZhouAI Agent Reliability LLM Agent Evaluation
The University of Sydney · Nanjing University
Debugging the Debuggers: Failure-Anchored Structured Recovery for Software Engineering Agents
May 9, 2026Chenyu Zhao, Shenglin Zhang, Yihang Lin +7Agent Failure Analysis Software Engineering Agents
Nankai University Tianjin, China · Tsinghua University Beijing, China · Microsoft Redmond, USA
AgentCollabBench: Diagnosing When Good Agents Make Bad Collaborators
May 9, 2026Aritra Mazumder, Shubhashis Roy Dipta, Nusrat Jahan Lia +10AI Agent Reliability Multi-Agent Collaboration
University of Utah · University of Maryland, Baltimore County · University of Dhaka +7
Why Retrying Fails: Context Contamination in LLM Agent Pipelines
May 8, 2026Zhanfu YangLLM Context Management LLM Agent Reliability
Department of Computer Science, Rutgers University, Piscataway, NJ 08854, USA
Business Utility of Large Language Models as Exploratory Data Analysis Agents
May 8, 2026Rafał Łabędzki, Patryk Miziuła, Hubert Rutkowski +5LLM Agent Evaluation AI Agent Benchmarks
SGH Warsaw School of Economics · Bydgoszcz University of Science and Technology · At the time of contribution aﬀiliated with deepsense.ai, present aﬀiliation is Google
Can Agents Price a Reaction? Evaluating LLMs on Chemical Cost Reasoning
May 8, 2026Yuyang Wu, Yue Huang, Shuaike Shen +8AI Agents for Scientific Discovery LLM Agent Evaluation
Carnegie Mellon University · University of Notre Dame · University of North Carolina, Chapel Hill +1
EnvSimBench: A Benchmark for Evaluating and Improving LLM-Based Environment Simulation
May 8, 2026Yi Liu, TingFeng Hui, Wei Zhang +4AI Agent Evaluation AI Agent Benchmarks
Beijing University of Posts and Telecommunications · The Hong Kong University of Science and Technology · Chongqing University
MedExAgent: Training LLM Agents to Ask, Examine, and Diagnose in Noisy Clinical Environments
May 8, 2026Yicheng Gao, Xiaolin Zhou, Yahan Li +2Healthcare Medical Diagnosis
University of Southern California · Arizona State University
PrefixGuard: Online Failure Warning and Trace-Grounded Diagnosis for LLM Agents
May 7, 2026Xinmiao Huang, Jinwei Hu, Qisong He +4Fault Detection AI Agent Monitoring
University of Liverpool · Université Grenoble Alpes
A Self-Healing Framework for Reliable LLM-Based Autonomous Agents
May 7, 2026Cheonsu Jeong, Younggun ShinSoftware Engineering Agents Fault Detection
AX Center, SAMSUNG SDS, Seoul, South Korea · Dept. of Artificial Intelligence, Graduate School of Engineering, Yonsei University, Seoul, South Korea
TACT: Mitigating Overthinking and Overacting in Coding Agents via Activation Steering
May 7, 2026Yuan Sui, Yulin Chen, Yibo Li +6AI Coding Agents Long-Horizon LLM Agents
National University of Singapore · Peking University · Meta
Say the Mission, Execute the Swarm: Agent-Enhanced LLM Reasoning in the Web-of-Drones
May 5, 2026Andrea Iannoli, Lorenzo Gigli, Luca Sciullo +2Aerial Robotics Multi-Robot Systems
Department of Computer Science and Engineering University of Bologna Bologna, Italy
Robust Agent Compensation (RAC): Teaching AI Agents to Compensate
May 5, 2026Srinath Perera, Kaviru Hapuarachchi, Frank Leymann +1AI Agent Reliability LLM Agent Reliability
WSO2 Santa Clara, USA · University of Stuttgart Stuttgart, Germany
What Happens Inside Agent Memory? Circuit Analysis from Emergence to Diagnosis
May 5, 2026Xutao Mao, Jinman Zhao, Gerald Penn +1Agent Memory Mechanistic Interpretability
City University of Hong Kong · University of Toronto
ARIS: Autonomous Research via Adversarial Multi-Agent Collaboration
May 4, 2026Ruofeng Yang, Yongcan Li, Shuai LiAI Agent Reliability Multi-Agent LLM Systems
Shanghai Jiao Tong University · Shanghai Innovation Institute
Foundation-Model-Based Agents in Industrial Automation: Purposes, Capabilities, and Open Challenges
May 4, 2026Vincent Henkel, Felix Gehlhoff, David Kube +13AI Agent Evaluation LLM Agent Reliability
Institute of Automation Technology, Helmut Schmidt University / University of the Federal Armed Forces Hamburg, Hamburg, Germany. · Siemens AG, Nuremberg, Germany; Institute for Technologies and Management of Digital Transformation, Bergische Universität Wuppertal, Germany. · Artiquare GmbH, Ingolstadt, Germany. +11

May 11, 2026, cs.AIJ/K move · Enter open · S save

From Accuracy to Auditability: A Survey of Determinism in Financial AI Systems

Ruizhe Zhou, Xiaoyang Liu, Gaoyuan Du, Yi Zheng+3

Amazon.com USA

Algorithmic Auditing ML Reproducibility Finance LLM Agent Reliability

Open paper AskPDF arXiv

Deploying machine learning in regulated financial environments -- credit risk, fraud detection, and anti-money laundering -- exposes critical vulnerabilities in algorithmic reproducibility. While early financial ML addressed statistical challenges such as backtest overfitting, deep neural networks and Generative AI have introduced mechanical nondeterminism rooted in hardware and architecture. This survey provides a systems perspective on reproducibility failures across three modalities now dominant in financial AI: tabular models (post-hoc explanation variance), graph networks (stochastic sampling and temporal asynchrony), and LLM-based agentic workflows (batch-dependent divergence and trajectory drift). We supplement the literature analysis with first-party experiments on public financial datasets -- quantifying explanation rank instability in credit scoring, prediction flip rates in GNN-based fraud detection, and tensor-parallel-induced output divergence in LLM entity extraction. We propose a layered evaluation framework linking modality-specific metrics (RBO, D_cos, TDI, PSD) to audit readiness, and empirically validate the complementarity of logit-level and semantic-level determinism measures.

Momentum

Related topics

Latest papers 558