AI Agent Monitoring

Follow this topic Ask about this topic

Momentum

20 papers in the last four weeks, up 186% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 170

All topics

Strained Coherence: A Pre-Failure Signal in Coding Agent Execution Trajectories
Jun 5, 2026Marut Pandya, Kasey Zhang, Baiqing LyuAI Coding Agents AI Agent Monitoring
TRACE: Trajectory Reasoning through Adaptive Cross-Step Evidence Aggregation for LLM Agents
Jun 5, 2026Vijitha Mittapalli, Shreyaa Jayant Dani, Satya Srujana Pilli +7Long-Horizon Agent Evaluation AI Agent Security
University of Massachusetts at Amherst · Adobe Research · Dolby Labs +2
From Reward-Hack Activations to Agentic Risk States: Context-Calibrated Mechanistic Monitoring in LLM Agents
Jun 4, 2026Patrick Wilhelm, Odej KaoReward Hacking AI Agent Safety
Distributed Operating Systems, Technische Universität Berlin, Berlin, Germany · BIFOLD - Berlin Institute for the Foundations of Learning and Data, Berlin, Germany
Entropy-Based Observability for AI Agent Behavior
Jun 4, 2026Olasimbo Ayodeji ArigbabuAI Agent Monitoring
Coding with "Enemy": Can Human Developers Detect AI Agent Sabotage?
Jun 4, 2026Jingheng Ye, Huiqi Zou, Simon Yu +1AI Coding Agents AI Agent Monitoring
Northeastern University
From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents
Jun 3, 2026Yiqi Wang, Jiaqi Zhang, Taotao Cai +8AI Agent Reliability Data Provenance
Griffith University · Jiangsu University · University of Southern Queensland +5
The Saturation Trap and the Subjectivity of Intervention Timing: Why Affect-Based Triggers and LLM Judges Fail to Time Interventions on Autonomous Agents
Jun 2, 2026Manvendra ModgilLLM-as-a-Judge Human-in-the-Loop Evaluation
Notarized Agents: Receiver-Attested Confidential Receipts for AI Agent Actions
Jun 2, 2026Juan FigueraData Provenance AI Agent Security
Independent Researcher, Sello Project
Tracking the Behavioral Trajectories of Adapting Agents
Jun 1, 2026Jonah Leshin, Manish Shah, Ian TimmisAgent Evaluation AI Agent Monitoring
Project VAIL, San Francisco, CA, USA
Monitoring Agentic Systems Before They're Reliable
Jun 1, 2026Marisa Ferrara Boston, Glen Hanson, Effi Georgala +2Agent Failure Analysis Agent Reliability
Reins AI USA · Veraitech USA
POIROT: Interrogating Agents for Failure Detection in Multi-Agent Systems
Jun 1, 2026Iñaki Dellibarda Varela, R. Sendra-Arranz, Pablo Romero-Sorozabal +5Agent Failure Analysis AI Agent Reliability
Center for Automation and Robotics, Spanish National Research Council (CSIC-UPM), Madrid, Spain. · ETSI Telecomunicación, Universidad Politécnica de Madrid (UPM), Madrid, Spain.
Agent System Operations: Categorization, Challenges, and Future Directions
Jun 1, 2026Zexin Wang, Changhua Pei, Yuanhao Liu +10Agent Failure Analysis AI Agent Monitoring
CNIC, CAS · Hangzhou Institute for Advanced Study, UCAS · Singapore Management University +1
From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents
Jun 1, 2026Laksh AdvaniAgent Failure Analysis AI Agent Reliability
University of Colorado
Agent Operating Systems (AOS): Integrating Agentic Control Planes into, and Beyond, Traditional Operating Systems
Jun 1, 2026Ankur Sharma, Deep ShahAI Agent Security AI Agent Monitoring
Independent Researcher San Francisco, USA
Early Diagnosis of Wasted Computation in Multi-Agent LLM Systems via Failure-Aware Observability
May 31, 2026Xianyou Li, Weiran Yan, Yichao Wu +4LLM Inference Efficiency Multi-Agent LLM Systems
New York University · New York, NY 10012, USA · Independent Researcher +7
BraveGuard: From Open-World Threats to Safer Computer-Use Agents
May 31, 2026Yunhao Feng, Xiaohu Du, Xinhao Deng +13LLM Guardrails AI Agent Safety
1Fudan University · Ant Group · 8ShanghaiUniversity +5
Stateful Online Monitoring Catches Distributed Agent Attacks
May 29, 2026Davis Brown, Samarth Bhargav, Arav Santhanam +7LLM Agent Security AI Agent Monitoring
University of Pennsylvania · 2Carnegie Mellon University
Syll: Open-Source Personal Automation with Cross-Surface Execution
May 28, 2026Bo Zhang, Borui Zhang, Chenghao Jiang +5LLM Agent Skill Learning Human-in-the-Loop AI
Tsinghua University · GigaAI
Dissociative Identity: Language Model Agents Lack Grounding for Reputation Mechanisms
May 28, 2026Botao Amber Hu, Helena Rong, Max Van KleekAI Agent Monitoring AI Agent Governance
University of Oxford, UK · New York University Shanghai, China
Training Deliberative Monitors for Black-Box Scheming Detection
May 28, 2026Aditya Sinha, Akshat Naik, Victor Gillioz +5AI Agent Safety AI Agent Monitoring
1Independent · 2MATS Research† · 4Apollo Research +1
OpenClawBench: Benchmarking Process-side Anomalies in Real-world Agent Execution Trajectories
May 28, 2026Yibing Liu, Yangze Liu, Xiaolong Yin +4Agent Failure Analysis Anomaly Localization
School of Software, Shandong University, Jinan, China · School of Artificial Intelligence, Nanjing University; State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing 210023, China · Center for Medical Artificial Intelligence; Qingdao Academy of Chinese Medical Sciences; Institute of Marine Traditional Chinese Medicine, Shandong University of Traditional Chinese Medicine,2026 Qingdao 266112, China +1
The Best-Laid SCHEMEs: Coordinated Sabotage and Monitoring in Multi-Agent Systems
May 27, 2026Nikolay Radev, Lennart Haas, Benjamin Arnav +1AI Agent Security Benchmarks AI Agent Monitoring
The Importance of Out-of-Band Metadata for Safe Autonomous Agents: The Redpanda Agentic Data Plane
May 27, 2026Tyler Akidau, Tyler Rockwood, Johannes Brüderl +1AI Agent Security AI Agent Safety
Redpanda · USA / Germany
FinHarness: An Inline Lifecycle Safety Harness for Finance LLM Agents
May 26, 2026Haoxuan Jia, Yang Liu, Bin Chong +10AI Agent Reliability LLM Guardrails
Nanyang Technological University · Tsinghua University · Peking University +8
Benchmarks are Not Enough: RAMP for Runtime Assessing of Agentic Models in Production Systems
May 26, 2026Yipeng Ouyang, Xin Huang, Bingjie Liu +3Software Engineering Agents LLM Agent Evaluation
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China
Retrying vs Resampling in AI Control
May 25, 2026James Lucassen, Adam KaufmanAI Agent Safety AI Agent Monitoring
Redwood Research
Agent-ToM: Learning to Monitor Autonomous LLM Agents via Theory-of-Mind Reasoning
May 22, 2026Nesreen K. Ahmed, Nima NafisiLLM Agent Security AI Agent Monitoring
Cisco Outshift
Causal Past Logic for Runtime Verification of Distributed LLM Agent Workflows
May 20, 2026Benedikt BolligLLM Agent Orchestration Agentic Workflows
Universit´e Paris-Saclay, CNRS, ENS Paris-Saclay, LMF, Gif-sur-Yvette, France
An Application-Layer Multi-Modal Covert-Channel Reference Monitor for LLM Agent Egress
May 20, 2026Alfredo MetereLLM Agent Security AI Agent Monitoring
Enclawed, LLC, California, USA
CASPIAN: Online Detection and Attribution of Cascade Attacks in LLM Multi-Agent Systems via Cross-Channel Causal Monitoring
May 19, 2026Kavana Venkatesh, Jafar Isbarov, Saad Amin +2Causal Attribution AI Agent Monitoring
Virginia Tech, Blacksburg, VA

Jun 5, 2026, cs.LGJ/K move · Enter open · S save

Strained Coherence: A Pre-Failure Signal in Coding Agent Execution Trajectories

Marut Pandya, Kasey Zhang, Baiqing Lyu

AI Coding Agents AI Agent Monitoring LLM Agent Reliability

Open paper AskPDF arXiv

LLM-based coding agents sometimes acknowledge a problem in their own reasoning and then proceed anyway. We call this pattern strained coherence: a safety-relevant failure mode in which an agent has information that should change its behavior, states that information, and still acts against it. The pattern overlaps with verbalized reward hacking, where an agent names a tension between a task proxy and the underlying goal yet optimizes the proxy anyway. We give an operational definition, build a Claude Sonnet 4.6 judge that reads full trajectories and flags spans where the pattern occurs, and evaluate it on 44 Terminal-bench-2 trajectories using a Qwen3.5-35B-A3B backbone. Flagged trajectories fail 94% of the time versus 46% for unflagged trajectories (47-point gap, Fisher's exact p = 0.003; 46 points after excluding three prompt-embedded examples, p = 0.006). At matched selectivity, the detector reaches 94% precision versus 88% for a lexical discourse-marker baseline; the 10-trajectory intersection of the two methods has a 100% failure rate (Clopper-Pearson 95% CI [69%, 100%]). We replicate on Gemma4-31B with 43 trajectories: the overall signal is directionally consistent but not significant (20-point gap, p = 0.31), with attenuation driven largely by 13 trajectories with zero think content, where the detector has no substrate to analyze. In the high-verbosity Gemma tertile, the gap is +30 points; in the mid- and high-verbosity Qwen tertiles, it is +40 points each. The first flag appears at a median of 83-84% of elapsed trajectory time across both models, and the binary flag survives paraphrases that soften explicit conflict markers (8/8 trajectories). Unlike univariate predictors, the detector emits interpretable span-level output -- quoted acknowledgment, quoted action, and typed conflict -- showing what the agent saw and ignored.

Momentum

Related topics

Latest papers 170