cs.AIAug 16, 2026

Agent Gym: A Framework for Continuous Evaluation and Evolution of LLM Agents Through Human-in-the-Loop Feedback

Authors: Pouya Ghiasnezhad Omran, Michael Zimmermann, Duncan Cambridge, Ashmita Kapoor, Tanya Dixit

Organizations: Google Cloud

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Large Language Model (LLM) agents deployed in production environments face a fundamental tension: the agent's behavior is frozen at deployment time, while the business rules and edge cases it must handle continue to evolve. Existing approaches address agent construction and one-time evaluation but provide no structured mechanism for continuous post-deployment behavioral correction without modifying the agent's source code. Most of the approaches offered in the market, require intense collection of logs and traces, and re-examining the agent design by the engineering team, a process which is heavy, long and negates the economical value of agentic transformation. We introduce Agent Gym, a modular, domain-agnostic framework that wraps any existing LLM-based agent in a continuous evaluation-and-evolution loop. The framework provides six composable capabilities --- Act, Evaluate, Investigate, Correct, Learn, and Observe --- organized across three architectural zones: a constitution layer that codifies domain knowledge in configuration artifacts, a runtime inference pipeline that chains acting, investigation, and adaptive correction, and a learning loop that enables subject matter experts to discover and validate new correction rules through natural language interaction. The key technical contributions include a hybrid deterministic-LLM correction engine with 21 condition operators and three-tier actions, a three-layer investigation architecture for ground-truth-free compliance validation, and a programmatic safety loop that guarantees rule correctness before human approval. We further introduce the Spec-to-Note Gap, an autoencoder-inspired view of agentic system transparency. An open-source reference implementation for invoice processing demonstrates that the framework is fully operational and ready for adoption.

Figures & tables

Figure 1: Three-zone architecture of Agent Gym . Zone 1 provides the constitutional foundation (rules book and master data). Zone 2 executes the runtime inference pipeline: Acting Agent →\rightarrow Investigation Agent →\rightarrow ALF engine. Zone 3 enables human-guided evolution through SME feedback and the Rule Learning Agent. Frequent correction patterns are periodically promoted into the Acting Agent’s permanent logic.
ApproachPost-Deploy CorrectionDomain AgnosticHuman GovernanceNo Agent ModificationGT-Free Evaluation
ReAct / LangChain / AutoGen–✓–––
HELM / AgentBench / SWE-bench–∼\sim–✓–
Self-Refine / Reflexion∼\sim✓–––
RLHF–∼\sim✓––
Traditional BRMS✓∼\sim∼\sim✓–
Agent Gym (ours)✓✓✓✓✓
Table 1: Comparison of Agent Gym with related approaches across key dimensions for production agent management. ✓ indicates full support; ∼\sim indicates partial support; – indicates no support.
LayerMethodLLMScope
1DeterministicNoPer-field
2Rule discoveryYes ∗Global
3Cross-validationYesPer-group
∗ Cached by content hash; amortized to zero after first run.
Table 2: Three-layer investigation architecture. Each layer adds progressively deeper validation with controlled LLM usage.
TierActionLLMScope
1Field editNoSingle field
2Surgical patchYesTarget fields
3Pipeline cont.YesFull output
Table 3: ALF action tiers. Tier selection depends on the nature and extent of the error.
Figure 2: ALF Collect-Plan-Execute pipeline. Conditions are evaluated deterministically (Collect), actions are grouped by tier (Plan), and executed in order: Tier 3 →\rightarrow Tier 2 →\rightarrow Tier 1. A full audit trail is maintained throughout.
Figure 3: Programmatic safety loop for rule discovery. Candidate rules are iteratively validated, checked for target coverage, and auto-tightened to eliminate collateral matches before SME approval.
Figure 4: The Spec-to-Note autoencoder. Natural language specification is encoded into an implemented system, then decoded back to natural language by an LLM auditor. The gap between spec and transparency note surfaces missing or unintended behaviors.
More in this topic

LLM Agent Evaluation

Large language model agent evaluation measures whether tool-using systems complete tasks reliably, safely, and efficiently.

See all 1,203 papers

Explore similar work

CardsList
May 21, 2026

Agentic CLEAR: Automating Multi-Level Evaluation of LLM Agents

Agentic systems are becoming more capable: agents define strategies, take actions, and interact with different environments. This autonomy poses serious challenges for overseeing and assessing agent behavior. Most current tools are limited, focusing on observability with basic evaluation capabilities or imposing static, hand-crafted error taxonomies that cannot adapt to new domains. To address this gap, we present Agentic CLEAR, an automatic, dynamic, and easy-to-use evaluation framework. It produces textual insights into the agent behavior on three levels of granularity: system, trace, and node. Agentic CLEAR operates above the observability layer, enabling seamless integration and featuring an intuitive UI that makes agent evaluation highly accessible. In our experiments on four benchmarks, seven agentic settings, and tens of thousands of LLM calls, we show that Agentic CLEAR produces high-quality, data-driven, insightful feedback. Our analysis shows strong alignment with human-annotated errors and the ability to predict task success rate.

Asaf Yehudai, Lilach Eden, Michal Shmueli-Scheuer

IIBM Research

LLM Agent EvaluationAI Agent BenchmarksAutomated EvaluationAutomated Agent Evaluation
AskPDF
Jun 16, 2026

SEAGym: An Evaluation Environment for Self-Evolving LLM Agents

Self-evolving LLM-based agents improve mainly by changing their agent harness: the structured execution layer around a base model, including prompts, memory, tools, middleware, runtime state, and the model-tool interaction loop. Existing evaluations often reduce this process to isolated task scores or a single sequential curve, obscuring whether an update produces reusable improvement, overfits recent tasks, increases cost, or harms older behavior. We introduce SEAGym, an evaluation environment for measuring agent harness updates across training, validation, test, replay, and cost records. SEAGym turns Harbor-compatible benchmarks into dynamic self-evolution task sources with train batches, frozen update-validation, held-out ID and OOD transfer views, replay diagnostics, and saved snapshot and metric records. Instantiating SEAGym on Terminal-Bench 2.0 and HLE, we compare ACE, TF-GRPO, and AHE under a shared epoch/batch protocol. The results show that these evaluation views provide complementary signals about the evolution process: frequent updates may fail to improve held-out performance, useful intermediate snapshots may collapse later, and source diversity and model backend can affect harness reliability.

Congjie Zheng, Chuanyi Xue, Bin Liang +2

Department of Automation, Tsinghua University, Beijing, 100084, China · Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University, Beijing, 100084, China

LLM Agent EvaluationAI Agent BenchmarksAutomated Agent EvaluationLLM Agent ReliabilitySelf-Evolving Agents
AskPDF
Jun 14, 2026

LLM-as-Code: Agentic Programming for Agent Harness

Every major LLM agent framework gives the LLM the role of orchestrator; the model decides what to do next, when to call tools, and when to stop. We argue that token explosion, control-flow hallucination, and unreliable completion are not implementation bugs but architectural consequences of assigning the deterministic work of looping, branching, and sequencing to a probabilistic system. A better prompt or a stronger model cannot guarantee the reliability of the LLM agent. We therefore propose Agentic Programming, in which the program governs all control flow, and the LLM is itself part of it, an adaptive component we call LLM-as-Code and invoke only where a task calls for reasoning or generation. Within each call the model keeps full flexibility, but it cannot alter the program's execution path. With control in the program, the LLM's context is built from the execution history's call tree and forms a directed acyclic graph (DAG). Each call's context length is then determined by its call depth rather than by accumulation over steps. A case study of computer-use agents shows that the design is practical, not just a theoretical stance, substantially improving the stability of long visual operation sequences.

Junjia Qi, Zichuan Fu, Jingtong Gao +4

City University of Hong Kong · Tencent Jarvis Lab

AI Agent ReliabilityLLM Agent OrchestrationLong-Horizon LLM AgentsLLM Agent Reliability
AskPDF