cs.AIFeb 12, 2026
SaveLearning to Configure Agentic AI Systems
Organizations: School of Computing and Augmented Intelligence Arizona State University Tempe, AZ, USA
Abstract
Configuring LLM-based agent systems involves choosing workflows, tools, token budgets, and prompts from a large combinatorial design space, and is typically handled today by fixed templates or hand-tuned heuristics that apply the same configuration regardless of query difficulty, leading to brittle behavior and wasted compute. To address this, we formulate agent configuration as a semi-Markov decision process (SMDP) where each configuration acts as a temporally extended option that determines how an agent system processes a query, and introduce introduce ARC (Agentic Resource & Configuration learner), a lightweight hierarchical policy that dynamically selects query-specific agent configurations. Across reasoning, tool-use, and agentic benchmarks, ARC consistently improves over budget-matched tool-augmented LLMs, increasing average reasoning accuracy by 31.3%, tool-use accuracy by 13.95%, and doubling τ-Bench (Airline) Pass^1 success from 9.0% to 18.0%. These results demonstrate that learning per-query agent configurations is a powerful alternative to "one size fits all" designs.
Figures & tables
Figure 1: Overview of ARC. (a) ARC learns to select a query-specific agent configuration from a large combinatorial design space. (b) Unlike standard MDP actions with fixed duration, agent configurations induce temporally extended executions with variable numbers of LLM calls, motivating our SMDP formulation. (c) Across reasoning, tool-use, and agentic benchmarks, ARC improves performance over budget-matched base models and strong baselines for Qwen 2.5 7B.
Figure 2: Training pipeline. The structure policy selects workflows, tools, and budgets while the prompt policy composes instructions. During RL training, episodes are stored in a memory buffer. After RL converges, high-reward episodes are filtered and used for supervised fine-tuning (SFT), which consolidates successful strategies and improves consistency.
| Reasoning Capability | Tool-Use Capability | |||||||
| Method | Model | GSM8k | DROP | MedQA | Avg / | HotpotQA | GAIA | Avg / |
| Base Model w/ Tools * | Qwen | 37.8% | 36.4% | 49.1% | 41.1% | 10.1% | 2.0% | 6.1% |
| Gemini | 62.3% | 45.9% | 10.0% | 39.4% | 18.8% | 3.0% | 10.9% | |
| Grid Search | Qwen | 74.0% | 54.3% | 53.1% | 60.5% ( ) | 27.9% | 1.0% | 14.45% ( ) |
| Greedy Search | Qwen | 78.2% | 57.5% | 54.9% | 63.5% ( ) | 28.6% | 3.0% | 15.8% ( ) |
| AutoGen | Qwen | 74.8% | 58.0% | 70.5% | 67.8% ( ) | 34.1% | 1.0% | 17.55% ( ) |
Table 1: Performance comparison across reasoning and tool-use benchmarks. We compare our approach (ARC and ARC w/o SFT) against base models with tools, search-based methods (Grid/Greedy Search), optimization frameworks (AutoGen, DSPy, GEPA, LLM as a policy (LAP)), and flat PPO policy baselines (RL Bandits, RL Episodes). Our method achieves the best results on most tasks. Bold and underline indicates best and second best performance over each task.
| Airline Task | |||
| Method | Pass^1 | Pass^2 | Pass^3 |
| Base Model * | 9.0% | 4.5% | 1.5% |
| DSPy | 3.0% | 0.5% | 0.0% |
| ARC w/o SFT | 16.0% | 7.5% | 3.0% |
| ARC | 18.0% | 11.0% | 6.5% |
Table 2: Performance on -Bench (Airline). We evaluate using Qwen 2.5. Each result reports Pass^k, where larger measures whether the agent succeeds consistently across trials runs.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Overview of the nine agentic workflows: Direct (0), Reason+Ans (1), Reason+Verify+Ans (2), Routing (3), Parallel-Sectioning (4), Parallel-Voting (5), Orchestrator-Workers (6), Evaluator-Optimizer (7), and Autonomous-Agent (8). Each workflow defines a distinct pattern of LLM calls and agent interactions.
Figure 7: Overall score vs. runtime vs. multimodality with the Pareto frontier under a three-dimensional dominance criterion (runtime , score , multimodality ). Circles denote text-only models and triangles denote multimodal models. The selected multimodal model (MetaCLIP-H14) is highlighted.
| Embedder | Mode | ARI | Cls. Acc | Complexity | Decision | Overall | Time (s) |
| sentence-t5-base (768D) | native | 0.5603 0.1019 | 0.8733 0.0048 | 0.9261 0.0099 | 0.7221 0.0070 | 0.7704 0.0305 | 38.39 1.24 |
| sentence-t5-base (768D) | projected | 0.4867 0.1017 | 0.8635 0.0084 | 0.9189 0.0109 | 0.7138 0.0018 | 0.7457 0.0229 | 37.56 0.89 |
| MetaCLIP-H14 (1024D) | native | 0.4074 0.0525 | 0.8514 0.0043 | 0.8944 0.0112 | 0.7198 0.0139 | 0.7183 0.0197 | 44.04 1.07 |
| MetaCLIP-2-Worldwide-L14 (768D) | native | 0.3470 | 0.8553 | 0.9131 | 0.7451 | 0.7151 | 43.91 |
| MetaCLIP-2-Worldwide-L14 (768D) | projected | 0.3272 | 0.8470 | 0.9114 | 0.7245 | 0.7025 | 49.20 |
| Jina-CLIP-v2 | projected | 0.3736 0.0415 | 0.8476 0.0028 | 0.8833 0.0124 | 0.6798 0.0027 | 0.6961 0.0129 | 236.38 2.23 |
Table 6: Evaluation of embedding models for downstream RL tasks. The highlighted sentence-t5-base (text-only) and MetaCLIP-H14 (multimodal) are the final candidates, with their performance–efficiency trade-off shown as a Pareto frontier in Fig. 7 .
| Diversity | Quality | ||||||||
| Model | Size | Dataset | Unique | Cover | Sem. | Coher. | Spec. | Clar. | Score |
| GPT-5.2 | Closed | gsm8k | 0.18 | 0.83 | 0.08 | 8.76 | 0.24 | 5.50 | 0.48 |
| GPT-5.2 | Closed | medqa | 0.24 | 0.67 | 0.19 | 8.94 | 0.26 | 5.50 | 0.49 |
| GPT-5.2 | Closed | gaia | 0.19 | 0.83 | 0.10 | 8.56 | 0.25 | 5.50 | 0.48 |
| GPT-5.2 | Closed | hotpotqa | 0.17 | 0.83 | 0.05 | 8.88 | 0.27 | 5.50 | 0.48 |
| GPT-5.2 | Closed | aime25 | 0.15 | 0.83 | 0.18 | 8.81 | 0.23 | 5.50 | 0.49 |
Table 7: Ablation study comparing LLM models for generating prompt atoms. Each row is a model-dataset combination, ranked by combined score (40% diversity + 60% quality). Diversity metrics : Uniqueness (1 mean pairwise cosine similarity), Coverage (fraction of 6 strategy types), Semantic (silhouette score from -means). Quality metrics : Coherence and Clarity (GPT-5 rated 1–10), Specificity (distance from base atoms).
Figure 8: Training dynamics of ARC across datasets. Left: cumulative reward over episodes, showing steady improvement as the policy discovers higher-value configurations on GSM8K, DROP, MedQA, HotpotQA, and GAIA. Middle: rolling mean standard deviation of per-episode reward, indicating reduced variance and stabilization over time. Right: running validation accuracy, demonstrating that reward gains translate into improved task performance with dataset-specific convergence levels.
Figure 9: Tool usage during training. Running average number of tools used per episode for each dataset. ARC quickly learns sparse tool usage and gradually adjusts invocation patterns, with different steady-state levels reflecting task-specific reliance on tools.
Figure 10: Evolution of workflow selection during training. Stacked area plots show, for GSM8K, HotpotQA, and GAIA, the fraction of episodes assigned to each workflow as training progresses. The structure policy quickly prunes suboptimal patterns and concentrates mass on a small set of task-appropriate workflows (e.g., Evaluator–Optimizer on GSM8K, Orchestrator–Workers on HotpotQA).
Figure 11: Accuracy by workflow and dataset. Each bar shows the average accuracy of a fixed workflow on a given benchmark. Performance varies substantially across workflows and tasks no single workflow is uniformly optimal—highlighting the importance of learning query-adaptive configurations rather than relying on a fixed architecture.
Figure 12: Reward distribution by workflow. Violin plots show the distribution of per-episode rewards for each workflow across datasets. Higher-performing workflows exhibit both higher central reward and tighter spread, illustrating that certain structural patterns not only achieve better returns but also yield more stable behavior during training.
More in this topic
LLM Agent Workflow Optimization
LLM agent workflow optimization studies how to design and run multi-step agents for better task success, reliability, latency, and cost.
See all 256 papers