Think Once, Act Fast: Cost-Effective Recursive Self-Improvement for Decision Model Harness
Organizations: AI Thrust, Hong Kong University of Science and Technology (Guangzhou) · LIGHTSPEED
Abstract
Fast System-1 models enable responsive actions but struggle with complex or unfamiliar tasks. Large language model (LLM)-based System-2 provides strong reasoning capabilities, yet repeated invocation incurs substantial latency and computational costs. To bridge this gap, we propose JevRSI, a System-1/System-2 collaboration framework that transforms LLM reasoning into transferable decision support through recursive harness improvement. A Jev-style System-1 model enables low-latency action selection, while an asynchronous System-2 LLM develops a modular harness. The harness integrates procedures, state memory, and world models to provide decision contexts spanning the past, present, and future. Utility-driven retrieval selects modules based on both contextual relevance and their utility on previous tasks. Environmental feedback guides targeted revisions and enables recursive self-improvement. Experiments across three datasets show that JevRSI consistently improves decision quality, matching or surpassing LLM agents in several settings. It preserves low-latency execution with favorable performance-cost trade-offs. Moreover, System-2 development interventions decrease as the harness evolves, amortizing System-2 costs through continuous improvement. Our project is available at https://github.com/usail-hkust/JevRSI.
Figures & tables
| ALFWorld | BabyAI | TextCraft | |
|---|---|---|---|
| Training size | 200 | 200 | 200 |
| ID size | 140 | 150 | 150 |
| OOD size | 134 | 150 | 150 |
| Baselines | ALFWorld | BabyAI | TextCraft | |||
|---|---|---|---|---|---|---|
| ID | OOD | ID | OOD | ID | OOD | |
| LLM Agents | ||||||
| GPT-6.1 Sol | 93.57% | 96.26% | 99.33% | 100.0% | 100.0% | 82.00% |
| DeepSeek-V4.1-Flash | 80.71% | 82.83% | 72.66% | 92.00% | 100.0% | 62.00% |
| Qwen3.8-27B | 82.85% | 76.11% | 78.66% | 64.00% | 100.0% | 55.33% |
| Decision Models | ||||||
LLM Agent Self-Improvement
Large language model agent self-improvement studies how agents use experience and feedback to improve their behavior, tools, and workflows.
See all 298 papers