GainCtrl: Predicting Remaining Gains for Efficient LLM Agent Execution
Organizations: University of Science and Technology of China · Ant Digital Technologies, Ant Group
Abstract
Large language models (LLMs) are widely used to power agents that interact with tools and environments to solve complex tasks. Repeated attempts improve the quality of tool-using agents, but continued execution incurs additional costs even when it is unlikely to improve the best result ultimately returned. This raises the question of when to stop an attempt to reduce execution cost while preserving task-level Best-of-N quality. To address this question, we introduce GainCtrl, a gain-based execution controller for LLMs. In GainCtrl, a lightweight CPU predictor uses interaction histories to estimate expected positive gains over retained scores, which are then used to guide stopping decisions with calibrated thresholds to limit the loss in mean Best-of-N task quality. We evaluate the controller through trajectory replay across three environments and three LLM actors. Comparisons of interpolated quality-cost Pareto frontiers show that GainCtrl generally uses fewer tokens than baselines at matched target quality. Fixed policies also achieve quality close to that of full execution while substantially reducing token consumption. The predictor runs at millisecond latency on a single CPU thread, supporting repeated monitoring without access to LLM internals.
Figures & tables
| Setting | Full | FailFast | GainCtrl-T | GainCtrl-P |
|---|---|---|---|---|
| ScienceWorld | ||||
| 4B | 294.2 | 227.1 | 161.3 | 143.5 (36.8%) |
| 9B | 175.8 | 176.7 | 133.8 | 110.8 (37.3%) |
| Luna | 27.9 † | 29.7 | 27.1 | 27.2 (8.3%) |
| ALFWorld | ||||
| 4B | 791.1 | 494.4 | 234.3 | 234.3 (52.6%) |
| Setting | Budget (k) | Full | FailFast | GainCtrl-T | GainCtrl-P |
|---|---|---|---|---|---|
| ScienceWorld | |||||
| 4B | 242.8 | 41.42 | 44.86 | 48.30 | 49.80 (+4.94) |
| 9B | 180.0 | 52.38 | 52.14 | 55.16 | 59.18 (+7.04) |
| Luna | 85.0 | 86.75 | 84.21 | 85.14 | 88.43 (+4.22) |
| ALFWorld | |||||
| 4B | 466.6 | 60.30 | 65.38 | 67.50 † | 67.50 † (+2.12) |
| Setting | |||
|---|---|---|---|
| ScienceWorld | |||
| 4B | 96.79 / 40.52 | 95.98 / 45.54 | 96.05 / 48.54 |
| 9B | 97.88 / 33.54 | 97.80 / 38.30 | 98.25 / 38.20 |
| Luna | 97.14 / 43.34 | 97.02 / 46.24 | 97.74 / 44.81 |
| ALFWorld | |||
| 4B | 99.04 / 53.64 | 96.88 / 68.61 | 97.12 / 72.56 |
| Setting | Statistics | Statistics + Action text | Statistics + Action text + Feedback text |
|---|---|---|---|
| ScienceWorld | |||
| 4B | 96.50 / 31.27 | 96.05 / 48.54 | 96.96 / 41.22 |
| 9B | 97.60 / 28.85 | 98.25 / 38.20 | 98.60 / 36.55 |
| Luna | 97.31 / 31.33 | 97.74 / 44.81 | 97.93 / 44.04 |
| ALFWorld | |||
| 4B | 98.56 / 57.02 | 97.12 / 72.56 | 99.28 / 61.32 |
| Predictor input | Dim. | Retain (%) | Save (%) |
|---|---|---|---|
| ScienceWorld | |||
| Statistics | 14 | 96.50 | 31.27 |
| Action text | 64 | 98.37 | 27.89 |
| Statistics + Action text | 78 | 96.05 | 48.54 |
| ALFWorld | |||
| Statistics | 140 | 98.56 | 57.02 |
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
| Environment | Train/val/test | Train trials | Cap | Checkpoints | Features |
|---|---|---|---|---|---|
| ScienceWorld | 897/200/200 | 3 | 20 | 2,6,10,14,18 | 78 |
| ALFWorld | 2220/200/200 | 3 | 30 | 5,10,15,20 | 204 |
| WebShop | 1161/200/200 | 3 | 10 | 2,4,6 | 155 |
| Algorithm 1: Training and Inference of the Gain Predictor | |
|---|---|
| Input: training tasks ; three selected complete trajectories per task; environment , step cap , checkpoints , and score retention rule; grid , . | |
| Output: fitted feature map , one ten-output forest , and the gain predictor . | |
| Training | |
| 1 | Initialize training records . |
| 2 | For each training task : initialize . |
| 3 | For each selected trajectory : |
| Setting | GainCtrl-P bracket | FailFast bracket | ||
|---|---|---|---|---|
| ScienceWorld/4B | D10/10–D10/11 | 0.241 | F5/12–F5/13 | 0.645 |
| ScienceWorld/9B | D10/6–D10/7 | 0.326 | F5/12–F5/13 | 0.131 |
| ScienceWorld/Luna | D2/1–D1/1 | 0.137 | F5/1–F5/2 | 0.279 |
| ALFWorld/4B | D5/15–D2/13 | 0.051 | F5/12–F5/13 | 0.654 |
| ALFWorld/9B | D2/12–D2/13 | 0.213 | F5/14–F5/15 | 0.925 |
| ALFWorld/Luna | D5/8–D5/9 | 0.917 | F10/9–F10/10 | 0.457 |
| Setting | (%) | Full | GainCtrl-P | Saved (%) | FailFast-task | Saved (%) |
|---|---|---|---|---|---|---|
| ScienceWorld/4B | 1 | 55.25 | 54.83 | 29.2 | 54.91 | 10.5 |
| ScienceWorld/4B | 2 | 55.25 | 53.07 | 48.5 | 54.01 | 14.5 |
| ScienceWorld/4B | 10 | 55.25 | 46.97 | 74.0 | 47.06 | 58.8 |
| ScienceWorld/9B | 1 | 65.00 | 64.37 | 28.2 | 64.12 | 12.8 |
| ScienceWorld/9B | 2 | 65.00 | 63.86 | 38.2 | 63.34 | 22.8 |
| ScienceWorld/9B | 10 | 65.00 | 59.05 | 63.6 | 59.54 | 43.6 |
| Setting | (%) | GainCtrl-T | GainCtrl-P | FF | FF floor | FF hits |
|---|---|---|---|---|---|---|
| ScienceWorld/4B | 1 | 0.01000 | 0.00200 | 0.9882 | 0.0 | count 1 |
| ScienceWorld/4B | 2 | 0.02512 | 0.01000 | 0.9632 | 0.0 | count 4 |
| ScienceWorld/4B | 10 | 0.06310 | 0.03981 | 0.9773 | 0.0 | count 1 |
| ScienceWorld/9B | 1 | 0.01585 | 0.00316 | 0.9783 | 0.0 | count 1 |
| ScienceWorld/9B | 2 | 0.03162 | 0.00631 | 0.9683 | 0.0 | count 2 |
| ScienceWorld/9B | 10 | 0.06310 | 0.03162 | 0.9686 | 0.0 | count 1 |
| Environment | Input | Score difference [95% CI] | Saving difference [95% CI] |
|---|---|---|---|
| ScienceWorld | Statistics | +0.245 [-0.639, +1.132] | -17.27 [-19.55, -15.02] |
| ScienceWorld | Action text | +1.277 [+0.364, +2.411] | -20.65 [-23.26, -18.02] |
| ALFWorld | Statistics | +1.000 [-1.000, +3.000] | -15.54 [-20.39, -10.71] |
| ALFWorld | Action text | -0.500 [-2.500, +1.000] | -18.93 [-24.27, -13.82] |
| WebShop | Statistics | +0.714 [+0.082, +1.615] | -10.22 [-13.35, -7.19] |
| WebShop | Action text | +0.378 [-0.205, +1.319] | -12.25 [-14.60, -9.97] |
| Environment | Predictor input | Retain (%) | Save (%) |
|---|---|---|---|
| ScienceWorld | Statistics | 94.01 | 49.89 |
| ScienceWorld | Action text | 92.44 | 47.64 |
| ScienceWorld | Statistics + Action text | 91.32 | 62.57 |
| ALFWorld | Statistics | 97.12 | 71.49 |
| ALFWorld | Action text | 94.96 | 68.21 |
| ALFWorld | Statistics + Action text | 94.96 | 79.11 |
| GainCtrl | FailFast | |||
|---|---|---|---|---|
| Setting | Fit (s) | Check (ms) | Fit (min) | Fwd (ms) |
| ScienceWorld/4B | 4.44 | 7.29 | 31.5 | 66.08 |
| ScienceWorld/9B | 4.15 | 7.11 | 26.7 | 69.51 |
| ScienceWorld/Luna | 4.01 | 6.97 | 12.8 | 81.12 |
| ALFWorld/4B | 12.88 | 10.74 | 47.7 | 127.21 |
| ALFWorld/9B | 11.84 | 11.86 | 47.0 | 89.77 |
| Actor | (%) | Retain (%) | Save (%) | |
|---|---|---|---|---|
| 4B | 1 | 54.83 | 99.24 | 29.19 |
| 4B | 2 | 53.07 | 96.05 | 48.54 |
| 4B | 5 | 50.45 | 91.32 | 62.57 |
| 4B | 10 | 46.97 | 85.02 | 73.99 |
| 4B | 25 | 40.39 | 73.11 | 86.00 |
| 9B | 1 | 64.37 | 99.03 | 28.18 |
| Actor | (%) | Retain (%) | Save (%) | |
|---|---|---|---|---|
| 4B | 1 | 67.50 | 97.12 | 69.96 |
| 4B | 2 | 67.50 | 97.12 | 72.56 |
| 4B | 5 | 66.00 | 94.96 | 79.11 |
| 4B | 10 | 62.50 | 89.93 | 84.10 |
| 4B | 25 | 56.50 | 81.29 | 91.14 |
| 9B | 1 | 79.50 | 98.15 | 47.71 |
| Actor | (%) | Retain (%) | Save (%) | |
|---|---|---|---|---|
| 4B | 1 | 34.90 | 96.86 | 16.19 |
| 4B | 2 | 34.41 | 95.52 | 37.24 |
| 4B | 5 | 34.39 | 95.46 | 41.13 |
| 4B | 10 | 32.75 | 90.89 | 59.99 |
| 4B | 25 | 28.29 | 78.53 | 78.30 |
| 9B | 1 | 33.03 | 100.00 | 21.99 |