TRACE-CASH: Trial-History-Conditioned Reinforcement Learning for Adaptive Configuration Exploration in Time-Series CASH

Yu-Han Huang; Yujia Wu; Vincent S. Tseng

cs.LGAug 17, 2026

TRACE-CASH: Trial-History-Conditioned Reinforcement Learning for Adaptive Configuration Exploration in Time-Series CASH

Authors: Yu-Han Huang, Yujia Wu, Vincent S. Tseng

Organizations: Department of Computer Science, National Yang Ming Chiao Tung University, Taiwan

Abstract

Combined algorithm selection and hyperparameter optimization (CASH) searches a conditional space in which the selected model determines which hyperparameters are active. In time-series forecasting, temporal choices, chronological validation, and costly evaluations further complicate this search. Controlled comparisons of heterogeneous search methods under a shared time-series CASH (TS-CASH) evaluation protocol remain limited. Within this setting, we study TRACECASH, a task-local hybrid sequential optimizer combining grouped actor-critic candidate generation with fixed rules for model coverage, validation-guided exploitation, and exploration after stalled progress. A model actor proposes an initial forecasting model; three model-conditioned actors generate temporal, architectural, and training actions; and a modelspecific decoder constructs the configuration ultimately evaluated. We compare TRACE-CASH with six alternatives spanning random, Bayesian, evolutionary, multi-objective, and language-model-assisted search across 41 dataset-frequency task variants. TRACE-CASH has the lowest mean rank on both MASE and WQL. Descriptively, it also has the lowest window-averaged test-MASE rank in the predefined full and late windows. These results support the complete TRACECASH procedure as competitive among the evaluated methods.

Figures & tables

Figure 1: Overview of TRACE-CASH. (a) Grouped actors generate an initial model proposal and model-conditioned temporal, architectural, and training actions. Exploration or fixed rules may revise the model before the model-specific decoder constructs the evaluated configuration. Recorded outcomes update the trial-history state and replay buffer, while validation MASE determines the incumbent. (b) Immediate-reward actor–critic update. The reward shown applies to valid trials. Panel (b) shows the single-trial residual based on the sum of critic scores; Eq. ( 4 ) gives the corresponding minibatch objective.

Component	Study setting
Tasks and seeds	41 dataset–frequency task variants; three seeds per method–task pair
Budget	200 attempted trials per task, method, and seed
Temporal evaluation	Fixed chronological validation and test routes
Search space	Common nine-family conditional design
Incumbent	Lowest validation MASE; earliest evaluation breaks ties
Reported metrics	MASE and WQL

Table 1: Aligned evaluation conditions and reported endpoints.

Method	MASE	WQL
Random Search	5.24	5.05
SMAC3	3.59	3.46
TPE	3.85	3.95
NSGA-III	4.78	4.46
MOTPE	4.24	4.41
SLLMBO *	3.37	3.76

Table 2: Full-budget mean ranks for the two reported metrics (lower is better;

N=41

task variants for each). Seeds are averaged within task before methods are ranked. SLLMBO * denotes the source-based adaptation.

Figure 2: Mean test-MASE rank over attempted trials; lower is better. Curves are smoothed only for display using centered 17-point means; symbols mark 25-trial intervals. Window summaries use the unsmoothed task-level rank trajectories.

Method	Early (1–75)	Late (76–200)	Full (1–200)
Random Search	4.147	4.811	4.562
SMAC3	4.152	3.441	3.708
TPE	3.693	3.908	3.828
NSGA-III	4.565	4.774	4.695
MOTPE	4.203	4.464	4.366
SLLMBO *	3.497	3.441	3.462

Table 3: Descriptive window-averaged test-MASE incumbent ranks (lower is better;

N=41

task variants). Ranks are averaged over attempts within task and then across tasks. Values are rounded to three decimals; emphasis follows the unrounded values.

Explore similar work

Bandits in Prod: Hyperparameter Optimization at Inference Time
Sep 1, 2026Louis Abraham, Tuan-Anh Nguyen, Nicolas DevatineMulti-Armed Bandits Inference-Time Optimization
Tiime Paris, France
HARBOR: Automated Harness Optimization
Apr 22, 2026Biswa Sengupta, Jinhua WangBayesian Optimization Agent Harness Optimization
LLM Suite Team, JP Morgan Chase & Co.
TW3Cast: A Frozen Router of Lightly Fine-Tuned Foundation Models for Time-Series Forecasting on GIFT-Eval, Selected Entirely on the Training Split
Sep 16, 2026Nathan Thierry, Andre-Louis RochetTime Series Forecasting Time-Series Foundation Models
TW3 Partners

Sep 1, 2026J/K move · Enter open · S save

Bandits in Prod: Hyperparameter Optimization at Inference Time

Louis Abraham, Tuan-Anh Nguyen, Nicolas Devatine

Tiime Paris, France

Multi-Armed Bandits Inference-Time Optimization Hyperparameter Optimization

Open paper AskPDF arXiv

Many production systems can assess a configuration only by using it on live requests and observing noisy feedback. Modern agentic systems are a prominent example, with inference-time choices such as model selection, retrieval depth, prompting strategy, and decoding temperature, yet often with no representative validation data. We formalize this setting as Online Hyperparameter Optimization (OHPO) and cast it as an infinitely many-armed bandit over mixed and conditional search spaces. We introduce IMABO, a general framework that combines any bandit policy for choosing among already sampled configurations with any oracle for proposing new ones. We instantiate it with IMOSS, a restart-free anytime policy whose active set grows as

t^β

, and prove an expected cumulative quantile-regret bound of

O(p_ρ^{-1/β} + T^{(1+β)/2})

, where

β\in(0,1)

controls active-set growth and

p_ρ

lower-bounds the probability that a proposed configuration falls in the top-

ρ

fraction of the search space. We combine IMOSS with three practical oracles: a Tree-structured Parzen Estimator, an incumbent-mutation oracle driven by a per-coordinate bandit, and a pretrained tabular foundation model, all three improving over the uniform random oracle baseline. IMABO outperforms all baselines in terms of regret across diverse OHPO settings, from tuning classical machine-learning models to configuring LLM-based agents. Our implementation is available at https://github.com/Tiime-Software/IMABO.

Abstract

Explore similar work

Bandits in Prod: Hyperparameter Optimization at Inference Time

HARBOR: Automated Harness Optimization

TW3Cast: A Frozen Router of Lightly Fine-Tuned Foundation Models for Time-Series Forecasting on GIFT-Eval, Selected Entirely on the Training Split