Human-inspired, Task-Dimension-Guided Exploration for Efficient Learning in High Dimensions

Fanyu Zhu; Jiahui An; Ni Ji

cs.LGSep 29, 2026

Human-inspired, Task-Dimension-Guided Exploration for Efficient Learning in High Dimensions

Authors: Fanyu Zhu, Jiahui An, Ni Ji

Organizations: Beijing Institute for Brain Research, Chinese Academy of Medical Sciences & Peking Union Medical College, Chinese Institute for Brain Research, Beijing, Beijing Key Laboratory of Brain Science and Brain-Machine Interface · State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University

Abstract

Efficient exploration in high-dimensional decision spaces remains a central challenge for decision-making systems. Humans, in contrast, can navigate large decision spaces with remarkable efficiency. Recent behavioral studies suggest that humans reduce dimensionality in large decision spaces by probing candidate feature dimensions, identifying reward-relevant ones, and restricting the effective decision space. Inspired by this mechanism, we propose TDGE (Task-Dimension-Guided Exploration), a human-inspired, model-agnostic algorithm with an automatically constructed task-dimension--feature--item hierarchy. TDGE follows a top-down exploration strategy: it first selects task-relevant feature dimensions, then identifies informative features within those dimensions, and finally recommends concrete items based on the selected features. Experiments on MovieLens-20M, LastFM, and Amazon recommendation datasets show that TDGE substantially improves exploration efficiency and cold-start adaptation over baseline algorithms. Comparisons with other structured algorithms and ablation studies attribute these gains to TDGE's hierarchical structure and semantic feature-space exploration, with robust results across clustering methods and hierarchy depths. Recommendation-trajectory visualizations also show exploration patterns similar to human dimension-guided behavior.

Figures & tables

Figure 1: Human exploration through dimension-level search. (a) Behavioral task setup. (b) Reward-relevant dimensions reduce the effective action space. (c) Association between the proportion of DGE rounds and the probability of achieving a full score. (d) Schematic illustration of the observed dimension-wise exploration strategy. Adapted from An et al. [ 1 ] with permission.

Panel of Figure 2 — Figure 2: (a) Normalized KGS penalty versus cluster count on MovieLens-20M; $k^{*}$ minimizes the penalty among candidate partitions satisfying the cluster-size constraint. (b) Representative portion of the feature dendrogram: semantically related tags of the same color form a task dimension.

Figure 3: The TDGE exploration schedule. The dimension agent selects promising semantic aspects, the feature agent identifies specific attributes and guides candidate generation, and the item agent makes the final recommendation. The observed reward updates the item agent and the matched selected features and their source dimensions.

Metric	Dataset	Backbone	Baseline	TDGE
Online CReg $\downarrow$	MovieLens-20M	LinUCB	$2299.697{\pm}20.439$	$1573.142{\pm}43.499^{\ast\ast\ast}$
		NeuralUCB	$2417.798{\pm}15.161$	$1669.303{\pm}91.162^{\ast\ast\ast}$
		NeuralTS	$2418.077{\pm}15.964$	$1705.542{\pm}52.769^{\ast\ast\ast}$
	Last.fm	LinUCB	$2438.154{\pm}43.877$	$2453.269{\pm}27.544\ \mathrm{n.s.}$
		NeuralUCB	$2233.387{\pm}13.230$	$1960.563{\pm}57.620^{\ast\ast\ast}$
		NeuralTS	$2234.352{\pm}16.528$	$1975.365{\pm}81.538^{\ast\ast}$

Table 1: Online cumulative regret at

T=10{,}000

and final-round cold-start regret at

t=10

for TDGE and its corresponding item-level baselines. Lower is better. Results are mean

\pm

sample standard deviation over five seeds. Superscripts compare TDGE with its corresponding backbone using two-sided Welch

t

-tests.

Metric	LinUCB	CoFineUCB	TDGE-LinUCB	NeuralUCB	H 2 N-Bandit	TDGE-NeuralUCB
Online CReg $\downarrow$	$2299.697{\pm}18.281$	$2271.580{\pm}34.295$	$\mathbf{1573.142{\pm}38.906}^{\ast\ast\ast}$	$2417.798{\pm}13.561$	$2379.700{\pm}17.122$	$\mathbf{1669.303{\pm}81.538}^{\ast\ast\ast}$
Cold-start Regret ( $t=10$ ) $\downarrow$	$0.234{\pm}0.004$	$0.237{\pm}0.010$	$\mathbf{0.080{\pm}0.011}^{\ast\ast\ast}$	$0.239{\pm}0.011$	$0.235{\pm}0.008$	$\mathbf{0.060{\pm}0.014}^{\ast\ast\ast}$

Table 2: Online cumulative regret at

T=10{,}000

and final-round cold-start regret at

t=10

on MovieLens-20M. Lower is better. Results are mean

\pm

standard deviation over five seeds. Superscripts compare TDGE-LinUCB with CoFineUCB and TDGE-NeuralUCB with H 2 N-Bandit using two-sided Welch

t

-tests.

Metric	Backbone	Item clustering	Hierarchical	$k$ -means	Spectral	GMM
Online CReg	LinUCB	$2101.500\pm 37.535^{\ast\ast\ast}$	$1573.142\pm 38.906$	$1608.404\pm 49.511$	$1639.962\pm 135.799$	$1558.282\pm 63.691$
	NeuralUCB	$2223.900\pm 45.983^{\ast\ast\ast}$	$1669.303\pm 81.538$	$1839.836\pm 124.657$	$1854.616\pm 42.444$	$1786.975\pm 35.025$
	NeuralTS	$2295.800\pm 13.952^{\ast\ast\ast}$	$1705.542\pm 47.198$	$1815.254\pm 112.595$	$1792.321\pm 180.387$	$1709.182\pm 94.690$
Cold-start Regret ( $t=10$ )	LinUCB	$0.219\pm 0.011^{\ast\ast\ast}$	$0.080\pm 0.011$	$0.127\pm 0.005$	$0.118\pm 0.016$	$0.133\pm 0.024$
	NeuralUCB	$0.210\pm 0.014^{\ast\ast\ast}$	$0.060\pm 0.014$	$0.075\pm 0.011$	$0.072\pm 0.015$	$0.072\pm 0.015$
	NeuralTS	$0.212\pm 0.018^{\ast\ast\ast}$	$0.078\pm 0.023$	$0.077\pm 0.007$	$0.065\pm 0.007$	$0.069\pm 0.014$

Table 3: Feature-space and clustering-method ablations on MovieLens-20M. Lower regret is better. The superscript indicates that hierarchical feature-space TDGE outperforms the corresponding item-clustering alternative under a two-sided Welch

t

-test.

Metric	Backbone	TDGE (3 levels)	TDGE (4 levels)
Online CReg	LinUCB	$1573.142\pm 38.906$	$1586.160\pm 69.115$
	NeuralUCB	$1669.303\pm 81.538$	$1768.740\pm 142.438$
	NeuralTS	$1705.542\pm 47.198$	$1606.220\pm 90.449$
Cold-start Regret ( $t=10$ )	LinUCB	$0.080\pm 0.011$	$0.090\pm 0.022$
	NeuralUCB	$0.060\pm 0.014$	$0.078\pm 0.017$
	NeuralTS	$0.078\pm 0.023$	$0.060\pm 0.017$

Table 4: Online cumulative regret and cold-start final-round regret for the hierarchy-depth ablation on MovieLens-20M. Lower is better.

Panel of Figure 4 — Figure 4: Feature-dimension trajectories over 10 cold-start interactions and the distribution of consecutive same-dimension run lengths on MovieLens-20M. (a) TDGE-NeuralUCB. (b) NeuralUCB. Each step in (a,b) is represented by the dominant feature dimension of the recommended item. (c) Same-dimension run-length distributions aggregated across cold-start users.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	# Users	# Items	# Features	# Dimensions ( $k^{\ast}$ )	# Interactions	Sparsity
MovieLens-20M	5,000	10,000	1,128	47	5,068,810	89.86%
Last.fm	1,892	10,000	2,074	37	84,365	99.55%
Amazon	5,000	43,724	5,743	35	494,083	99.77%

Appendix

Table 5: Recommendation-dataset statistics after preprocessing.

Parameter	Value
Online rounds ( $T$ )	10,000
Independent runs	5
Final item budget ( $K$ )	10
Train/cold-start user split	90%/10%
Cold-start users per run	100
Cold-start steps per user	10

Appendix

Table 6: Shared recommendation hyperparameters.

Method	MovieLens-20M	Last.fm	Amazon
TDGE-LinUCB	$(5,10)$	$(2,4)$	$(1,6)$
TDGE-NeuralUCB	$(5,10)$	$(3,10)$	$(3,24)$
TDGE-NeuralTS	$(5,10)$	$(1,24)$	$(5,24)$

Appendix

Table 7: Selected dimension and feature retention budgets

(K_{1},K_{2})

Backbone	MovieLens-20M	Last.fm	Amazon
LinUCB	$(0.01,1)$	$(0.01,1)$	$(1,0.01)$
NeuralUCB	$(0.01,0.1)$	$(0.01,0.01)$	$(1,0.1)$
NeuralTS	$(0.1,10)$	$(0.01,10)$	$(0.1,10)$

Appendix

Table 8: Selected exploration and regularization parameters

(\alpha,\lambda)

for the flat backbones.

Method	MovieLens-20M	Last.fm	Amazon
TDGE-LinUCB	$(1,1)/(1,1)/(0.01,1)$	$(0.1,1)/(1,0.1)/(0.01,1)$	$(1,1)/(0.1,1)/(1,0.01)$
TDGE-NeuralUCB	$(0.1,1)/(0.1,0.1)/(0.01,0.1)$	$(1,10)/(0.1,10)/(0.01,0.01)$	$(0.1,10)/(0.1,10)/(1,0.1)$
TDGE-NeuralTS	$(0.1,1)/(0.1,1)/(0.1,10)$	$(0.1,10)/(0.1,10)/(0.01,10)$	$(0.1,0.1)/(0.1,0.1)/(0.1,10)$

Appendix

Table 9: Selected TDGE parameters. Each cell lists

(\alpha,\lambda)_{\mathrm{dim}}/(\alpha,\lambda)_{\mathrm{feat}}/(\alpha,\lambda)_{\mathrm{item}}

. For each dataset–backbone pair, the item-level parameters are identical to those of the corresponding flat backbone in Table 8 .

Metric	Backbone	NoFD	Full TDGE
Online CReg	LinUCB	$1885.934\pm 33.885$	$\mathbf{1573.142\pm 38.906}^{\ast\ast\ast}$
	NeuralUCB	$2059.076\pm 29.659$	$\mathbf{1669.303\pm 81.538}^{\ast\ast\ast}$
	NeuralTS	$2074.051\pm 15.427$	$\mathbf{1705.542\pm 47.198}^{\ast\ast\ast}$
Cold-start Regret ( $t=10$ )	LinUCB	$0.170\pm 0.014$	$\mathbf{0.080\pm 0.011}^{\ast\ast\ast}$
	NeuralUCB	$0.173\pm 0.015$	$\mathbf{0.060\pm 0.014}^{\ast\ast\ast}$
	NeuralTS	$0.175\pm 0.008$	$\mathbf{0.078\pm 0.023}^{\ast\ast\ast}$

Appendix

Table 10: Online cumulative regret and cold-start final-round regret for the feature-dimension-level ablation on MovieLens-20M. Lower is better. Superscripts compare full TDGE with NoFD using two-sided Welch

t

-tests.

Figure 5: Online cumulative regret over 10,000 interaction rounds. Curves show the mean over five runs, and shaded regions denote one standard deviation. Dashed curves are the item-level baselines, and solid curves are the corresponding TDGE variants. Dotted and dash-dotted curves denote CoFineUCB and H 2 N-Bandit, respectively, where results are available. Lower is better.

Figure 6: Cold-start cumulative regret over 10 interactions per held-out user. Within each run, regret at each step is averaged over 100 users before accumulation. Curves show the mean over five runs, and shaded regions denote one standard deviation. Dashed curves are the item-level baselines, and solid curves are the corresponding TDGE variants. Dotted and dash-dotted curves denote CoFineUCB and H 2 N-Bandit, respectively, where results are available. Lower is better.

Figure 7: Average instantaneous regret at each step of the 10-step cold-start trajectory. At each step, regret is averaged over 100 held-out users within each run. Curves show the mean over five runs, and shaded regions denote one standard deviation. The values at step 10 correspond to the final-round regret reported in Table 1 . Lower is better.

Method	Median (ms)	P95 (ms)
NeuralUCB	1.6716	1.7666
TDGE-NeuralUCB	1.9353	1.9538
NeuralTS	1.7218	1.7504
TDGE-NeuralTS	2.0404	2.0606

Appendix

Table 11: Per-decision neural inference latency with 100,000 item arms. TDGE additionally scores 5,000 dimension arms and 5,000 feature arms. Measurements use an NVIDIA GeForce RTX 4090 after 50 warm-up passes; 150 passes are timed.

Explore similar work

RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems
Jul 31, 2026Haoran Ling, Yuecheng Li, Zeyu Song +5Prompt Optimization Bandits
Georgia Institute of Technology · Kuaishou Technology, China
LeAP: Learnable Adaptive Permutation for Feature Selection in Heterogeneous and Sparse Recommender Systems
May 31, 2026Yihong Huang, Chen Chu, Fei Chen +3Feature Selection Real-World Content Recommendation Problem
Bilibili Inc., Shanghai, China
Delightful Exploration
May 13, 2026Ian OsbandStochastic Exploration Exploration
Google DeepMind

Jul 31, 2026, cs.IRJ/K move · Enter open · S save

RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems

Haoran Ling, Yuecheng Li, Zeyu Song, Jing Yao+4

Georgia Institute of Technology · Kuaishou Technology, China

Prompt Optimization Bandits

Open paper AskPDF arXiv

Optimizing modern recommender models still depends heavily on engineers manually iterating over architectural, objective, and training-strategy changes. While LLM-based agents can automate this trial-and-error process, allowing the LLM to both select modification directions and generate concrete hypotheses often leads to unstable search under limited experiment budgets. Inspired by the above challenge, we propose RecHarness, a Bandit-Routed Agentic Harness for automated recommender model optimization. RecHarness separates the optimization process into two steps: a bandit router selects the next modification direction according to historical validation feedback, while the LLM generates a concrete optimization hypothesis and executable code edit within the selected direction. To sustain long-horizon exploration, RecHarness uses a jump-basin mechanism to activate a structural-jump arm when local edits stagnate. Across multiple recommendation tasks, datasets, and model backbones, RecHarness achieves more stable performance improvements and uses limited trial budgets more effectively than LLM-reasoning search. During a 7-day online A/B test on a large-scale short-video advertising platform, the selected candidate improves ADVV by 2.084%, Revenue by 0.534%, and Exposure by 0.559%. Code is available at https://github.com/6lyc/RecHarness.

Abstract

Explore similar work

RecHarness: A Bandit-Routed Agentic Harness for Self-Evolving Recommender Systems

LeAP: Learnable Adaptive Permutation for Feature Selection in Heterogeneous and Sparse Recommender Systems

Delightful Exploration