This work presents a game-theoretic framework for interpretable hyperparameter-objective interaction analysis rather than proposing a new optimization algorithm. In the proposed framework, Shapley Effects are employed for global sensitivity analysis, while Pareto front sets are utilized to identify effective hyperparameter configurations and support early-stage model evaluation. The resulting analysis reveals which players (hyperparameters) are most influential with respect to different objectives in a given game (application). Consequently, the proposed framework provides interpretable insights into objective-aware hyperparameter interactions, enabling practitioners to guide subsequent optimization, reduce the search space, and perform early-stage model evaluation. The effectiveness of the proposed framework is demonstrated using three distinct neural network architectures across different problem domains under multi-objective settings.
Hyperparameter Optimization (HPO) is essential for building high-performing ML/DL models, yet conventional optimizers often struggle in high-dimensional spaces where evaluations are costly and progress is diluted across many low-impact variables. We propose Greedy Importance First (GIF), an importance-aware scheduling strategy that uses a small-sample warm start to estimate hyperparameter importance, forms importance-based groups, allocates trials proportionally, and retains a full-space fallback. We evaluate GIF under fixed evaluation budgets on five anisotropic analytic functions, Bayesmark, and NAS-Bench-301. On the higher-dimensional benchmarks, GIF reaches better incumbents with faster convergence than TPE, BOHB, Random Search, and Sequential Grouping. On Bayesmark, where the effective dimensionality is smaller, GIF remains competitive but the margins are smaller. Ablation studies show that importance estimation, proportional allocation, and the fallback step all contribute to the gains. We also verify that the HIA component recovers the intended anisotropy on the analytic benchmarks. These results suggest that GIF is a simple and plug-compatible way to improve sample efficiency in high-dimensional HPO.
Interpretability aims to explain the behavior of deep neural networks. Despite rapid growth, there is mounting concern that much of this work has not translated into practical impact, raising questions about its relevance and utility. This position paper argues that the central missing ingredient is not new methods, but evaluation criteria: interpretability should be evaluated by actionability--the extent to which insights enable concrete decisions and interventions beyond interpretability research itself. We define actionable interpretability along two dimensions--concreteness and validation--and analyze the barriers currently preventing real-world impact. To address these barriers, we identify five domains where interpretability offers unique leverage and present a framework for actionable interpretability with evaluation criteria aligned with practical outcomes. Our goal is not to downplay exploratory research, but to establish actionability as a core objective of interpretability research.
We study hyperparameter optimization (HPO) from a numerical-optimization perspective and propose a multi-objective damped Newton--Gauss--Newton search method. Rather than perturbing each hyperparameter separately or treating model evaluations as independent trials, the method uses performance changes between successive full hyperparameter vectors to construct an iterative secant approximation of the local sensitivity matrix. Each iteration therefore requires only one new full-vector model evaluation while jointly updating all optimized hyperparameters. A Tikhonov-regularized Gauss--Newton system stabilizes the update when the number of hyperparameters exceeds the number of performance objectives. The search is initialized from readily available empirical/default settings of the underlying learner, without requiring a separate initialization search. We evaluate four-dimensional XGBoost HPO on three public classification datasets against exhaustive grid search, random search, and tree-structured Parzen estimator (TPE) optimization. On Breast Cancer Wisconsin, the proposed method matches the best validation accuracy of a 320-configuration grid search with slightly better log loss and ROC--AUC. Across three datasets and three seeds, predictive performance remains competitive with random search and TPE while using substantially fewer search iterations. A supplementary proprietary eight-dimensional threshold-optimization case study further demonstrates joint multi-parameter optimization under competing performance targets and reveals non-monotonic, oscillatory trajectories that motivate damping and best-iterate selection. Overall, the results establish iterative secant sensitivity as an evaluation-efficient local alternative to exhaustive HPO.