DOT: Dynamic Knob Selection and Online Sampling for Automated Database Tuning
Authors: Yifan Wang, Debabrota Basu, Pierre Bourhis, Romain Rouvoy, Patrick Royer
Organizations: Orange / Inria / Univ. Lille, France France · Inria / Univ. Lille / CNRS, France France · CNRS / Univ. Lille / Inria, France France · Univ. Lille / CNRS / Inria, France France · Orange, France France
Database Management Systems (DBMS) are crucial for efficient data management and access control, but their administration remains challenging for Database Administrators (DBAs). Tuning, in particular, is known to be difficult. Modern systems have many tuning parameters, but only a subset significantly impacts performance. Focusing on these influential parameters reduces the search space and optimizes performance. Current methods rely on costly warm-up phases and human expertise to identify important tuning parameters. In this paper, we present DOT, a dynamic knob selection and online sampling DBMS tuning algorithm. DOT uses Recursive Feature Elimination with Cross-Validation (RFECV) to prune low-importance tuning parameters and a Likelihood Ratio Test (LRT) strategy to balance exploration and exploitation. For parameter search, DOT uses a Bayesian Optimization (BO) algorithm to optimize configurations on-the-fly, eliminating the need for warm-up phases or prior knowledge (although existing knowledge can be incorporated). Experiments show that DOT achieves matching or outperforming performance compared to state-of-the-art tuners while substantially reducing tuning overhead.
Figures & tables
Benchmark
Type
Mean
SD
CV (%)
TPC-H
OLAP
80.13 sec.
0.42
0.52
TPC-C
OLTP
79.33 TPS
4.95
6.24
Sysbench
OLTP
608.63 TPS
26.53
4.36
Table 1. Variability of performance measurements
Benchmark
Iterations
Time (hours)
Mean
SD
CV (%)
Mean
SD
CV (%)
TPC-H
57.0
41.3
72.5
1.21
0.9
72.9
TPC-C
215.6
122.2
56.7
6.9
4.4
64.3
Sysbench
118.4
40.3
34.1
3.4
1.2
34.6
Table 2. Convergence variability of the BO tuner: iterations & time to reach near-optimal performance across benchmarks
Workload
Sensitivity
P & B
Lasso
CART
Experience
LLM
Sysbench (TPS)
1411.1
939.1
1469.04
1588.7
1294.36
1411.1
TPC-H (Exec. s)
46.22
74.35
46.02
45.76
45.72
46.35
Table 3. Performance tuned with TOP-5 knobs selected by different methods
Figure 1. Variability of CART knob selection across varying sample sizes & sampling strategies
Samples
Rank (Max TPS)
Rank (tuning time [min])
Sampling
Test stat: 7.32
Test stat: 1.32
time (min)
p-value: 6.2E−2
p-value: 7.2E−1
520
1.8 (1,708.0 ± 29.1)
2.8 (216.2 ± 32.6)
866,7
300
2.0 (1,684.3 ± 48.9)
2.0 (160.8 ± 79.5)
500,0
200
2.4 (1,690.8 ± 65.2)
2.8 (214.9 ± 116.9)
333.3
100
3.8 (1,602.3 ± 59.5)
2.4 (181.3 ± 36.1)
166.7
Table 4. Sysbench tuning with CART on knob sets from varying sample sizes (statistically significant at p< 5E-2)
Sysbench
TPC-C
TPC-H
EXP
Sysbench
—
0.317±0.035
0.325±0.033
0.258±0.016
TPC-C
0.317±0.035
—
0.472±0.041
0.243±0.029
TPC-H
0.325±0.033
0.472±0.041
—
0.409±0.024
EXP
0.258±0.016
0.243±0.029
0.409±0.024
—
Table 5. Jaccard index between knob sets.
Rank (Max TPS)
Rank (tuning time [min])
Test stat: 12.7, p-value: 2.7E-2
Test stat: 17.8, p-value: 3E-3
Sysbench
2.4 (1,708.0 ± 29.1)
2.4 (216.2 ± 32.6)
TPC-C
4.0 (1,653.2 ± 27.2)
3.0 (218.4 ± 65.3)
TPC-H
4.0 (1,638.8 ± 35.2)
2.8 (188.3 ± 57.9)
EXP
1.4 (1,725.2 ± 31.1)
2.6 (217.8 ± 42.7)
Shared
3.2 (1,677.1 ± 35.2)
5.8 (447.3 ± 26.2)
Table 6. Sysbench tuning with knob sets for different workloads ranked with CART (statistically significant at p< 5E-2)
Sysbench
TPC-C
TPC-H
Rank (Max TPS) Test stat: 13.3 p-value: 1E-2
Rank (tuning time [mins]) Test stat: 9.1 p-value: 5.8E-2
Rank (Max TPS) Test stat: 12.0 p-value: 1.7E-2
Rank (tuning time [mins]) Test stat: 5.1 p-value: 2.8E-1
Rank (Exec (s)) Test stat: 6.1 p-value: 1.9E-1
Rank (tuning time [mins]) Test stat: 11.8 p-value: 1.9E-2
Top 5 knobs
4.6 (1,486.8 ± 159.0)
3.0 (311.5 ± 194.2)
5.0 (120.5 ± 22.1)
2.4 (151.7 ± 161.1)
3.6 (47.7 ± 0.3)
1.6 (46.1 ± 54.5)
Top 10 knobs
4.2 (1,674.9 ± 34.4)
2.6 (272.0 ± 122.8)
3.2 (338.5 ± 13.8)
2.0 (153.8 ± 186.7)
3.8 (47.7 ± 0.5)
2.0 (32.7 ± 14.6)
Top 20 knobs
2.0 (1,725.2 ± 31.1)
2.0 (217.8 ± 42.7)
1.8 (354.2 ± 14.8)
3.4 (411.9 ± 264.9)
3.2 (47.8 ± 1.4)
3.2 (73.0 ± 53.2)
Top 30 knobs
2.0 (1,728.4 ± 46.3)
2.6 (267.9 ± 136.3)
2.4 (347.6 ± 25.8)
3.2 (261.0 ± 287.1)
2.8 (47.8 ± 2.0)
3.6 (189.6 ± 286.5)
All 52 knobs
2.2 (1,720.0 ± 33.0)
4.8 (732.6 ± 266.7)
2.6 (346.0 ± 7.5)
4.0 (560.4 ± 327.7)
1.6 (46.5 ± 0.6)
4.6 (334.1 ± 175.5)
Table 7. Performance, tuning time, and Friedman-test ranks for different numbers of knobs (statistically significant at p < 5e-2)
Figure 2. DOT overview
Figure 3. MAPE of Partial vs. Full Benchmark Results
Figure 4. MAPE vs. Benchmark-Query Coverage
Figure 5. Performance vs. tuning-time of different algorithms on Sysbench , TPC-C, TPC-H with knob importance knowledge
Figure 6. Performance vs. tuning-time of different algorithms on Sysbench , TPC-C, TPC-H without knob importance knowledge
Sysbench
TPC-C
TPC-H
Rank (Max TPS) Test stat: 37.4 p-value: 2.2E-05
Rank (tuning time [mins]) Test stat: 32.2 p-value: 1.9E-04
Rank (Max TPS) Test stat: 31.4 p-value: 2.5e-04
Rank (tuning time [mins]) Test stat: 37.9 p-value: 1.7E-05
Rank (exec [s]) Test stat: 18.382 p-value: 3.1e-02
Rank (tuning time [mins]) Test stat: 38.4 p-value: 1.5E-05
DOT +EXP
2.6 (1728.3 ± 28.3)
2.0 (57.7 ± 28.2)
1.2 (377.7 ± 16.9)
2.4 (121.6 ± 72.1)
4.4 (46.6 ± 2.2)
3.4 (57.6 ± 54.3)
DOT +RAN
3.0 (1724.7 ± 50.6)
4.2 (171.1 ± 121.2)
2.0 (370.8 ± 13.6)
3.8 (187.3 ± 107.5)
2.6 (45.5 ± 1.3)
2.0 (33.9 ± 63.9)
LLM knowledge based
8.0 (1489.1 ± 57.5)
3.4 (129.3 ± 99.1)
9.2 (264.9 ± 10.1)
5.4 (288.0 ± 207.9)
6.6 (47.0 ± 0.4)
2.0 (21.1 ± 10.0)
BO+CART
3.6 (1708.0 ± 29.1)
10.0 (1080.3 ± 32.6)
5.0 (356.1 ± 9.4)
9.8 (960.4 ± 57.6)
3.6 (46.4 ± 0.5)
10.0 (1027.4 ± 106.6)
CDBTune cold+EXP
8.6 (1358.1 ± 152.9)
3.2 (218.6 ± 310.1)
8.2 (295.2 ± 51.7)
1.2 (33.5 ± 40.9)
-
-
Table 8. Performance, tuning time, and Friedman-test ranks for different methods (statistically significant at p< 5E-2)
Figure 7. DOT behavior on Sysbench workload with and without knob importance prior
Figure 8. Tuning Performance of DOT on periodic workloads composed of TPC-C and Sysbench
Figure 9. Max TPS Across Configurations and Workloads
Figure 10. Tuning Performance of DOT on Sysbench for various configurations without knob importance knowledge
Figure 11. Tuning Performance of DOT on Sysbench for various k0 without knob importance knowledge
Figure 12. Tuning Performance of DOT on TPC-C for various Tcut budget with knob importance knowledge
Selecting appropriate values for the configurable parameters of Database Management Systems (DBMS) to improve performance is a significant challenge. Recent machine learning (ML)-based tuning systems have shown strong potential, but their practical adoption is often limited by the high tuning cost. This cost arises from two main factors: (1) the system needs to evaluate a large number of configurations to identify a satisfactory one, and (2) for each configuration, the system must execute the entire target workload on the DBMS, which is both time-consuming. Existing studies have primarily addressed the first factor by improving sample efficiency, that is, by reducing the number of configurations evaluated. However, the second factor, improving runtime efficiency by reducing the time required for each evaluation, has received limited attention and remains an underexplored direction. We develop WAter, a runtime-efficient and workload-adaptive tuning system that finds near-optimal configurations at a fraction of the tuning cost compared with state-of-the-art methods. We divide the tuning process into multiple time slices and evaluate only a small subset of queries from the workload in each slice. Different subsets are evaluated across slices, and a runtime profile is used to dynamically identify more representative subsets for evaluation in subsequent slices. At the end of each time slice, the most promising configurations are evaluated on the original workload to measure their actual performance. Evaluations demonstrate that WAter identifies the best-performing configurations with up to 73.5% less tuning time and achieves up to 16.2% higher performance than the best-performing alternative.
Yibo Wang, Jiale Lao, Chen Zhang +3
Purdue University, West Lafayette, Indiana, USA · Cornell University, Ithaca, New York, USA · Sichuan University, Chengdu, China
Modern DBMSs expose hundreds of configuration knobs, resulting in a high-dimensional and heterogeneous search space that makes automated tuning costly. Existing ML-based tuning systems typically treat the configuration domain as box-constrained and rely on workload feedback to implicitly capture inter-knob relationships. However, DBMS documentation specifies deterministic knob dependency constraints, particularly ordering constraints, that characterize structurally valid regions of the configuration space. We present CATune, a constraint-aware Bayesian optimization (BO) framework that models deterministic inter-knob ordering constraints as structural components of the search domain. Instead of learning feasibility boundaries through sampled violations, CATune performs optimization within a constraint-consistent subspace. We develop a topology-aware sampling strategy that respects dependency structure during exploration and avoids the inefficiencies of post-hoc constraint handling. To enable automated constraint discovery, we further design a precision-first extraction pipeline that combines LLM-based parsing with reliability safeguards to mitigate hallucinated dependencies. Experiments on PostgreSQL and MySQL using TPC-C and TPC-H workloads show that CATune substantially improves both sample efficiency and final tuning quality across surrogate models and BO frameworks. Under default ranges, CATune reaches the baseline optimum up to 12.5x faster and improves throughput by up to 63.37%. The improvements persist under knowledge-guided reduced ranges and alternative optimization implementations. These results demonstrate that explicitly modeling system-defined deterministic ordering constraints enhances optimization robustness and system stability.
Documentation has long guided computer system tuning by distilling expert knowledge into per-parameter recommendations. Yet such guides capture only what experts conclude, discarding how they reason. This fundamental gap manifests in three concrete deficiencies: documentation grows stale as software evolves, fails under heterogeneous workloads, and ignores inter-parameter dependencies. We propose shifting from static documentation to dynamic action for system tuning. We introduce PerfEvolve, which translates expert tuning methodologies into executable skills that equip LLM-based agents to perform version-consistency verification, workload-specific profiling, and multi-parameter joint optimization. Evaluated on PostgreSQL under TPC-C and TPC-H benchmarks, PerfEvolve outperforms state-of-the-art documentation-driven tuning baselines by up to 35.2%. The tool is available at https://github.com/ISCAS-OSLab/PerfEvolve.
Hongyu Lin, Mingyu Li, Weichen Zhang +4
Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Key Laboratory of System Software (Chinese Academy of Sciences) +3