AdaBox: Adaptive Density-Based Box Clustering with Parameter Generalization
Organizations: Independent Researcher
Abstract
Density-based clustering algorithms like DBSCAN and HDBSCAN are foundational tools for discovering arbitrarily shaped clusters, yet their practical utility is undermined by acute hyperparameter sensitivity -- parameters tuned on one dataset frequently fail to transfer to others, requiring expensive re-optimization for each deployment. We introduce AdaBox (Adaptive Density-Based Box Clustering), a grid-based density clustering algorithm designed for robustness across diverse data geometries. AdaBox features a six-parameter design where parameters capture cluster structure rather than pairwise point relationships. Four parameters are inherently scale-invariant, one self-corrects for sampling bias, and one is adjusted via a density scaling stage, enabling reliable parameter transfer across 30-200x scale factors. AdaBox processes data through five stages: adaptive grid construction, liberal seed initialization, iterative growth with graduation, statistical cluster merging, and Gaussian boundary refinement. Comprehensive evaluation across 111 datasets demonstrates three key findings: (1) AdaBox significantly outperforms DBSCAN and HDBSCAN across five evaluation metrics, achieving the best score on 78% of datasets with p < 0.05; (2) AdaBox uniquely exhibits parameter generalization. Protocol A (direct transfer to 30-100x larger datasets) shows AdaBox maintains performance while baselines collapse. (3) Ablation studies confirm the necessity of all five architectural stages for maintaining robustness.
Figures & tables
| Method | Arbitrary Shapes | Noise Handling | Variable Density | Params | Generalization |
|---|---|---|---|---|---|
| DBSCAN | Yes | Yes | Partial | 2 | No |
| HDBSCAN | Yes | Yes | Yes | 2–3 | No |
| OPTICS | Yes | Yes | Yes | 2–3 | No |
| DENCLUE | Yes | Partial | Yes | 2+ | No |
| DPC | Yes | Partial | Partial | 2 | No |
| AdaBox | Yes | Yes | Yes | 6 | Yes |
| Parameter | Description | Stage |
|---|---|---|
| n_boxes | Grid resolution (cells per dimension) | Grid Construction |
| min_density | Minimum cell density for clustering | Seed & Growth |
| regular_threshold_factor | Growth threshold multiplier | Iterative Growth |
| merge_adjacent | Enable statistical merging | Cluster Merging |
| refinement_sigma | Gaussian smoothing bandwidth | Boundary Refinement |
| min_cluster_size | Minimum cluster size filter | Post-processing |
| Metric | AdaBox | DBSCAN | HDBSCAN | -value |
|---|---|---|---|---|
| ARI | 0.724 | 0.640 | 0.676 | |
| NMI | 0.726 | 0.674 | 0.695 | |
| AMI | 0.724 | 0.670 | 0.692 | |
| V-Measure | 0.726 | 0.674 | 0.695 | |
| Fowlkes-Mallows | 0.752 | 0.678 | 0.713 |
| Algorithm | Sample ARI | Full ARI | ARI | % Change |
|---|---|---|---|---|
| AdaBox | 0.724 | 0.778 | +0.053 | +7.3% |
| DBSCAN | 0.575 | 0.176 | ||
| HDBSCAN | 0.697 | 0.204 |
| Algorithm | Datasets Tested | Mean ARI | Mean ARI at 100K | Transfer Success |
|---|---|---|---|---|
| AdaBox | 10/10 | 0.733 | 10/10 (100%) | |
| DBSCAN | 2/10 | 0.193 | 1/10 (10%) | |
| HDBSCAN | 4/10 | 0.522 | 2/10 (20%) |
| Component Removed | Mean ARI | ARI | % Change | -value |
|---|---|---|---|---|
| Full AdaBox | 0.880 | — | — | — |
| No Adaptive Merging | 0.695 | |||
| No Boundary Refinement | 0.800 | |||
| Fixed Grid | 0.716 |