stat.MLNov 13, 2026

Taking Band-Limited Maximum Likelihood to High Dimensions: Linear Memory, the Parametric Rate, and a Self-Tuning Cut-off

Authors: Rahul Agarwal, Ankit Agarwal

Organizations: Independent researcher (Ph.D., Biomedical Engineering, Johns Hopkins University) Sammamish, WA, USA · Independent researcher (M.B.A., University of Southern California) San Jose, CA, USA

Abstract

Band-limited maximum likelihood (BLML) estimates a density as the square of a band-limited function. The estimate is non-negative, and when the square root of the true density is band-limited its error falls at the parametric rate n−1n^{-1}. Its algorithm bins the data, which is fast only in one or two dimensions: from three on every point has its own bin, the kernel system is dense, and memory grows as n2n^2, 40 GB at 10510^5 points. Whether the rate survives in higher dimension, where kernel estimators slow down, had not been tested. We study dense data, densities that fill their space, not data near a low-dimensional manifold. We show that the binned equations are the stationarity condition of a strictly convex objective and minimise it by block coordinate descent without forming the kernel matrix: memory is linear in nn, a sweep costs O(n2d)O(n^2 d) and a fit takes about ten, and one A100 GPU fits 10610^6 points in eight dimensions in about ten minutes. On band-limited truths the error then decays as n−1n^{-1} in every dimension we tested, up to eight, while the slopes of kernel estimators flatten from -0.67 to -0.28, and it is 2 to 18 times lower than a neural spline flow's. A streaming fit learns the cut-off, BLML's only tuning parameter, from analytic gradients of the held-out likelihood, for about the cost of one more fit. On real data BLML is the strongest nonparametric estimator we tested: it leads kernel and nearest-neighbour estimators on electricity prices by 0.27 to 0.47 nats per hour, and a tuned kernel estimator on the full POWER and GAS benchmarks by 0.39 and 1.58 nats per point, placing it between early and recent normalizing flows. It loses on densities with a kink.

Figures & tables

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Jul 29, 2026

Breaking the Curse with BAND: Nonparametric Distribution Estimation in High Dimensions

Minimax-optimal rates for multivariate distribution estimation are known to suffer from the curse of dimensionality. We propose a sparse Bayesian network approach in which each conditional probability is estimated using sparsity-aware conditional mean methods. The resulting estimator, BAyesian Network Distribution regression (BAND), handles mixed data types in high-dimensional time series and achieves polynomial total variation convergence rates while allowing the feature dimension to grow polynomially with the sample size. These rates are substantially faster than the classical optimal rates for multivariate histogram density estimators that lack sparsity. Empirical evaluations show that BAND performs competitively for data sampling and confidence region forecasting against a range of state-of-the-art benchmarks.
May 13, 2026

Adaptive Kernel Density Estimation with Pre-training

Density estimation in high-dimensional settings is an important and challenging statistical problem.Traditional methods based on kernel smoothing are inefficient in high dimensions due to the difficulties in specifying appropriate location-adaptive kernels. In this work, we introduce pre-training, a key idea behind many cutting-edge AI technologies, to the context of non-parametric density estimation. By establishing a pre-trained neural network that can recommend an appropriate location-adaptive kernel for each sample point, efficient density estimation with adaptive kernels is achieved in high dimensions. A wide range of numerical experiments show that this strategy is highly effective for improving density-estimation accuracy, when the target distribution is close to the distribution family for pre-training. When the target distribution is substantially different from the pre-training distribution family, the benefit from the proposed pre-training strategy may be diluted, but can be reactivated by an additional fine-tuning procedure.
Jun 22, 2026

Exact Schur-Sylvester Dimensionality Reductions for Non-Smooth Stochastic Complexity and Manifold Sampling

The exact computation of the Normalized Maximum Likelihood (NML) codelength for regular non-smooth estimators (e.g., Lasso) has been historically limited by the cubic scaling walls of manifold-constrained projection and volume integration. At each step of the geometric Propose-and-Project Metropolis--Hastings (PPMH) sampler, evaluating the projection operator requires inverting an (N+k)×(N+k)(N+k) \times (N+k) generalized KKT matrix, while calculating the volume factor requires the determinant of an (N−k)×(N−k)(N-k) \times (N-k) Gram matrix. This paper presents an exact, mathematically equivalent formulation that bypasses both bottlenecks by utilizing the block Schur complement and Sylvester's determinant identity. We prove that the computational complexity of both operations collapses from O(N3)\mathcal{O}(N^3) to O(k3+N2k)\mathcal{O}(k^3 + N^2 k) per step. We generalize this reduction to Sparse Support Vector Machines (SVMs), Elastic Net, and Group Lasso. Finally, we provide a rigorous numerical stability analysis and evaluate the sampler's efficiency using the Effective Sample Size (ESS) per second. Our empirical benchmarks on high-dimensional datasets confirm a constant speedup exceeding 14,100×14{,}100\times while maintaining double-precision numerical equivalence, rendering exact non-smooth NML estimation highly tractable for large-scale statistical inference.