Deep Bootstrap
Organizations: Joint Laboratory of Data Science and Business Intelligence, Southwestern University of Finance and Economics, Chengdu, Sichuan 611130, China. · State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China. · School of Artificial Intelligence, Hubei Key Laboratory of Computational Science, Wuhan University, Wuhan 430072, China. · Institute for Math and AI, Hubei Key Laboratory of Computational Science, School of Artificial Intelligence, Wuhan University, Wuhan 430072, China. ·
Abstract
In this work, we propose a novel deep bootstrap framework for nonparametric regression based on conditional diffusion models. Specifically, we construct a conditional diffusion model to learn the distribution of the response variable given the covariates. This model is then used to generate bootstrap samples by pairing the original covariates with newly synthesized responses. We reformulate nonparametric regression as conditional sample mean estimation, which is implemented directly via the learned conditional diffusion model. Unlike traditional bootstrap methods that decouple the estimation of the conditional distribution, sampling, and nonparametric regression, our approach integrates these components into a unified generative framework. With the expressive capacity of diffusion models, our method facilitates both efficient sampling from high-dimensional or multimodal distributions and accurate nonparametric estimation. We establish rigorous theoretical guarantees for the proposed method. In particular, we derive optimal end-to-end convergence rates in the Wasserstein distance between the learned and target conditional distributions. Building on this foundation, we further establish the convergence guarantees of the resulting bootstrap procedure. Numerical studies demonstrate the effectiveness and scalability of our approach for complex regression tasks.
Figures & tables
| Metrics | Mathematical formulation |
|---|---|
| CP | |
| Interval Length |
| CP | Interval Length | |||
|---|---|---|---|---|
| (I) | 0.9480 | 0.0007 | 0.0011 | 0.1166 |
| (II) | 0.9452 | 0.0137 | 0.0542 | 0.2895 |
| CP | Interval Length | |||
|---|---|---|---|---|
| (I) | 0.9464 | 0.0006 | 0.0011 | 0.1185 |
| (II) | 0.9380 | 0.0015 | 0.0019 | 0.1581 |
| (III) | 0.9424 | 0.0054 | 0.0092 | 0.2777 |
| (I) | (II) | (I) | (II) | (III) | |
| dimension | 5 | 5 | 10 | 10 | 10 |
| 0.05 | 0.05 | 0.05 | 0.05 | 0.05 | |
| test set size | 2500 | 2500 | 2500 | 2500 | 2500 |
| test set ratio | 0.02 | 0.02 | 0.025 | 0.0125 | 0.0125 |
| hidden layers | [48,48] | [56,56] | [48,48] | [56,56] | [64,64] |
Nonparametric Regression
Nonparametric regression estimates flexible relationships between predictors and outcomes without fixing a finite-dimensional model in advance.
See all 56 papers