Kernel Debiased Plug-in Estimation based on the Universal Least Favorable Submodel

Haiyi Chen; Yang Liu; Ivana Malenica

math.STMar 9, 2026

Kernel Debiased Plug-in Estimation based on the Universal Least Favorable Submodel

Authors: Haiyi Chen, Yang Liu, Ivana Malenica

Organizations: Department of Biostatistics, UNC at Chapel Hill, USA · Department of Statistics & Operations Research, UNC at Chapel Hill, USA

Abstract

We propose ULFS-KDPE, a kernel debiased plug-in estimator based on the universal least favorable submodel, for estimating pathwise differentiable parameters in nonparametric models. The method constructs a data-adaptive debiasing flow in a reproducing kernel Hilbert space (RKHS), producing a plug-in estimator that achieves semiparametric efficiency without requiring explicit derivation or evaluation of efficient influence functions. We place ULFS-KDPE on a rigorous functional-analytic foundation by formulating the universal least favorable update as a nonlinear ordinary differential equation on probability densities. We establish existence, uniqueness, stability, and finite-time convergence of the empirical score along the induced flow. Under standard regularity conditions, the resulting estimator is regular, asymptotically linear, and attains the semiparametric efficiency bound simultaneously for a broad class of pathwise differentiable parameters. The method admits a computationally tractable implementation based on finite-dimensional kernel representations and principled stopping criteria. In finite samples, the combination of solving a rich collection of score equations with RKHS-based smoothing and avoidance of direct influence-function evaluation leads to improved numerical stability. Simulation studies illustrate the method and support the theoretical results.

Figures & tables

	Parameter	Method	#Cov.	Bias( $\times 100$ )	Var a	RMSE
		ULFS-KDPE	475	-0.824	0.0561	0.0567
DGP1	$\psi_{ATE}$	KDPE	343	-1.438	0.0546	0.0565
		TMLE	500	-0.004	0.0618	0.0618
		One-step TMLE	500	-0.247	0.0605	0.0606
		ULFS-KDPE	475	-1.082	0.1724	0.1728
	$\psi_{RR}$	KDPE	343	-3.213	0.1584	0.1616

Table 1. Bias, Convergence with 100 Steps, Variance and Root Mean Squared Error (RMSE) for ATE, RR and OR with different estimators (ULFS-KDPE, KDPE, TMLE, One-step TMLE).

Figure 1. Finite-sample behavior of the different ATE estimators (ULFS-KDPE, KDPE, TMLE, One-step TMLE). True asymptotic distribution is depicted in purple.

	Parameter	Stopping Criterion	#Cov.	Bias ( $\times 100$ )	Var a	RMSE
		Density stabilization	500	-0.840	0.0553	0.0560
DGP1	$\psi_{ATE}$	Empirical score stabilization	493	-0.806	0.0566	0.0572
		Vanishing update direction	500	-1.357	0.0547	0.0564
		Variance-dominated updates	500	-1.573	0.0550	0.0573
		EIF approximately solved	447	-1.156	0.0561	0.0573
		Density stabilization	500	-1.142	0.1700	0.1705

Table 2. Bias, Convergence with 200 Steps, Var, RMSE for ATE, RR and OR under different stopping criteria for ULFS-KDPE.

Figure 2. ULFS-KDPE under different stopping criteria. True asymptotic distribution is depicted in purple.

	Iteration Limit	Stopping Criterion	#Cov.	$\psi_{\text{ATE}}$	$\psi_{\text{RR}}$	$\psi_{\text{OR}}$
		Density plateau	475	0.0567	0.1728	0.3975
DGP1	100	Score plateau	404	0.0573	0.1676	0.3792
		Vanishing Direction	498	0.0563	0.1669	0.3784
		Variance dominate	500	0.0567	0.1684	0.3803
		EIF sovled	407	0.0547	0.1721	0.3943
		Density plateau	500	0.0560	0.1702	0.3896

Table 3. Convergence and RMSE for ATE, RR and OR with ULFS-KDPE under different iteration limits (# of steps allowed).

Semiparametric Inference

Semiparametric inference quantifies uncertainty for low-dimensional targets when models also contain flexible, infinite-dimensional nuisance functions.

See all 33 papers

Parametric Rates for Nonparametric Conditional-Moment Bilevel Optimization
May 20, 2026Fares El Khoury, Houssam Zenati, Nathan Kallus +2Semiparametric Inference Hypergradient Optimization
Université Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK · Gatsby Computational Neuroscience Unit, University College London · Cornell University +1
Debiased Prior-Data Fitted Networks for Amortized Bayesian Semi-Parametric Inference
Oct 20, 2026Valentyn Melnychuk, Emil Javurek, Stefan FeuerriegelPrior-Data Fitted Networks Amortized Inference
LMU Munich & Munich Center for Machine Learning, Munich, Germany
Prediction-Powered Causal Inference by Automatic Debiased Machine Learning and Semi-Supervised Riesz Regression
Jun 11, 2026Masahiro KatoSemiparametric Inference Parameter Estimation
Data Analytics Department, Mizuho-DL Financial Technology, Co., Ltd.

May 20, 2026J/K move · Enter open · S save

Parametric Rates for Nonparametric Conditional-Moment Bilevel Optimization

Fares El Khoury, Houssam Zenati, Nathan Kallus, Michael Arbel+1

Université Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK · Gatsby Computational Neuroscience Unit, University College London · Cornell University+1

Semiparametric Inference Hypergradient Optimization Bilevel Optimization Differentiable Optimization

Open paper AskPDF arXiv

We study functional bilevel optimization with conditional moments, where plug-in hypergradients can inherit first-order errors from nonparametrically learned conditional expectations. We introduce OBiGrad, an orthogonalized hypergradient estimator whose bias is a product of nuisance errors. Under our regression and complexity conditions, projected gradient descent with the direct plug-in hypergradient yields expected stationarity error

O(T^{-1/2}+n^{-1/2}+n^{-ν/b})

for the best iterate, whereas OBiGrad achieves

O(T^{-1/2}+n^{-1/2}+n^{-2ν/b})

, where

b=ν(2+p)+γ

combines the approximation and complexity exponents. Consequently, when

ν(2-p)\geγ

, OBiGrad attains the parametric statistical rate

O(T^{-1/2}+n^{-1/2})

, improving the sufficient sample-size bound relative to the direct plug-in bound for a fixed iteration budget. We illustrate these gains on instrumental-variable and fitted-Q-regression problems, examining stationary-point estimation, returned population stationarity, and regularization sensitivity.

Abstract

Semiparametric Inference

Explore similar work

Parametric Rates for Nonparametric Conditional-Moment Bilevel Optimization

Debiased Prior-Data Fitted Networks for Amortized Bayesian Semi-Parametric Inference

Prediction-Powered Causal Inference by Automatic Debiased Machine Learning and Semi-Supervised Riesz Regression