Domain Adaptation (DA) often lacks worst-case guarantees, while Distributionally Robust Optimization (DRO) based only on source data centers its ambiguity set at the source law and ignores available target structure. To bridge this gap, we introduce a doubly-anchored DRO framework whose ambiguity set is the intersection of φ-divergence balls centered at the source law and a source-completed target reference law, the latter pairing the target covariate law with the source conditional law. We derive dual-induced adversarial bridge geometries for symmetric and asymmetric divergence pairings, notably introducing a Kullback--Leibler/squared-Hellinger (KL/HD) bridge. This asymmetric formulation yields a Lambert-W geometry in which source-side exponential risk tilting and target-side Hellinger stabilization enter through distinct terms, attenuating, but not bounding, the effect of large likelihood ratios. Furthermore, without imposing covariate shift, we establish finite-sample generalization bounds for the minimizer of a structural, loss-agnostic density-bridge risk under bounded-overlap conditions; these bounds do not apply directly to the loss-aware DRO min--max estimator. We translate our framework into a bridge-weighted Nadaraya--Watson estimator, proving uniform consistency for the source regression function and pointwise asymptotic normality, with target recovery when the source and target regression functions coincide, as under covariate shift. Finally, an empirical evaluation on a domain-shifted Fashion-MNIST dataset illustrates the finite-sample stability of the asymmetric KL/HD bridge under severe synthetic target-covariate corruption.
Figures & tables
Figure 1: Scale-normalized bridge maps gαB(ϱ)/gαB(1) as a function of the likelihood ratio ϱ (top row) or as a function of the bridge parameter α (bottom row). For KL/HD, the scale-normalized map is extended to α=1 by its pointwise likelihood-ratio limit.
Figure 2: Target-domain images from the Fashion-MNIST ( Xiao et al., 2017 ) data set. Figure (a) shows 4 randomly sampled images from the target domain (two pullovers and two coats). The top row shows the original target images, whereas the bottom row shows the same images after an artificial domain shift. Figure (b) shows projections onto the first 2 principal components of the 10-dimensional latent representation used by the domain-adapted NW estimator. The top and bottom rows again show the original and shifted target domains, respectively. The gray points in the background show the source domain images, which are the same in both rows.
Figure 3: Sensitivity analysis and target accuracy of the domain-adapted NW estimators applied to the Fashion-MNIST data set with shifted target images. The left column uses loss-agnostic weights and the right uses the loss-aware iterative refinement; KL/HD has γ=0 . Rows 1–3 use one target-shift replication. The plug-in effective sample size in the first row is neff={∑i=1nw^θ(XiS)}2/∑i=1nw^θ(XiS)2≤12,000=n . Let pj denote the fitted Pullover probability for target image j . The summed binary entropy in the third row is Ew^=−∑j=1m[pjlog(pj)+(1−pj)log(1−pj)] . Here 0log0 is interpreted as zero. The bottom row gives the mean target accuracy over 100 independent redraws of the Laplace perturbations, with the images, source-trained autoencoder, and source–target split held fixed; error bars show plus or minus one standard deviation.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Exploratory sensitivity of the loss-aware refinement to ν , plotted against α for KL/KL (left), HD/HD (center), and KL/HD with γ=0 (right), using the single target-shift replication used for tuning. Rows report the plug-in effective sample size of the normalized source weights, the summed binary entropy of the fitted target probabilities, and the proportion of target images classified as Pullover. The HD/HD curves terminate when the interior-response check ν>maxi,tGh(t)(XiS) fails. The retained values in these panels were ν≈0.016 , ν=1+10−8 , and ν=0.1 , respectively.
Correlation alignment and the maximum mean discrepancy are two widely used distribution-matching frameworks for unsupervised domain adaptation (UDA). However, high variance in these losses has been shown to undermine their effectiveness in minibatch optimisation settings. Furthermore, the losses lack finite-sum structure, which renders them incompatible with classical stochastic variance reduction (SVR) methods. This paper proposes Paired Sampling for Domain Adaptation (PSDA), a novel SVR technique tailored to such objectives. PSDA pairs observations both within and across domains, to form quadruplets that are always sampled together during training. The pairings are designed to minimise expected gradient variance, and reduce to solving a set of linear assignment problems. Our simulations demonstrate reduced variance compared to related methods, and experiments on three domain shift datasets show improved target domain accuracy.
Andrea Napoli
Department of Electronics and Computer Science University of Southampton, UK
We propose a unified framework for addressing three key challenges of distribution shift: (1) estimating a model's performance on an unlabeled target domain, (2) explaining the shift by identifying the features responsible, and (3) improving the target domain performance. Our method, Entropic Projection Alignment (EPA), aligns the source distribution to the target by matching carefully selected moments while simultaneously minimising the KL divergence from the source. This formulation yields a unique closed-form solution for importance weights, achieving robustness through implicit variance control. Drawing on domain adaptation theory, we establish that moment matching is sufficient for reliable estimation and adaptation, avoiding the need for full density ratio recovery. Extensive experiments, together with strong theoretical guarantees, demonstrate that EPA consistently outperforms state-of-the-art baselines while offering substantial computational efficiency.
We introduce and analyze Target-Induced Loss Tilting (TILT) for unsupervised domain adaptation under covariate shift. It is based on a novel objective function that decomposes the source predictor as f+b, fits f+b on labeled source data while simultaneously penalizing the auxiliary component b on unlabeled target inputs. The resulting fit f is deployed as the final target predictor. At the population level, we show that this target-side penalty implicitly induces relative importance weighting at the population level, but in terms of an estimand bf∗ that is self-localized to the current error, and remains uniformly bounded for any source-target pair (even those with disjoint supports). We prove a general finite-sample oracle inequality on the excess risk, and use it to give an end-to-end guarantee for training with sparse ReLU networks. Experiments on controlled regression problems and shifted CIFAR-100 distillation show that TILT improves target-domain performance over source-only training, exact importance weighting, and relative density-ratio baselines, with a stable dependence on the regularization parameter.
Kakei Yamamoto, Martin J. Wainwright
Lab for Information and Decision Systems · Statistics and Data Science Center · EECS† +2