Data Augmentation via Causal-Residual Bootstrapping

Mateusz Gajewski; Sophia Xiao; Bijan Mazaheri

cs.LGMar 16, 2026

Data Augmentation via Causal-Residual Bootstrapping

Authors: Mateusz Gajewski, Sophia Xiao, Bijan Mazaheri

Organizations: Poznań University of Technology, Poznań, Poland · AKCES NCBR, Warsaw, Poland · Thayer School of Engineering, Dartmouth, Hanover, NH, USA · Eric and Wendy Schmidt Center, Broad Institute of MIT and Harvard, Cambridge, MA, USA

Abstract

Data augmentation integrates domain knowledge into a dataset by making domain-informed modifications to existing data points. For example, image data can be augmented by duplicating images in different tints or orientations, thereby incorporating the knowledge that images may vary in these dimensions. Recent work by Teshima and Sugiyama has explored the integration of causal knowledge (e.g, A causes B causes C) up to conditional independence equivalence. We suggest a related approach for settings with additive noise that can incorporate information beyond a Markov equivalence class. The approach, built on the principle of independent mechanisms, permutes the residuals of models built on marginal probability distributions. Predictive models built on our augmented data demonstrate improved accuracy, for which we provide theoretical backing in linear Gaussian settings.

Figures & tables

Figure 1: Empirical validation of MSE improvement rate. Left: Simple chain $A\to B\to C$ . Right: Confounded structure $A\to B\leftarrow D$ , $B\to C$ . Both configurations show MSE improvement following the predicted $1/N$ decay. In both cases B is the predicted value

Panel of Figure 2 — Figure 2: Performance of the PC algorithm on datasets augmented by alternative methods. The plots show the average Structural Hamming Distance (SHD) between the true DAG and the estimated CPDAG returned by the PC algorithm as the number of augmented points increases; the shaded region indicates one standard deviation. Higher SHD indicates worse performance.

Panel of Figure 3 — Figure 3: Performance of DirectLINGAM algorithm.

Figure 4: Mean MSE across all variables by augmentation method (Known Graph,

n=100

). Lower is better. CRB achieves the best performance, while non-causal augmenters often increase error relative to no augmentation.

Figure 5: Per-variable MSE comparison (Known Graph,

n=100

Method	Best Var.	Improv.	Worst Var.	Worsen.
CRB	green	0.227	ir_3	0.006
ADMG-Tian	angle_1	0.000	l_21	0.169
ARF	l_12	0.118	angle_1	0.889
CTGAN	l_12	0.001	green	0.451
DDPM	red	0.016	l_11	0.244
TVAE	current	$-$ 0.024	green	0.221

Table 1: Best and worst per-variable performance relative to no augmentation. “Improv.” corresponds to MSE reduction (positive = better) on the best performing variable; “Worsen.” corresponds to MSE increase on the worst-performing variable.

Method	Boston	Sachs	Wine (red)	Wine (white)	Causal Chamber
ADMGTian	1.886 (1.45, 2.13)	0.596 (0.54, 0.67)	0.831 (0.79, 0.86)	0.874 (0.83, 0.93)	0.474 (0.47, 0.48)
ARF	0.625 (0.59, 0.70)	0.830 (0.81, 0.85)	0.981 (0.92, 1.02)	1.034 (1.03, 1.04)	0.738 (0.71, 0.76)
CTGAN	0.612 (0.58, 0.66)	0.876 (0.81, 0.95)	0.769 (0.73, 0.80)	0.912 (0.86, 1.02)	0.579 (0.55, 0.61)
CRB	0.457 (0.45, 0.47)	0.766 (0.55, 1.50)	0.635 (0.62, 0.65)	0.689 (0.67, 0.71)	0.383 (0.38, 0.38)
DDPM	0.371 (0.36, 0.39)	0.545 (0.48, 0.71)	0.586 (0.58, 0.60)	0.757 (0.73, 0.78)	0.483 (0.48, 0.49)
NFLOW	0.489 (0.48, 0.51)	0.783 (0.72, 0.85)	0.668 (0.63, 0.72)	0.799 (0.77, 0.84)	0.537 (0.50, 0.60)

Table 2: Mean MSE (

\downarrow

) across datasets with learned causal graph (100 samples). 95% CI in parentheses. Full results in Ap. D .

Figure 6: Mean MSE across all variables as sample size increases (Known Graph). Lower is better. CRB maintains strong performance across sample sizes.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Augmenter	angle_1	angle_2	blue	current	green	ir_1	ir_2	ir_3	l_11	l_12
ADMGTian	0.007	0.008	0.193	0.563	0.503	0.134	0.134	0.151	1.209	1.227
ARF	0.895	0.613	0.555	0.530	0.794	0.306	0.291	0.300	1.026	1.021
CTGAN	0.420	0.415	0.451	0.596	0.862	0.299	0.290	0.282	1.134	1.139
CRB	0.003	0.007	0.064	0.489	0.184	0.100	0.106	0.129	1.023	1.020
DDPM	0.019	0.011	0.120	0.559	0.412	0.106	0.101	0.116	1.295	1.279
None	0.006	0.007	0.127	0.501	0.410	0.122	0.108	0.123	1.051	1.139

Appendix

Table 3: Per-variable MSE by Augmenter (Known Graph) - Bold indicates best or statistically tied

Augmenter	Mean MSE $\downarrow$	$\alpha$ -Precision $\uparrow$	$\beta$ -Recall $\uparrow$	DCR $\uparrow$	$\delta$ -Presence $\uparrow$
ADMGTian	1.886 (1.43, 2.12)	0.327 (0.24, 0.48)	0.014 (0.01, 0.03)	0.126 (0.10, 0.15)	0.999 (1.00, 1.00)
ARF	0.625 (0.59, 0.70)	0.855 (0.84, 0.87)	0.045 (0.03, 0.06)	0.394 (0.37, 0.42)	1.000
CTGAN	0.612 (0.58, 0.66)	0.873 (0.84, 0.90)	0.065 (0.04, 0.08)	0.287 (0.26, 0.33)	1.000
CRB	0.457 (0.45, 0.47)	0.724 (0.70, 0.78)	0.114 (0.10, 0.13)	0.249 (0.24, 0.26)	1.000
DDPM	0.371 (0.36, 0.39)	0.886 (0.86, 0.91)	0.304 (0.29, 0.32)	0.119 (0.11, 0.13)	0.996 (0.99, 1.00)
NFLOW	0.489 (0.48, 0.51)	0.833 (0.77, 0.87)	0.097 (0.08, 0.12)	0.285 (0.25, 0.33)	1.000

Appendix

Table 4: Performance metrics for Boston dataset (100 samples) - Bold indicates best or statistically tied

Augmenter	Mean MSE $\downarrow$	$\alpha$ -Precision $\uparrow$	$\beta$ -Recall $\uparrow$	DCR $\uparrow$	$\delta$ -Presence $\uparrow$
ADMGTian	0.596 (0.54, 0.67)	0.925 (0.90, 0.94)	0.010 (0.01, 0.01)	0.020 (0.02, 0.03)	0.081 (0.06, 0.10)
ARF	0.830 (0.81, 0.85)	0.577 (0.52, 0.62)	0.010 (0.01, 0.01)	0.050 (0.04, 0.06)	0.515 (0.46, 0.57)
CTGAN	0.876 (0.81, 0.95)	0.870 (0.81, 0.91)	0.038 (0.03, 0.05)	0.016 (0.01, 0.02)	0.153 (0.10, 0.19)
CRB	0.766 (0.55, 1.58)	0.863 (0.80, 0.90)	0.027 (0.02, 0.03)	0.020 (0.02, 0.02)	0.153 (0.12, 0.20)
DDPM	0.545 (0.48, 0.72)	0.863 (0.81, 0.90)	0.057 (0.05, 0.06)	0.013 (0.01, 0.01)	0.244 (0.21, 0.30)
NFLOW	0.783 (0.72, 0.85)	0.816 (0.76, 0.86)	0.028 (0.02, 0.04)	0.022 (0.02, 0.02)	0.260 (0.22, 0.30)

Appendix

Table 5: Performance metrics for Sachs dataset (100 samples) - Bold indicates best or statistically tied

Augmenter	Mean MSE $\downarrow$	$\alpha$ -Precision $\uparrow$	$\beta$ -Recall $\uparrow$	DCR $\uparrow$	$\delta$ -Presence $\uparrow$
ADMGTian	0.831 (0.79, 0.86)	0.780 (0.72, 0.82)	0.020 (0.02, 0.03)	0.211 (0.20, 0.23)	0.643 (0.58, 0.72)
ARF	0.981 (0.92, 1.02)	0.814 (0.80, 0.83)	0.027 (0.02, 0.03)	0.303 (0.29, 0.31)	1.000
CTGAN	0.769 (0.73, 0.80)	0.826 (0.70, 0.88)	0.104 (0.08, 0.12)	0.217 (0.20, 0.26)	1.000 (1.00, 1.00)
CRB	0.635 (0.62, 0.65)	0.876 (0.85, 0.91)	0.141 (0.13, 0.15)	0.205 (0.20, 0.21)	1.000
DDPM	0.586 (0.58, 0.60)	0.846 (0.82, 0.87)	0.136 (0.13, 0.15)	0.160 (0.16, 0.16)	0.907 (0.89, 0.92)
NFLOW	0.668 (0.63, 0.72)	0.789 (0.68, 0.87)	0.096 (0.07, 0.11)	0.244 (0.23, 0.28)	1.000 (1.00, 1.00)

Appendix

Table 6: Performance metrics for Wine (red) dataset (100 samples) - Bold indicates best or statistically tied

Augmenter	Mean MSE $\downarrow$	$\alpha$ -Precision $\uparrow$	$\beta$ -Recall $\uparrow$	DCR $\uparrow$	$\delta$ -Presence $\uparrow$
ADMGTian	0.874 (0.83, 0.93)	0.831 (0.76, 0.88)	0.009 (0.01, 0.01)	0.175 (0.16, 0.21)	0.627 (0.57, 0.69)
ARF	1.034 (1.03, 1.04)	0.880 (0.75, 0.93)	0.014 (0.01, 0.02)	0.211 (0.20, 0.24)	1.000
CTGAN	0.912 (0.86, 1.03)	0.754 (0.67, 0.84)	0.038 (0.03, 0.05)	0.169 (0.15, 0.20)	1.000 (1.00, 1.00)
CRB	0.689 (0.67, 0.71)	0.878 (0.86, 0.90)	0.064 (0.06, 0.07)	0.147 (0.14, 0.16)	1.000
DDPM	0.757 (0.73, 0.78)	0.827 (0.79, 0.86)	0.047 (0.04, 0.05)	0.130 (0.12, 0.14)	0.884 (0.84, 0.93)
NFLOW	0.799 (0.77, 0.84)	0.848 (0.73, 0.92)	0.037 (0.03, 0.04)	0.180 (0.17, 0.20)	1.000

Appendix

Table 7: Performance metrics for Wine (white) dataset (100 samples) - Bold indicates best or statistically tied

Augmenter	Mean MSE $\downarrow$	$\alpha$ -Precision $\uparrow$	$\beta$ -Recall $\uparrow$	DCR $\uparrow$	$\delta$ -Presence $\uparrow$
ADMGTian	0.474 (0.47, 0.48)	0.925 (0.90, 0.94)	0.009 (0.01, 0.01)	0.606 (0.60, 0.62)	1.000
ARF	0.738 (0.72, 0.76)	0.937 (0.90, 0.96)	0.002 (0.00, 0.00)	0.779 (0.77, 0.79)	1.000
CTGAN	0.579 (0.55, 0.61)	0.914 (0.89, 0.93)	0.018 (0.01, 0.02)	0.670 (0.65, 0.69)	1.000
CRB	0.383 (0.38, 0.38)	0.944 (0.91, 0.96)	0.090 (0.09, 0.09)	0.573 (0.57, 0.58)	1.000
DDPM	0.483 (0.48, 0.49)	0.758 (0.74, 0.78)	0.037 (0.03, 0.04)	0.606 (0.60, 0.61)	1.000
NFLOW	0.537 (0.50, 0.60)	0.615 (0.46, 0.75)	0.011 (0.01, 0.02)	0.751 (0.71, 0.82)	1.000

Appendix

Table 8: Performance metrics for Causal Chambers dataset (100 samples) - Bold indicates best or statistically tied

Augmenter	Mean MSE $\downarrow$	$\alpha$ -Precision $\uparrow$	$\beta$ -Recall $\uparrow$	DCR $\uparrow$	$\delta$ -Presence $\uparrow$
ADMGTian	0.436 (0.39, 0.54)	0.869 (0.83, 0.90)	0.029 (0.02, 0.04)	0.016 (0.01, 0.02)	0.126 (0.11, 0.15)
ARF	0.628 (0.60, 0.69)	0.692 (0.56, 0.74)	0.102 (0.07, 0.11)	0.039 (0.03, 0.07)	0.371 (0.31, 0.55)
CTGAN	0.841 (0.71, 1.30)	0.812 (0.60, 0.89)	0.148 (0.10, 0.18)	0.025 (0.01, 0.07)	0.231 (0.15, 0.48)
CRB	0.372 (0.36, 0.38)	0.863 (0.84, 0.88)	0.141 (0.13, 0.15)	0.021 (0.02, 0.02)	0.166 (0.14, 0.20)
DDPM	0.400 (0.37, 0.43)	0.938 (0.93, 0.95)	0.341 (0.33, 0.35)	0.011 (0.01, 0.01)	0.121 (0.10, 0.14)
NFLOW	0.697 (0.65, 0.78)	0.910 (0.83, 0.94)	0.200 (0.17, 0.21)	0.018 (0.01, 0.03)	0.164 (0.12, 0.26)

Appendix

Table 9: Performance metrics for Sachs dataset (500 samples) - Bold indicates best or statistically tied

Augmenter	Mean MSE $\downarrow$	$\alpha$ -Precision $\uparrow$	$\beta$ -Recall $\uparrow$	DCR $\uparrow$	$\delta$ -Presence $\uparrow$
ADMGTian	1.210 (1.14, 1.32)	0.283 (0.24, 0.32)	0.016 (0.01, 0.02)	0.205 (0.19, 0.23)	0.734 (0.69, 0.80)
ARF	0.597 (0.58, 0.62)	0.872 (0.86, 0.89)	0.278 (0.26, 0.29)	0.260 (0.25, 0.27)	1.000
CTGAN	0.624 (0.60, 0.64)	0.919 (0.89, 0.95)	0.375 (0.35, 0.40)	0.207 (0.20, 0.22)	1.000 (1.00, 1.00)
CRB	0.494 (0.47, 0.53)	0.835 (0.81, 0.86)	0.438 (0.43, 0.45)	0.203 (0.20, 0.21)	1.000
DDPM	0.411 (0.40, 0.42)	0.912 (0.88, 0.94)	0.509 (0.50, 0.52)	0.168 (0.16, 0.18)	0.997 (0.99, 1.00)
NFLOW	0.578 (0.55, 0.64)	0.921 (0.89, 0.94)	0.391 (0.37, 0.41)	0.212 (0.20, 0.22)	1.000

Appendix

Table 10: Performance metrics for Wine (red) dataset (500 samples) - Bold indicates best or statistically tied

Augmenter	Mean MSE $\downarrow$	$\alpha$ -Precision $\uparrow$	$\beta$ -Recall $\uparrow$	DCR $\uparrow$	$\delta$ -Presence $\uparrow$
ADMGTian	0.630 (0.55, 0.92)	0.908 (0.65, 0.98)	0.098 (0.06, 0.11)	0.126 (0.12, 0.13)	0.611 (0.60, 0.62)
ARF	0.688 (0.68, 0.69)	0.915 (0.89, 0.93)	0.128 (0.12, 0.14)	0.182 (0.17, 0.19)	1.000 (1.00, 1.00)
CTGAN	0.736 (0.72, 0.76)	0.874 (0.81, 0.92)	0.153 (0.13, 0.17)	0.161 (0.15, 0.18)	1.000 (1.00, 1.00)
CRB	0.557 (0.55, 0.56)	0.877 (0.85, 0.90)	0.214 (0.21, 0.22)	0.151 (0.15, 0.16)	1.000 (1.00, 1.00)
DDPM	0.552 (0.55, 0.56)	0.869 (0.83, 0.90)	0.238 (0.22, 0.25)	0.128 (0.12, 0.13)	0.986 (0.97, 0.99)
NFLOW	0.652 (0.64, 0.68)	0.865 (0.77, 0.91)	0.159 (0.13, 0.18)	0.167 (0.16, 0.18)	1.000 (1.00, 1.00)

Appendix

Table 11: Performance metrics for Wine (white) dataset (500 samples) - Bold indicates best or statistically tied

Augmenter	Mean MSE $\downarrow$	$\alpha$ -Precision $\uparrow$	$\beta$ -Recall $\uparrow$	DCR $\uparrow$	$\delta$ -Presence $\uparrow$
ADMGTian	0.412 (0.41, 0.41)	0.967 (0.96, 0.97)	0.042 (0.04, 0.05)	0.584 (0.58, 0.59)	1.000
ARF	0.556 (0.55, 0.59)	0.961 (0.95, 0.97)	0.031 (0.03, 0.03)	0.727 (0.72, 0.73)	1.000
CTGAN	0.468 (0.46, 0.49)	0.897 (0.83, 0.94)	0.124 (0.10, 0.14)	0.647 (0.63, 0.67)	1.000
CRB	0.362 (0.36, 0.36)	0.976 (0.97, 0.98)	0.331 (0.33, 0.33)	0.583 (0.58, 0.59)	1.000
DDPM	0.443 (0.44, 0.45)	0.781 (0.74, 0.82)	0.249 (0.24, 0.26)	0.561 (0.55, 0.57)	1.000
NFLOW	0.421 (0.41, 0.43)	0.879 (0.80, 0.92)	0.137 (0.11, 0.17)	0.647 (0.63, 0.68)	1.000

Appendix

Table 12: Performance metrics for Causal chambers dataset (500 samples) - Bold indicates best or statistically tied

Augmenter	Mean MSE $\downarrow$	$\alpha$ -Precision $\uparrow$	$\beta$ -Recall $\uparrow$	DCR $\uparrow$	$\delta$ -Presence $\uparrow$
ADMGTian	0.416 (0.38, 0.47)	0.753 (0.73, 0.77)	0.034 (0.03, 0.05)	0.018 (0.02, 0.02)	0.202 (0.19, 0.22)
ARF	0.570 (0.51, 0.70)	0.691 (0.56, 0.76)	0.180 (0.11, 0.22)	0.036 (0.03, 0.06)	0.392 (0.30, 0.60)
CTGAN	0.659 (0.62, 0.69)	0.941 (0.91, 0.96)	0.351 (0.32, 0.38)	0.013 (0.01, 0.02)	0.147 (0.14, 0.16)
CRB	0.349 (0.33, 0.41)	0.857 (0.84, 0.88)	0.238 (0.22, 0.25)	0.022 (0.02, 0.02)	0.176 (0.16, 0.19)
DDPM	0.341 (0.33, 0.36)	0.954 (0.94, 0.96)	0.564 (0.56, 0.57)	0.011 (0.01, 0.01)	0.081 (0.07, 0.09)
NFLOW	0.630 (0.59, 0.68)	0.893 (0.85, 0.92)	0.345 (0.32, 0.37)	0.016 (0.01, 0.02)	0.109 (0.08, 0.17)

Appendix

Table 13: Performance metrics for Sachs dataset (1000 samples) - Bold indicates best or statistically tied

Augmenter	Mean MSE $\downarrow$	$\alpha$ -Precision $\uparrow$	$\beta$ -Recall $\uparrow$	DCR $\uparrow$	$\delta$ -Presence $\uparrow$
ADMGTian	1.639 (1.52, 1.90)	0.164 (0.14, 0.19)	0.016 (0.01, 0.02)	0.272 (0.25, 0.29)	0.815 (0.74, 0.88)
ARF	0.516 (0.50, 0.52)	0.898 (0.88, 0.91)	0.587 (0.57, 0.60)	0.266 (0.26, 0.27)	1.000
CTGAN	0.587 (0.55, 0.61)	0.924 (0.91, 0.93)	0.637 (0.62, 0.65)	0.213 (0.21, 0.22)	1.000 (1.00, 1.00)
CRB	0.452 (0.44, 0.46)	0.807 (0.78, 0.83)	0.669 (0.66, 0.68)	0.216 (0.21, 0.22)	1.000
DDPM	0.344 (0.33, 0.36)	0.920 (0.88, 0.94)	0.736 (0.72, 0.75)	0.185 (0.18, 0.20)	0.999 (1.00, 1.00)
NFLOW	0.522 (0.49, 0.55)	0.923 (0.89, 0.94)	0.640 (0.62, 0.66)	0.232 (0.22, 0.24)	1.000

Appendix

Table 14: Performance metrics for Wine (red) dataset (1000 samples) - Bold indicates best or statistically tied

Augmenter	Mean MSE $\downarrow$	$\alpha$ -Precision $\uparrow$	$\beta$ -Recall $\uparrow$	DCR $\uparrow$	$\delta$ -Presence $\uparrow$
ADMGTian	0.816 (0.48, 1.41)	0.816 (0.52, 0.97)	0.173 (0.10, 0.21)	0.125 (0.11, 0.14)	0.666 (0.61, 0.77)
ARF	0.641 (0.63, 0.65)	0.934 (0.92, 0.94)	0.229 (0.21, 0.24)	0.182 (0.17, 0.19)	1.000 (1.00, 1.00)
CTGAN	0.683 (0.67, 0.70)	0.913 (0.87, 0.94)	0.257 (0.23, 0.27)	0.155 (0.15, 0.16)	1.000 (1.00, 1.00)
CRB	0.534 (0.52, 0.55)	0.841 (0.82, 0.88)	0.321 (0.31, 0.33)	0.148 (0.14, 0.16)	1.000 (1.00, 1.00)
DDPM	0.494 (0.49, 0.50)	0.923 (0.89, 0.94)	0.352 (0.34, 0.36)	0.139 (0.13, 0.15)	0.998 (1.00, 1.00)
NFLOW	0.636 (0.61, 0.66)	0.919 (0.88, 0.95)	0.264 (0.25, 0.28)	0.166 (0.16, 0.18)	1.000 (1.00, 1.00)

Appendix

Table 15: Performance metrics for Wine (white) dataset (1000 samples) - Bold indicates best or statistically tied

Augmenter	Mean MSE $\downarrow$	$\alpha$ -Precision $\uparrow$	$\beta$ -Recall $\uparrow$	DCR $\uparrow$	$\delta$ -Presence $\uparrow$
ADMGTian	0.401 (0.40, 0.41)	0.980 (0.97, 0.98)	0.081 (0.07, 0.09)	0.580 (0.58, 0.58)	1.000
ARF	0.511 (0.51, 0.52)	0.970 (0.97, 0.97)	0.084 (0.08, 0.09)	0.712 (0.71, 0.72)	1.000
CTGAN	0.449 (0.43, 0.49)	0.917 (0.89, 0.94)	0.263 (0.23, 0.29)	0.633 (0.62, 0.64)	1.000
CRB	0.358 (0.36, 0.36)	0.989 (0.98, 0.99)	0.505 (0.50, 0.51)	0.579 (0.58, 0.58)	1.000
DDPM	0.418 (0.40, 0.43)	0.859 (0.82, 0.90)	0.472 (0.46, 0.48)	0.568 (0.56, 0.58)	1.000
NFLOW	0.421 (0.41, 0.43)	0.844 (0.78, 0.90)	0.202 (0.17, 0.23)	0.666 (0.65, 0.68)	1.000

Appendix

Table 16: Performance metrics for Causal chambers dataset (1000 samples) - Bold indicates best or statistically tied

Augmenter	Mean MSE $\downarrow$	$\alpha$ -Precision $\uparrow$	$\beta$ -Recall $\uparrow$	DCR $\uparrow$	$\delta$ -Presence $\uparrow$
ADMGTian	0.429 (0.38, 0.48)	0.533 (0.50, 0.57)	0.049 (0.03, 0.07)	0.039 (0.03, 0.05)	0.398 (0.37, 0.43)
ARF	0.490 (0.43, 0.62)	0.751 (0.64, 0.80)	0.363 (0.24, 0.42)	0.029 (0.02, 0.04)	0.342 (0.28, 0.49)
CTGAN	0.680 (0.63, 0.75)	0.884 (0.68, 0.94)	0.450 (0.31, 0.53)	0.019 (0.01, 0.03)	0.194 (0.15, 0.33)
CRB	0.317 (0.30, 0.35)	0.849 (0.83, 0.86)	0.367 (0.36, 0.38)	0.024 (0.02, 0.03)	0.214 (0.19, 0.24)
DDPM	0.316 (0.30, 0.35)	0.965 (0.96, 0.97)	0.757 (0.75, 0.76)	0.012 (0.01, 0.01)	0.071 (0.06, 0.08)
NFLOW	0.628 (0.55, 0.72)	0.901 (0.84, 0.94)	0.555 (0.53, 0.57)	0.015 (0.01, 0.02)	0.099 (0.07, 0.14)

Appendix

Table 17: Performance metrics for Sachs dataset (2000 samples) - Bold indicates best or statistically tied

Augmenter	Mean MSE $\downarrow$	$\alpha$ -Precision $\uparrow$	$\beta$ -Recall $\uparrow$	DCR $\uparrow$	$\delta$ -Presence $\uparrow$
ADMGTian	0.952 (0.52, 1.70)	0.734 (0.47, 0.91)	0.311 (0.17, 0.40)	0.178 (0.15, 0.22)	0.787 (0.73, 0.88)
ARF	0.606 (0.60, 0.63)	0.942 (0.94, 0.95)	0.396 (0.38, 0.41)	0.198 (0.18, 0.22)	1.000 (1.00, 1.00)
CTGAN	0.734 (0.66, 0.91)	0.847 (0.58, 0.93)	0.367 (0.24, 0.41)	0.202 (0.17, 0.30)	1.000 (1.00, 1.00)
CRB	0.504 (0.50, 0.51)	0.814 (0.80, 0.83)	0.464 (0.46, 0.47)	0.166 (0.15, 0.18)	1.000 (1.00, 1.00)
DDPM	0.449 (0.44, 0.46)	0.924 (0.91, 0.94)	0.519 (0.50, 0.53)	0.159 (0.15, 0.17)	0.999 (1.00, 1.00)
NFLOW	0.629 (0.60, 0.66)	0.925 (0.90, 0.94)	0.404 (0.38, 0.43)	0.186 (0.17, 0.21)	1.000 (1.00, 1.00)

Appendix

Table 18: Performance metrics for Wine (white) dataset (2000 samples) - Bold indicates best or statistically tied

Augmenter	Mean MSE $\downarrow$	$\alpha$ -Precision $\uparrow$	$\beta$ -Recall $\uparrow$	DCR $\uparrow$	$\delta$ -Presence $\uparrow$
ADMGTian	0.395 (0.39, 0.40)	0.983 (0.98, 0.99)	0.158 (0.14, 0.18)	0.580 (0.58, 0.58)	1.000
ARF	0.499 (0.48, 0.57)	0.970 (0.97, 0.97)	0.183 (0.13, 0.20)	0.704 (0.70, 0.73)	1.000
CTGAN	0.468 (0.43, 0.52)	0.917 (0.87, 0.95)	0.387 (0.32, 0.45)	0.630 (0.62, 0.64)	1.000
CRB	0.356 (0.36, 0.36)	0.988 (0.98, 0.99)	0.697 (0.69, 0.70)	0.582 (0.58, 0.59)	1.000
DDPM	0.379 (0.36, 0.39)	0.910 (0.87, 0.95)	0.696 (0.69, 0.70)	0.571 (0.56, 0.58)	1.000
NFLOW	0.415 (0.41, 0.42)	0.948 (0.94, 0.96)	0.376 (0.35, 0.40)	0.649 (0.64, 0.65)	1.000

Appendix

Table 19: Performance metrics for Causal chambers dataset (2000 samples) - Bold indicates best or statistically tied

Figure 7: Mean MSE across all variables as sample size increases (Known Graph). Lower is better. CRB maintains strong performance across sample sizes. Neural networks were used as predictors.

Panel of Figure 8 — Figure 8: Results for chain with linear relations

Panel of Figure 9 — Figure 9: Results for chain with non-linear relations

Data Augmentation

Data augmentation expands or transforms training examples to improve learning, data efficiency, and robustness to distribution shifts.

See all 250 papers

The geometry of invariant learning: an information-theoretic analysis of data augmentation and generalization
Feb 16, 2026Abdelali Bouyahia, Frédéric LeBlanc, Mario MarchandData Augmentation Generalization Bounds
Département d’informatique et génie logiciel Université Laval Québec, QC, Canada · Institut intelligence et données Université Laval Québec, G1V 0A6, Canada
Data Augmentation: A Fourier Analysis Perspective
Jun 23, 2026Behrooz Tahmasebi, Melanie Weber, Stefanie JegelkaData Augmentation Statistical Learning Theory
Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University, Cambridge, MA 02138, USA. · Technical University of Munich (CIT, MCML, MDSI) · MIT Computer Science and Artificial Intelligence Laboratory (CSAIL)
Counterfactual Residual Data Augmentation for Regression
Jun 26, 2026Hossein Mohebbi, Oliver Schulte, Ke Li +1Synthetic Tabular Data Generation Counterfactual Data Augmentation
University of Waterloo · Vector Institute · Simon Fraser University.

Feb 16, 2026J/K move · Enter open · S save

The geometry of invariant learning: an information-theoretic analysis of data augmentation and generalization

Abdelali Bouyahia, Frédéric LeBlanc, Mario Marchand

Département d’informatique et génie logiciel Université Laval Québec, QC, Canada · Institut intelligence et données Université Laval Québec, G1V 0A6, Canada

Data Augmentation Generalization Bounds Information-Theoretic Generalization Bounds Invariant Representation Learning

Open paper AskPDF arXiv

Data augmentation is one of the most widely used techniques to improve generalization in modern machine learning, often justified by its ability to promote invariance to label-irrelevant transformations. However, its theoretical role remains only partially understood. In this work, we propose an information-theoretic framework that systematically accounts for the effect of augmentation on generalization and invariance learning. Our approach builds upon mutual information-based bounds, which relate the generalization gap to the amount of information a learning algorithm retains about its training data. We extend this framework by modeling the augmented distribution as a composition of the original data distribution with a distribution over transformations, which naturally induces an orbit-averaged loss function. Under mild sub-Gaussian assumptions on the loss function and the augmentation process, we derive a new generalization bound that decompose the expected generalization gap into three interpretable terms: (1) a distributional divergence between the original and augmented data, (2) a stability term measuring the algorithm dependence on training data, and (3) a sensitivity term capturing the effect of augmentation variability. To connect our bounds to the geometry of the augmentation group, we introduce the notion of group diameter, defined as the maximal perturbation that augmentations can induce in the input space. The group diameter provides a unified control parameter that bounds all three terms and highlights an intrinsic trade-off: small diameters preserve data fidelity but offer limited regularization, while large diameters enhance stability at the cost of increased bias and sensitivity. We validate our theoretical bounds with numerical experiments, demonstrating that it reliably tracks and predicts the behavior of the true generalization gap.

Abstract

Data Augmentation

Explore similar work

The geometry of invariant learning: an information-theoretic analysis of data augmentation and generalization

Data Augmentation: A Fourier Analysis Perspective

Counterfactual Residual Data Augmentation for Regression