You May Be Running the Wrong Inception Crop

Jason Chuan-Chih Chou

cs.LGOct 2, 2026

You May Be Running the Wrong Inception Crop

Authors: Jason Chuan-Chih Chou

Organizations: Cohere Labs Community

Abstract

A decade after its inception, Inception crop has become the standard crop-based data augmentation method for training deep vision models. Not only is its practice of uniformly sampling crop scale and aspect ratio widely adopted, but also its lower and upper bounds, with the scale lower bound being the sole exception that is sometimes tuned. It is therefore surprising that the standard implementation in the TensorFlow / JAX ecosystem samples crop scale with probability density function $f(A) \propto \frac{1}{\sqrt{A}}$ unlike the PyTorch counterpart, which follows the original description. Motivated by this discovery, we train 522 ViT-S/16 models on the ImageNet-1k dataset with various training budgets and crop scale distributions. We reach $78.78\pm0.09$ top-1 val. accuracy with 90 epochs of training budget and find that 1. Higher training budget requires stronger augmentation; 2. Lower tail of the distribution of the crop scale determines the augmentation strength of Inception crop; 3. Models trained with higher training budget exhibit sparser saliency, regardless of the crop scale distribution or weight decay. Based on 2. we revisit the performance of Beta crop, whose softer cutoff allows it to optimize model performance across training budgets with less compromise. We replicate 1. and 3. with Scion optimizer in addition to AdamW, suggesting that the results may be general.

Figures & tables

Figure 1: Crop area distribution of the standard implementation of Inception crop in TensorFlow with default parameters for $256\times 512$ and $512\times 256$ images, $N=10^{7}$ . The probability density function of the distribution is approximately proportional to $\frac{1}{\sqrt{A}}$ but truncated at larger area when some aspect ratios become inadmissible. Note that landscape and portrait images have slightly different distributions.

Panel of Figure 2 — Figure 2: Probability heatmaps of the crop size $(h,w)$ given by the TensorFlow implementation for landscape and portrait images with default parameters $r_{\mathrm{min}}=\frac{3}{4},r_{\mathrm{max}}=\frac{4}{3},a_{\mathrm{min}}=0.05,a_{\mathrm{max}}=1,N=10^{7}$ .

Figure 3: Crop area distribution of Beta crop with selected values of $\alpha$ and $\beta$ for landscape image ( $h_{o}=256,w_{o}=512$ ), $r_{\min}=\frac{3}{4},r_{\max}=\frac{4}{3},a_{\min}=0.05,a_{\max}=1,N=10^{7}$ samples for each distribution. $\alpha=\beta=1$ results in uniform distribution as expected except truncation at higher crop scale due to inadmissible aspect ratios.

	TensorFlow Inception crop						PyTorch Inception crop
$a_{\mathrm{min}}$	0.025	0.05	0.1	0.15	0.2	0.25	0.025	0.05	0.1	0.15	0.2	0.25
30ep	64.59	65.55	67.03	68.06	68.78	69.15	67.08	67.35	68.26	68.74	69.26	69.67
60ep	72.77	73.43	74.29	74.77	74.99	75.25	74.35	74.77	74.86	75.08	75.20	75.55
90ep	75.76	76.25	76.77	77.03	77.21	77.16	76.81	76.92	77.21	77.21	77.35	77.16
150ep	77.90	78.29	78.60	78.59	78.52	78.39	78.58	78.64	78.75	78.70	78.51	78.43
300ep	79.43	79.75	79.64	79.49	79.20	79.01	79.80	79.73	79.62	79.18	79.13	78.86

Table 1: Mean top-1 val. accuracy (original label), TensorFlow vs. PyTorch Inception crop. As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound

a_{\mathrm{min}}

decreases (stronger augmentation).

Panel of Figure 4 — Figure 4: Top-1 val. accuracy (original label), TensorFlow vs. PyTorch Inception crop, with sample standard deviation $\sigma_{\mathrm{tf}},\sigma_{\mathrm{torch}},(\sigma^{2}_{\mathrm{tf}}+\sigma^{2}_{\mathrm{torch}})^{\frac{1}{2}}$ plotted as error bar. As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound $a_{\mathrm{min}}$ decreases (stronger augmentation).

	TensorFlow Inception crop						PyTorch Inception crop
$a_{\mathrm{min}}$	0.025	0.05	0.1	0.15	0.2	0.25	0.025	0.05	0.1	0.15	0.2	0.25
30ep	72.23	73.22	74.77	75.73	76.45	76.84	74.77	75.11	75.90	76.39	76.88	77.21
60ep	80.18	80.75	81.30	81.68	81.88	82.13	81.44	81.73	81.86	81.95	82.09	82.37
90ep	82.65	83.10	83.31	83.57	83.64	83.56	83.39	83.39	83.65	83.58	83.59	83.50
150ep	84.21	84.51	84.67	84.51	84.42	84.36	84.57	84.65	84.56	84.47	84.33	84.28
300ep	85.30	85.39	85.25	85.01	84.71	84.45	85.32	85.27	85.06	84.81	84.59	84.44

Table 2: Mean top-1 val. accuracy (ReaL), TensorFlow vs. PyTorch Inception crop. As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound

a_{\mathrm{min}}

decreases (stronger augmentation).

Panel of Figure 5 — Figure 5: Top-1 val. accuracy (ReaL), TensorFlow vs. PyTorch Inception crop, with sample standard deviation $\sigma_{\mathrm{tf}},\sigma_{\mathrm{torch}},(\sigma^{2}_{\mathrm{tf}}+\sigma^{2}_{\mathrm{torch}})^{\frac{1}{2}}$ plotted as error bar. As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound $a_{\mathrm{min}}$ decreases (stronger augmentation).

Figure 6: Top-1 val. accuracy (original label), Beta crop with $\alpha=1$ . Solid line: $\beta=3$ , Dashed line: $\beta=5$

Figure 7: Top-1 val. accuracy (original label), Beta crop with $\beta=1$ . Top-1 val. accuracy vs. $\alpha$ , $a_{\mathrm{min}}=0.05$ is plotted on the right. For $\alpha=\beta=1$ , we reuse the data points of the PyTorch Inception crop experiments.

	Original label						ReaL
$\alpha$	1.149	1.287	1.551	1.818	2.099	2.403	1.149	1.287	1.551	1.818	2.099	2.403
30ep	67.47	67.92	68.67	69.22	69.61	69.87	75.16	75.58	76.44	76.97	77.30	77.58
60ep	74.59	74.79	75.24	75.32	75.48	75.51	81.72	81.85	82.20	82.17	82.35	82.35
90ep	76.89	76.99	77.39	77.25	77.15	77.29	83.48	83.56	83.72	83.59	83.54	83.72
150ep	78.63	78.49	78.73	78.59	78.63	78.59	84.72	84.48	84.63	84.48	84.41	84.42
300ep	79.57	79.60	79.42	79.27	78.95	79.04	85.15	85.14	84.90	84.77	84.58	84.55

Table 3: Mean top-1 val. accuracy of Beta crop, matching strength, original label (left) and ReaL (right). As the training budget increases from 30 epochs to 300 epochs, the optimal

\alpha

decreases (stronger augmentation).

\alpha=1.149

may be an exception possibly due to too many crops near zero area. In future iterations we may want small but nonzero

a_{\min}

(say

0<a_{\min}\leq 0.025

) instead while keeping the same 5th percentile crop scale.

Panel of Figure 8 — Figure 8: Diff. in top-1 val. accuracy (Beta crop, matching strength - PyTorch), original label (left) and ReaL (right), with $(\sigma^{2}_{\mathrm{beta}}+\sigma^{2}_{\mathrm{torch}})^{\frac{1}{2}}$ plotted as error bar.

Panel of Figure 9 — Figure 9: Mean gradient entropy $\overline{S_{\mathrm{grad}}}$ defined in eq. 2 vs. top-1 val. accuracy for all 450 Inception crop and Beta crop experiments with AdamW optimizer, color-coded by the 5th percentile crop scale (left) and training budget (right). For reference and sanity check, $S_{\mathrm{grad}}\leq\log(224^{2})=10.82$ given our input resolution. Dashed lines mark the average of the experiments of the given training budget.

Panel of Figure 10 — Figure 10: Mean Grad-CAM entropy $\overline{S_{\mathrm{grad\text{-}cam}}}$ (left) and mean relevance entropy $\overline{S_{\mathrm{rel}}}$ (right) vs. top-1 val. accuracy for all 450 Inception crop and Beta crop experiments with AdamW optimizer, color-coded by the training budget. For reference and sanity check, $S_{\mathrm{grad\text{-}cam}},S_{\mathrm{rel}}\leq\log(14^{2})=5.28$ given our input resolution and patch size. Dashed lines mark the average of the experiments of the given training budget.

	Original label		ReaL
	baseline	optimized	baseline	optimized
30ep	73.31 $\pm 0.09$	74.46 $\pm 0.19$	80.83 $\pm 0.07$	81.70 $\pm 0.08$
60ep	77.44 $\pm 0.09$	77.76 $\pm 0.15$	84.09 $\pm 0.10$	84.16 $\pm 0.16$
90ep	78.68 $\pm 0.09$	78.78 $\pm 0.09$	84.94 $\pm 0.05$	84.96 $\pm 0.07$
150ep	79.65 $\pm 0.07$	—	85.41 $\pm 0.05$	—
300ep	80.10 $\pm 0.14$	80.23 $\pm 0.17$	85.66 $\pm 0.21$	85.71 $\pm 0.15$

Table 4: Top-1 val. accuracy of Scion experiments,

a_{\mathrm{min}}=0.05

baseline vs. optimized

a_{\mathrm{min}}

for the training budget, original label (left) and ReaL (right). As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound

a_{\mathrm{min}}

decreases (stronger augmentation) while

a_{\mathrm{min}}=0.05

is already expected to be optimal for 150ep.

Panel of Figure 11 — Figure 11: Mean gradient entropy $\overline{S_{\mathrm{grad}}}$ (left), mean Grad-CAM entropy $\overline{S_{\mathrm{grad\text{-}cam}}}$ (middle), and mean relevance entropy $\overline{S_{\mathrm{rel}}}$ (right) vs. top-1 val. accuracy for all 72 Scion experiments including that of table 4 and WD sweep ( table 18 ), color-coded by training budget.

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 12: Training loss of the linear head variant of ViT-S/16 from Beyer et al. (2022a) for 90 epochs using the Big Vision ( Beyer et al., 2022b ) first-party code (cyan) vs. our reimplementation in TorchVision ( maintainers and contributors, 2016 ) (red).

Figure 13: Probability heatmap of the crop size $(h,w)$ given by the PyTorch Inception crop ( RandomResizedCrop , RRC) for landscape image ( $h_{o}=256,w_{o}=512$ ) with the same parameters as in fig. 2 : $r_{\mathrm{min}}=\frac{3}{4},r_{\mathrm{max}}=\frac{4}{3},a_{\mathrm{min}}=0.05,a_{\mathrm{max}}=1,N=10^{7}$ . The counterpart for portrait image ( $h_{o}=512,w_{o}=256$ is indistinguishable after image transposition.

Panel of Figure 14 — Figure 14: Color grids, before and after calling Contrast( $\cdot$ , 1.9) and their RGB values.

Figure 15: Crop area distribution of the reimplementation of TensorFlow Inception crop in PyTorch with default parameters for $256\times 512$ and $512\times 256$ images, $N=10^{7}$ . The results are indistinguishable from fig. 1 .

$a_{\mathrm{min}}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	64.59 $\pm 0.11$	65.55 $\pm 0.07$	67.03 $\pm 0.36$	68.06 $\pm 0.19$	68.78 $\pm 0.05$	69.15 $\pm 0.06$
60ep	72.77 $\pm 0.21$	73.43 $\pm 0.13$	74.29 $\pm 0.15$	74.77 $\pm 0.05$	74.99 $\pm 0.08$	75.25 $\pm 0.10$
90ep	75.76 $\pm 0.04$	76.25 $\pm 0.05$	76.77 $\pm 0.09$	77.03 $\pm 0.08$	77.21 $\pm 0.13$	77.16 $\pm 0.16$
150ep	77.90 $\pm 0.07$	78.29 $\pm 0.06$	78.60 $\pm 0.06$	78.59 $\pm 0.10$	78.52 $\pm 0.06$	78.39 $\pm 0.13$
300ep	79.43 $\pm 0.07$	79.75 $\pm 0.07$	79.64 $\pm 0.14$	79.49 $\pm 0.12$	79.20 $\pm 0.10$	79.01 $\pm 0.14$

Appendix

Table 5: Top-1 val. accuracy (original label), TensorFlow Inception crop. As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound

a_{\mathrm{min}}

decreases (stronger augmentation).

$a_{\mathrm{min}}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	67.08 $\pm 0.41$	67.35 $\pm 0.33$	68.26 $\pm 0.20$	68.74 $\pm 0.20$	69.26 $\pm 0.13$	69.67 $\pm 0.17$
60ep	74.35 $\pm 0.08$	74.77 $\pm 0.08$	74.86 $\pm 0.13$	75.08 $\pm 0.22$	75.20 $\pm 0.07$	75.55 $\pm 0.11$
90ep	76.81 $\pm 0.16$	76.92 $\pm 0.13$	77.21 $\pm 0.19$	77.21 $\pm 0.17$	77.35 $\pm 0.14$	77.16 $\pm 0.09$
150ep	78.58 $\pm 0.02$	78.64 $\pm 0.18$	78.75 $\pm 0.04$	78.70 $\pm 0.10$	78.51 $\pm 0.17$	78.43 $\pm 0.11$
300ep	79.80 $\pm 0.06$	79.73 $\pm 0.12$	79.62 $\pm 0.05$	79.18 $\pm 0.03$	79.13 $\pm 0.04$	78.86 $\pm 0.01$

Appendix

Table 6: Top-1 val. accuracy (original label), PyTorch Inception crop. As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound

a_{\mathrm{min}}

decreases (stronger augmentation).

$a_{\mathrm{min}}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	72.23 $\pm 0.13$	73.22 $\pm 0.18$	74.77 $\pm 0.39$	75.73 $\pm 0.28$	76.45 $\pm 0.02$	76.84 $\pm 0.11$
60ep	80.18 $\pm 0.23$	80.75 $\pm 0.02$	81.30 $\pm 0.15$	81.68 $\pm 0.08$	81.88 $\pm 0.08$	82.13 $\pm 0.03$
90ep	82.65 $\pm 0.05$	83.10 $\pm 0.03$	83.31 $\pm 0.08$	83.57 $\pm 0.06$	83.64 $\pm 0.07$	83.56 $\pm 0.16$
150ep	84.21 $\pm 0.06$	84.51 $\pm 0.10$	84.67 $\pm 0.03$	84.51 $\pm 0.08$	84.42 $\pm 0.07$	84.36 $\pm 0.10$
300ep	85.30 $\pm 0.09$	85.39 $\pm 0.09$	85.25 $\pm 0.11$	85.01 $\pm 0.07$	84.71 $\pm 0.09$	84.45 $\pm 0.19$

Appendix

Table 7: Top-1 val. accuracy (ReaL), TensorFlow Inception crop. As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound

a_{\mathrm{min}}

decreases (stronger augmentation).

$a_{\mathrm{min}}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	74.77 $\pm 0.33$	75.11 $\pm 0.32$	75.90 $\pm 0.18$	76.39 $\pm 0.14$	76.88 $\pm 0.10$	77.21 $\pm 0.09$
60ep	81.44 $\pm 0.11$	81.73 $\pm 0.09$	81.86 $\pm 0.12$	81.95 $\pm 0.11$	82.09 $\pm 0.09$	82.37 $\pm 0.05$
90ep	83.39 $\pm 0.15$	83.39 $\pm 0.09$	83.65 $\pm 0.14$	83.58 $\pm 0.17$	83.59 $\pm 0.07$	83.50 $\pm 0.08$
150ep	84.57 $\pm 0.14$	84.65 $\pm 0.17$	84.56 $\pm 0.05$	84.47 $\pm 0.08$	84.33 $\pm 0.06$	84.28 $\pm 0.07$
300ep	85.32 $\pm 0.15$	85.27 $\pm 0.03$	85.06 $\pm 0.01$	84.81 $\pm 0.09$	84.59 $\pm 0.06$	84.44 $\pm 0.08$

Appendix

Table 8: Top-1 val. accuracy (ReaL), PyTorch Inception crop. As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound

a_{\mathrm{min}}

decreases (stronger augmentation).

$a_{min}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	5.1e-4	8.0e-4	6.6e-3	0.013	3.3e-3	7.5e-3
60ep	2.6e-4	1.e-4	7.3e-3	0.074	0.029	0.024
90ep	4.0e-4	1.2e-3	0.022	0.16	0.29	0.93
150ep	6.6e-5	0.034	0.021	0.25	0.91	0.68
300ep	2.5e-3	0.86	0.84	0.013	0.36	0.14

Appendix

Table 9: T-test p-values of the differences in top-1 val. accuracies (original) between the TensorFlow and PyTorch Inception crop, two-sided, equal variance, color-coded ( p <0.05 , p <0.01 , p <0.001 ).

$a_{min}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	2.3e-4	8.8e-4	0.01	0.024	1.6e-3	0.011
60ep	1.0e-3	5.7e-5	7.6e-3	0.027	0.034	2.4e-3
90ep	1.1e-3	7.2e-3	0.022	0.88	0.47	0.63
150ep	0.015	0.31	0.04	0.59	0.18	0.33
300ep	0.79	0.091	0.036	0.035	0.1	0.95

Appendix

Table 10: T-test p-values of the differences in top-1 val. accuracies (ReaL) between the TensorFlow and PyTorch Inception crop, two-sided, equal variance, color-coded ( p <0.05 , p <0.01 , p <0.001 ).

$a_{min}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	1.8e-4	1	2.1e-3	2.6e-5	3.e-7	2.5e-7
60ep	9.8e-3	1	1.7e-3	7.1e-5	6.2e-5	4.4e-5
90ep	1.9e-4	1	9.4e-4	1.5e-4	2.7e-4	7.0e-4
150ep	1.8e-3	1	2.9e-3	9.9e-3	9.8e-3	0.31
300ep	5.e-3	1	0.3	0.037	1.3e-3	1.2e-3

Appendix

Table 11: T-test p-values of the top-1 val. accuracies (original) between the default

a_{min}=0.05

and other values with various training budgets, TensorFlow Inception crop, two-sided, equal variance, color-coded ( p <0.05 , p <0.01 , p <0.001 ).

$a_{min}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	1.5e-3	1	3.4e-3	2.1e-4	7.4e-6	8.5e-6
60ep	0.013	1	3.7e-3	3.4e-5	2.3e-5	5.2e-7
90ep	1.8e-4	1	0.016	2.8e-4	3.1e-4	8.5e-3
150ep	0.013	1	0.071	0.96	0.26	0.13
300ep	0.27	1	0.16	4.6e-3	7.3e-4	1.4e-3

Appendix

Table 12: T-test p-values of the top-1 val. accuracies (ReaL) between the default

a_{min}=0.05

and other values with various training budget, TensorFlow Inception crop, two-sided, equal variance, color-coded ( p <0.05 , p <0.01 , p <0.001 ).

$a_{min}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	0.43	1	0.016	3.4e-3	7.3e-4	4.3e-4
60ep	2.7e-3	1	0.38	0.081	2.1e-3	5.3e-4
90ep	0.39	1	0.1	0.083	0.019	0.066
150ep	0.62	1	0.34	0.6	0.42	0.17
300ep	0.45	1	0.21	1.5e-3	1.2e-3	2.2e-4

Appendix

Table 13: T-test p-values of the top-1 val. accuracies (original) between

a_{min}=0.05

and other values with various training budgets, PyTorch Inception crop, two-sided, equal variance, color-coded ( p <0.05 , p <0.01 , p <0.001 ).

$a_{min}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	0.27	1	0.019	3.1e-3	7.4e-4	3.8e-4
60ep	0.023	1	0.18	0.055	6.9e-3	4.3e-4
90ep	0.98	1	0.057	0.17	0.042	0.19
150ep	0.6	1	0.47	0.18	0.039	0.027
300ep	0.56	1	2.6e-4	1.1e-3	4.9e-5	8.6e-5

Appendix

Table 14: T-test p-values of the top-1 val. accuracies (ReaL) between

a_{min}=0.05

and other values with various training budget, PyTorch Inception crop, two-sided, equal variance, color-coded ( p <0.05 , p <0.01 , p <0.001 ).

Panel of Figure 16 — Figure 16: Top-1 val. accuracy of PyTorch Inception crop (solid line) vs. TensorFlow Inception crop (dashed line), original label (left) and ReaL (right), plotted with the 5th percentile crop scale as the x-axis. TensorFlow Inception crop generally underperforms except 300ep where it matches PyTorch Inception crop even with such unified augmentation strength measure.

Figure 17: Top-1 val. accuracy (ReaL), Beta crop with $\alpha=1$ . Solid line: $\beta=3$ , Dashed line: $\beta=5$

Figure 18: Top-1 val. accuracy (ReaL), Beta crop with $\beta=1$ . Top-1 val. accuracy vs. $\alpha$ , $a_{min}=0.05$ is plotted on the right. For $\alpha=\beta=1$ , we reuse the data points of the PyTorch Inception crop experiments.

$\alpha$	1.149	1.287	1.551	1.818	2.099	2.403
30ep	67.47 $\pm 0.18$	67.92 $\pm 0.20$	68.67 $\pm 0.15$	69.22 $\pm 0.25$	69.61 $\pm 0.19$	69.87 $\pm 0.04$
60ep	74.59 $\pm 0.13$	74.79 $\pm 0.14$	75.24 $\pm 0.17$	75.32 $\pm 0.12$	75.48 $\pm 0.15$	75.51 $\pm 0.04$
90ep	76.89 $\pm 0.04$	76.99 $\pm 0.23$	77.39 $\pm 0.11$	77.25 $\pm 0.15$	77.15 $\pm 0.14$	77.29 $\pm 0.21$
150ep	78.63 $\pm 0.21$	78.49 $\pm 0.07$	78.73 $\pm 0.06$	78.59 $\pm 0.24$	78.63 $\pm 0.09$	78.59 $\pm 0.12$
300ep	79.57 $\pm 0.10$	79.60 $\pm 0.11$	79.42 $\pm 0.10$	79.27 $\pm 0.15$	78.95 $\pm 0.23$	79.04 $\pm 0.12$

Appendix

Table 15: Top-1 val. accuracy of Beta crop, matching strength (original label). As the training budget increases from 30 epochs to 300 epochs, the optimal

\alpha

decreases (stronger augmentation).

\alpha=1.149

may be an exception possibly due to too many crops near zero area. In future iterations we may want small but nonzero

a_{\min}

(say

0<a_{\min}\leq 0.025

) instead while keeping the same 5th percentile crop scale.

$\alpha$	1.149	1.287	1.551	1.818	2.099	2.403
30ep	75.16 $\pm 0.15$	75.58 $\pm 0.17$	76.44 $\pm 0.12$	76.97 $\pm 0.23$	77.30 $\pm 0.08$	77.58 $\pm 0.10$
60ep	81.72 $\pm 0.05$	81.85 $\pm 0.06$	82.20 $\pm 0.10$	82.17 $\pm 0.16$	82.35 $\pm 0.15$	82.35 $\pm 0.05$
90ep	83.48 $\pm 0.04$	83.56 $\pm 0.11$	83.72 $\pm 0.09$	83.59 $\pm 0.06$	83.54 $\pm 0.17$	83.72 $\pm 0.15$
150ep	84.72 $\pm 0.16$	84.48 $\pm 0.08$	84.63 $\pm 0.03$	84.48 $\pm 0.10$	84.41 $\pm 0.08$	84.42 $\pm 0.06$
300ep	85.15 $\pm 0.16$	85.14 $\pm 0.03$	84.90 $\pm 0.10$	84.77 $\pm 0.16$	84.58 $\pm 0.11$	84.55 $\pm 0.14$

Appendix

Table 16: Rop-1 val. accuracy of Beta crop, matching strength (ReaL). As the training budget increases from 30 epochs to 300 epochs, the optimal

\alpha

decreases (stronger augmentation).

Figure 19: Sample saliency maps given by the rectified gradient for the 1st images of the first 25 classes $c\in[0\ldotp\ldotp 24]$ in torchvision.datasets.ImageNet validation split. We use a model trained with 300ep training budget and the TensorFlow Inception crop with $a_{\mathrm{min}}=0.05$ . Though conceptually the simplest, the saliency maps given by the rectified gradient are the least interpretable of the 3.

Figure 20: Mean gradient entropy $\overline{S_{\mathrm{grad}}}$ defined in eq. 2 vs. top-1 val. accuracy (ReaL) for all 450 Inception crop and Beta crop experiments with AdamW optimizer, color-coded by $a_{\mathrm{min}}$ .

Panel of Figure 21 — Figure 21: Distribution of the gradient entropy $S_{\mathrm{grad}}$ defined in eq. 2 for the 50000 validation set images given by models trained with the TensorFlow Inception crop (left) and PyTorch Inception crop (right) with $a_{\mathrm{min}}=0.05$ , color-coded by the training budget.

Figure 22: Sample saliency maps given by Grad-CAM at the last layer for the 1st images of the first 25 classes $c\in[0\ldotp\ldotp 24]$ in torchvision.datasets.ImageNet validation split. We use a model trained with 300ep training budget and the TensorFlow Inception crop with $a_{\mathrm{min}}=0.05$ .

Panel of Figure 23 — Figure 23: Distribution of the Grad-CAM entropy $S_{\mathrm{grad-cam}}$ for the 50000 validation set images given by models trained with the TensorFlow Inception crop (left) and PyTorch Inception crop (right) with $a_{\mathrm{min}}=0.05$ , color-coded by the training budget.

Figure 24: Sample saliency maps given by LRP at the last layer for the 1st images of the first 25 classes $c\in[0\ldotp\ldotp 24]$ in torchvision.datasets.ImageNet validation split. We use a model trained with 300ep training budget and the TensorFlow Inception crop with $a_{\mathrm{min}}=0.05$ .

Panel of Figure 25 — Figure 25: Distribution of the relevance entropy $S_{\mathrm{rel}}$ for the 50000 validation set images given by models trained with the TensorFlow Inception crop (left) and PyTorch Inception crop (right) with $a_{\mathrm{min}}=0.05$ , color-coded by the training budget.

	Sign	Spectral	Bias
Norm	$d_{\mathrm{in}}\max_{i,j}\|A_{i,j}\|$	$\sqrt{\frac{d_{\mathrm{in}}}{d_{\mathrm{out}}}}\\|{\bm{A}}\\|_{\mathcal{S}_{\infty}}$	$\mathrm{RMS}$
LMO	${\bm{A}}\mapsto-\frac{\sign({\bm{A}})}{d_{\mathrm{in}}}$	${\bm{A}}\mapsto-\sqrt{\frac{d_{\mathrm{out}}}{d_{\mathrm{in}}}}{\bm{U}}{\bm{V}}^{\top}$	${\bm{b}}_{\ell}\mapsto-\tfrac{{\bm{b}}_{\ell}}{\\|{\bm{b}}_{\ell}\\|_{\mathrm{RMS}}}$

Appendix

Table 17: Norms and the associated

\operatorname{lmo}

s as normalized in our experiments. Sign and Spectral assume matrix weight

\bm{\theta}_{l}={\bm{A}}\in\mathbb{R}^{d_{\mathrm{out}}\times d_{\mathrm{in}}}

while Bias assumes vector weight

\bm{\theta}_{l}={\bm{b}}_{\ell}\in\mathbb{R}^{d_{\mathrm{out}}}

{\bm{U}}{\bm{V}}^{\top}

refers to the reduced SVD of the input matrix with unitary matrices

{\bm{U}}

and

{\bm{V}}^{\top}

from the full SVD

{\bm{A}}={\bm{U}}\text{diag}(\bm{\sigma}){\bm{V}}^{\top}

while

\|{\bm{A}}\|_{\mathcal{S}_{\infty}}=\max(\bm{\sigma})

is the spectral norm of the matrix.

	Original label				ReaL
$\lambda_{l}$	0.04	0.08	0.12	0.16	0.04	0.08	0.12	0.16
30ep	72.08 $\pm 0.19$	73.31 $\pm 0.09$	73.24 $\pm 0.21$	72.50 $\pm 0.22$	79.46 $\pm 0.18$	80.83 $\pm 0.07$	80.81 $\pm 0.13$	80.18 $\pm 0.20$
60ep	76.98 $\pm 0.09$	77.44 $\pm 0.09$	76.88 $\pm 0.14$	76.07 $\pm 0.16$	83.58 $\pm 0.08$	84.09 $\pm 0.10$	83.75 $\pm 0.15$	83.26 $\pm 0.18$
90ep	78.42 $\pm 0.19$	78.68 $\pm 0.09$	78.32 $\pm 0.04$	77.31 $\pm 0.12$	84.51 $\pm 0.08$	84.94 $\pm 0.05$	84.88 $\pm 0.05$	84.11 $\pm 0.10$
150ep	79.27 $\pm 0.08$	79.65 $\pm 0.07$	79.05 $\pm 0.15$	78.31 $\pm 0.07$	84.94 $\pm 0.09$	85.41 $\pm 0.05$	85.20 $\pm 0.14$	84.69 $\pm 0.08$
300ep	79.67 $\pm 0.06$	80.10 $\pm 0.14$	79.98 $\pm 0.10$	78.64 $\pm 0.32$	85.04 $\pm 0.10$	85.66 $\pm 0.21$	85.66 $\pm 0.09$	84.54 $\pm 0.35$

Appendix

Table 18: Top-1 val. accuracy of the Scion optimizer WD sweep, original label (left) and ReaL (right).

Panel of Figure 26 — Figure 26: Mean gradient entropy $\overline{S_{\mathrm{grad}}}$ vs. top-1 val. accuracy, original label (left) and ReaL (right), for all 72 experiments with Scion optimizer, color-coded by $\lambda_{l}$ .

Data Augmentation

Data augmentation expands or transforms training examples to improve learning, data efficiency, and robustness to distribution shifts.

See all 249 papers

Jul 21, 2026

InstructMixup: Instruction-Guided Salient Patch Editing for Robust Data Augmentation

In image and video technologies, data augmentation is widely used to improve the generalization of deep visual models, and mixup-based strategies that interpolate between samples have become the dominant approach. However, computing informative mixing regions adds substantial overhead, and blending content across different images frequently disrupts the semantic integrity of the resulting sample. We propose \our{}, a data augmentation method that constructs challenging yet label-consistent training samples entirely within a single visual sample. \our{} first extracts multi-scale salient patches from the sample using a lightweight saliency detector, refines each patch with an instruction-guided generative model, and blends the edited patch back into the non-salient regions of the same sample; because the generative edits are computed once and cached offline, this step adds negligible training cost. To further diversify the learned representation, \our{} injects self-similar fractal structure into the same salient regions at an adaptive ratio, so each training sample carries both fractal and non-fractal structure. We derive a second-order approximation of the resulting vicinal risk, showing that the method simultaneously enforces invariance to the generative edit and suppresses curvature along the perturbed salient directions, and we verify both predictions empirically. We evaluate on small to large backbones for instance Convolutional Neural Networks (CNNs), Vision Transformers (ViTs) and Vision-Language Foundational Models (VLMs) across seven benchmarks covering coarse- and fine-grained classification, robustness to corruption and occlusion, calibration, and transfer and self-supervised learning, InstructMixup outperforms nine competing augmentation methods, surpassing the strongest baseline across all benchmarks.

Khawar Islam, Arif Mahmood, Xin Jin +1

The University of Melbourne, Melbourne, Australia · The Information Technology University, Lahore, Pakistan · Westlake University, China

Synthetic Data Augmentation Data Augmentation

Ask PDF

Mar 25, 2026

Amplified Patch-Level Differential Privacy for Free via Random Cropping

Random cropping is one of the most common data augmentation techniques in computer vision, yet the role of its inherent randomness in training differentially private machine learning models has thus far gone unexplored. We observe that when sensitive content in an image is spatially localized, such as a face or license plate, random cropping can probabilistically exclude that content from the model's input. This introduces a third source of stochasticity in differentially private training with stochastic gradient descent, in addition to gradient noise and minibatch sampling. This additional randomness amplifies differential privacy without requiring changes to model architecture or training procedure. We formalize this effect by introducing a patch-level neighboring relation for vision data and deriving tight privacy bounds for differentially private stochastic gradient descent (DP-SGD) when combined with random cropping. Our analysis quantifies the patch inclusion probability and shows how it composes with minibatch sampling to yield a lower effective sampling rate. Empirically, we validate that patch-level amplification improves the privacy-utility trade-off across multiple segmentation architectures and datasets. Our results demonstrate that aligning privacy accounting with domain structure and additional existing sources of randomness can yield stronger guarantees at no additional cost.

Kaan Durmaz, Jan Schuchardt, Sebastian Schmidt +1

Technical University of Munich · Machine Learning Research, Morgan Stanley

Personalized Differential Privacy Privacy-Preserving ML Differential Privacy Differentially Private Stochastic Gradient Descent

Ask PDF

Jun 24, 2026

$S^{2}$ -FracMix: Label-Preserving Self-Saliency Mixup Augmentation

Data augmentation is known to improve generalization of deep visual models. Recent methods favor mixup strategies that generate interpolated samples to improve model performance. However, these techniques not only incur significant computational overhead, they also lead to semantic disruption of augmentation data due to cross-sample mixing. We first propose Self-Saliency (

S^2

) Mixup, which constructs challenging yet label-consistent samples by extracting multi-scale salient patches and reinserting them into non-salient regions of the same image. This promotes scale-invariant feature learning while avoiding cross-sample interference. To further enhance model robustness, we introduce FracMix, a mixing scheme that injects self-similarity patterns into salient regions using adaptive ratios. Collectively, our unified framework,

S^{2}

-FracMix, enables simultaneous learning from fractal and non-fractal structures within a single image, yielding a targeted and structurally coherent augmentation strategy. We theoretically analyze the advantage of our technique, and empirically establish its superiority over the existing methods by achieving state-of-the-art performance in extensive evaluation with seven benchmarks across classification (coarse and fine-grained), robustness, calibration, object detection, and transfer learning tasks. Project page is available at fracmix-data-augmentation.github.io

Khawar Islam, Arif Mahmood, Xin Jin +1

The University of Melbourne, Australia · Information Technology University, Pakistan · Westlake University, China

Data Augmentation Mixup

Ask PDF

Abstract

Data Augmentation

Explore similar work

InstructMixup: Instruction-Guided Salient Patch Editing for Robust Data Augmentation

Amplified Patch-Level Differential Privacy for Free via Random Cropping

S2S^{2}S2-FracMix: Label-Preserving Self-Saliency Mixup Augmentation

$S^{2}$ -FracMix: Label-Preserving Self-Saliency Mixup Augmentation