You May Be Running the Wrong Inception Crop

Jason Chuan-Chih Chou

cs.LGOct 2, 2026

You May Be Running the Wrong Inception Crop

Authors: Jason Chuan-Chih Chou

Organizations: Cohere Labs Community

Abstract

A decade after its inception, Inception crop has become the standard crop-based data augmentation method for training deep vision models. Not only is its practice of uniformly sampling crop scale and aspect ratio widely adopted, but also its lower and upper bounds, with the scale lower bound being the sole exception that is sometimes tuned. It is therefore surprising that the standard implementation in the TensorFlow / JAX ecosystem samples crop scale with probability density function $f(A) \propto \frac{1}{\sqrt{A}}$ unlike the PyTorch counterpart, which follows the original description. Motivated by this discovery, we train 522 ViT-S/16 models on the ImageNet-1k dataset with various training budgets and crop scale distributions. We reach $78.78\pm0.09$ top-1 val. accuracy with 90 epochs of training budget and find that 1. Higher training budget requires stronger augmentation; 2. Lower tail of the distribution of the crop scale determines the augmentation strength of Inception crop; 3. Models trained with higher training budget exhibit sparser saliency, regardless of the crop scale distribution or weight decay. Based on 2. we revisit the performance of Beta crop, whose softer cutoff allows it to optimize model performance across training budgets with less compromise. We replicate 1. and 3. with Scion optimizer in addition to AdamW, suggesting that the results may be general.

Figures & tables

Figure 1: Crop area distribution of the standard implementation of Inception crop in TensorFlow with default parameters for $256\times 512$ and $512\times 256$ images, $N=10^{7}$ . The probability density function of the distribution is approximately proportional to $\frac{1}{\sqrt{A}}$ but truncated at larger area when some aspect ratios become inadmissible. Note that landscape and portrait images have slightly different distributions.

Panel of Figure 2 — Figure 2: Probability heatmaps of the crop size $(h,w)$ given by the TensorFlow implementation for landscape and portrait images with default parameters $r_{\mathrm{min}}=\frac{3}{4},r_{\mathrm{max}}=\frac{4}{3},a_{\mathrm{min}}=0.05,a_{\mathrm{max}}=1,N=10^{7}$ .

Figure 3: Crop area distribution of Beta crop with selected values of $\alpha$ and $\beta$ for landscape image ( $h_{o}=256,w_{o}=512$ ), $r_{\min}=\frac{3}{4},r_{\max}=\frac{4}{3},a_{\min}=0.05,a_{\max}=1,N=10^{7}$ samples for each distribution. $\alpha=\beta=1$ results in uniform distribution as expected except truncation at higher crop scale due to inadmissible aspect ratios.

	TensorFlow Inception crop						PyTorch Inception crop
$a_{\mathrm{min}}$	0.025	0.05	0.1	0.15	0.2	0.25	0.025	0.05	0.1	0.15	0.2	0.25
30ep	64.59	65.55	67.03	68.06	68.78	69.15	67.08	67.35	68.26	68.74	69.26	69.67
60ep	72.77	73.43	74.29	74.77	74.99	75.25	74.35	74.77	74.86	75.08	75.20	75.55
90ep	75.76	76.25	76.77	77.03	77.21	77.16	76.81	76.92	77.21	77.21	77.35	77.16
150ep	77.90	78.29	78.60	78.59	78.52	78.39	78.58	78.64	78.75	78.70	78.51	78.43
300ep	79.43	79.75	79.64	79.49	79.20	79.01	79.80	79.73	79.62	79.18	79.13	78.86

Table 1: Mean top-1 val. accuracy (original label), TensorFlow vs. PyTorch Inception crop. As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound

a_{\mathrm{min}}

decreases (stronger augmentation).

Panel of Figure 4 — Figure 4: Top-1 val. accuracy (original label), TensorFlow vs. PyTorch Inception crop, with sample standard deviation $\sigma_{\mathrm{tf}},\sigma_{\mathrm{torch}},(\sigma^{2}_{\mathrm{tf}}+\sigma^{2}_{\mathrm{torch}})^{\frac{1}{2}}$ plotted as error bar. As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound $a_{\mathrm{min}}$ decreases (stronger augmentation).

	TensorFlow Inception crop						PyTorch Inception crop
$a_{\mathrm{min}}$	0.025	0.05	0.1	0.15	0.2	0.25	0.025	0.05	0.1	0.15	0.2	0.25
30ep	72.23	73.22	74.77	75.73	76.45	76.84	74.77	75.11	75.90	76.39	76.88	77.21
60ep	80.18	80.75	81.30	81.68	81.88	82.13	81.44	81.73	81.86	81.95	82.09	82.37
90ep	82.65	83.10	83.31	83.57	83.64	83.56	83.39	83.39	83.65	83.58	83.59	83.50
150ep	84.21	84.51	84.67	84.51	84.42	84.36	84.57	84.65	84.56	84.47	84.33	84.28
300ep	85.30	85.39	85.25	85.01	84.71	84.45	85.32	85.27	85.06	84.81	84.59	84.44

Table 2: Mean top-1 val. accuracy (ReaL), TensorFlow vs. PyTorch Inception crop. As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound

a_{\mathrm{min}}

decreases (stronger augmentation).

Panel of Figure 5 — Figure 5: Top-1 val. accuracy (ReaL), TensorFlow vs. PyTorch Inception crop, with sample standard deviation $\sigma_{\mathrm{tf}},\sigma_{\mathrm{torch}},(\sigma^{2}_{\mathrm{tf}}+\sigma^{2}_{\mathrm{torch}})^{\frac{1}{2}}$ plotted as error bar. As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound $a_{\mathrm{min}}$ decreases (stronger augmentation).

Figure 6: Top-1 val. accuracy (original label), Beta crop with $\alpha=1$ . Solid line: $\beta=3$ , Dashed line: $\beta=5$

Figure 7: Top-1 val. accuracy (original label), Beta crop with $\beta=1$ . Top-1 val. accuracy vs. $\alpha$ , $a_{\mathrm{min}}=0.05$ is plotted on the right. For $\alpha=\beta=1$ , we reuse the data points of the PyTorch Inception crop experiments.

	Original label						ReaL
$\alpha$	1.149	1.287	1.551	1.818	2.099	2.403	1.149	1.287	1.551	1.818	2.099	2.403
30ep	67.47	67.92	68.67	69.22	69.61	69.87	75.16	75.58	76.44	76.97	77.30	77.58
60ep	74.59	74.79	75.24	75.32	75.48	75.51	81.72	81.85	82.20	82.17	82.35	82.35
90ep	76.89	76.99	77.39	77.25	77.15	77.29	83.48	83.56	83.72	83.59	83.54	83.72
150ep	78.63	78.49	78.73	78.59	78.63	78.59	84.72	84.48	84.63	84.48	84.41	84.42
300ep	79.57	79.60	79.42	79.27	78.95	79.04	85.15	85.14	84.90	84.77	84.58	84.55

Table 3: Mean top-1 val. accuracy of Beta crop, matching strength, original label (left) and ReaL (right). As the training budget increases from 30 epochs to 300 epochs, the optimal

\alpha

decreases (stronger augmentation).

\alpha=1.149

may be an exception possibly due to too many crops near zero area. In future iterations we may want small but nonzero

a_{\min}

(say

0<a_{\min}\leq 0.025

) instead while keeping the same 5th percentile crop scale.

Panel of Figure 8 — Figure 8: Diff. in top-1 val. accuracy (Beta crop, matching strength - PyTorch), original label (left) and ReaL (right), with $(\sigma^{2}_{\mathrm{beta}}+\sigma^{2}_{\mathrm{torch}})^{\frac{1}{2}}$ plotted as error bar.

Panel of Figure 9 — Figure 9: Mean gradient entropy $\overline{S_{\mathrm{grad}}}$ defined in eq. 2 vs. top-1 val. accuracy for all 450 Inception crop and Beta crop experiments with AdamW optimizer, color-coded by the 5th percentile crop scale (left) and training budget (right). For reference and sanity check, $S_{\mathrm{grad}}\leq\log(224^{2})=10.82$ given our input resolution. Dashed lines mark the average of the experiments of the given training budget.

Panel of Figure 10 — Figure 10: Mean Grad-CAM entropy $\overline{S_{\mathrm{grad\text{-}cam}}}$ (left) and mean relevance entropy $\overline{S_{\mathrm{rel}}}$ (right) vs. top-1 val. accuracy for all 450 Inception crop and Beta crop experiments with AdamW optimizer, color-coded by the training budget. For reference and sanity check, $S_{\mathrm{grad\text{-}cam}},S_{\mathrm{rel}}\leq\log(14^{2})=5.28$ given our input resolution and patch size. Dashed lines mark the average of the experiments of the given training budget.

	Original label		ReaL
	baseline	optimized	baseline	optimized
30ep	73.31 $\pm 0.09$	74.46 $\pm 0.19$	80.83 $\pm 0.07$	81.70 $\pm 0.08$
60ep	77.44 $\pm 0.09$	77.76 $\pm 0.15$	84.09 $\pm 0.10$	84.16 $\pm 0.16$
90ep	78.68 $\pm 0.09$	78.78 $\pm 0.09$	84.94 $\pm 0.05$	84.96 $\pm 0.07$
150ep	79.65 $\pm 0.07$	—	85.41 $\pm 0.05$	—
300ep	80.10 $\pm 0.14$	80.23 $\pm 0.17$	85.66 $\pm 0.21$	85.71 $\pm 0.15$

Table 4: Top-1 val. accuracy of Scion experiments,

a_{\mathrm{min}}=0.05

baseline vs. optimized

a_{\mathrm{min}}

for the training budget, original label (left) and ReaL (right). As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound

a_{\mathrm{min}}

decreases (stronger augmentation) while

a_{\mathrm{min}}=0.05

is already expected to be optimal for 150ep.

Panel of Figure 11 — Figure 11: Mean gradient entropy $\overline{S_{\mathrm{grad}}}$ (left), mean Grad-CAM entropy $\overline{S_{\mathrm{grad\text{-}cam}}}$ (middle), and mean relevance entropy $\overline{S_{\mathrm{rel}}}$ (right) vs. top-1 val. accuracy for all 72 Scion experiments including that of table 4 and WD sweep ( table 18 ), color-coded by training budget.

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 12: Training loss of the linear head variant of ViT-S/16 from Beyer et al. (2022a) for 90 epochs using the Big Vision ( Beyer et al., 2022b ) first-party code (cyan) vs. our reimplementation in TorchVision ( maintainers and contributors, 2016 ) (red).

Figure 13: Probability heatmap of the crop size $(h,w)$ given by the PyTorch Inception crop ( RandomResizedCrop , RRC) for landscape image ( $h_{o}=256,w_{o}=512$ ) with the same parameters as in fig. 2 : $r_{\mathrm{min}}=\frac{3}{4},r_{\mathrm{max}}=\frac{4}{3},a_{\mathrm{min}}=0.05,a_{\mathrm{max}}=1,N=10^{7}$ . The counterpart for portrait image ( $h_{o}=512,w_{o}=256$ is indistinguishable after image transposition.

Panel of Figure 14 — Figure 14: Color grids, before and after calling Contrast( $\cdot$ , 1.9) and their RGB values.

Figure 15: Crop area distribution of the reimplementation of TensorFlow Inception crop in PyTorch with default parameters for $256\times 512$ and $512\times 256$ images, $N=10^{7}$ . The results are indistinguishable from fig. 1 .

$a_{\mathrm{min}}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	64.59 $\pm 0.11$	65.55 $\pm 0.07$	67.03 $\pm 0.36$	68.06 $\pm 0.19$	68.78 $\pm 0.05$	69.15 $\pm 0.06$
60ep	72.77 $\pm 0.21$	73.43 $\pm 0.13$	74.29 $\pm 0.15$	74.77 $\pm 0.05$	74.99 $\pm 0.08$	75.25 $\pm 0.10$
90ep	75.76 $\pm 0.04$	76.25 $\pm 0.05$	76.77 $\pm 0.09$	77.03 $\pm 0.08$	77.21 $\pm 0.13$	77.16 $\pm 0.16$
150ep	77.90 $\pm 0.07$	78.29 $\pm 0.06$	78.60 $\pm 0.06$	78.59 $\pm 0.10$	78.52 $\pm 0.06$	78.39 $\pm 0.13$
300ep	79.43 $\pm 0.07$	79.75 $\pm 0.07$	79.64 $\pm 0.14$	79.49 $\pm 0.12$	79.20 $\pm 0.10$	79.01 $\pm 0.14$

Appendix

Table 5: Top-1 val. accuracy (original label), TensorFlow Inception crop. As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound

a_{\mathrm{min}}

decreases (stronger augmentation).

$a_{\mathrm{min}}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	67.08 $\pm 0.41$	67.35 $\pm 0.33$	68.26 $\pm 0.20$	68.74 $\pm 0.20$	69.26 $\pm 0.13$	69.67 $\pm 0.17$
60ep	74.35 $\pm 0.08$	74.77 $\pm 0.08$	74.86 $\pm 0.13$	75.08 $\pm 0.22$	75.20 $\pm 0.07$	75.55 $\pm 0.11$
90ep	76.81 $\pm 0.16$	76.92 $\pm 0.13$	77.21 $\pm 0.19$	77.21 $\pm 0.17$	77.35 $\pm 0.14$	77.16 $\pm 0.09$
150ep	78.58 $\pm 0.02$	78.64 $\pm 0.18$	78.75 $\pm 0.04$	78.70 $\pm 0.10$	78.51 $\pm 0.17$	78.43 $\pm 0.11$
300ep	79.80 $\pm 0.06$	79.73 $\pm 0.12$	79.62 $\pm 0.05$	79.18 $\pm 0.03$	79.13 $\pm 0.04$	78.86 $\pm 0.01$

Appendix

Table 6: Top-1 val. accuracy (original label), PyTorch Inception crop. As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound

a_{\mathrm{min}}

decreases (stronger augmentation).

$a_{\mathrm{min}}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	72.23 $\pm 0.13$	73.22 $\pm 0.18$	74.77 $\pm 0.39$	75.73 $\pm 0.28$	76.45 $\pm 0.02$	76.84 $\pm 0.11$
60ep	80.18 $\pm 0.23$	80.75 $\pm 0.02$	81.30 $\pm 0.15$	81.68 $\pm 0.08$	81.88 $\pm 0.08$	82.13 $\pm 0.03$
90ep	82.65 $\pm 0.05$	83.10 $\pm 0.03$	83.31 $\pm 0.08$	83.57 $\pm 0.06$	83.64 $\pm 0.07$	83.56 $\pm 0.16$
150ep	84.21 $\pm 0.06$	84.51 $\pm 0.10$	84.67 $\pm 0.03$	84.51 $\pm 0.08$	84.42 $\pm 0.07$	84.36 $\pm 0.10$
300ep	85.30 $\pm 0.09$	85.39 $\pm 0.09$	85.25 $\pm 0.11$	85.01 $\pm 0.07$	84.71 $\pm 0.09$	84.45 $\pm 0.19$

Appendix

Table 7: Top-1 val. accuracy (ReaL), TensorFlow Inception crop. As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound

a_{\mathrm{min}}

decreases (stronger augmentation).

$a_{\mathrm{min}}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	74.77 $\pm 0.33$	75.11 $\pm 0.32$	75.90 $\pm 0.18$	76.39 $\pm 0.14$	76.88 $\pm 0.10$	77.21 $\pm 0.09$
60ep	81.44 $\pm 0.11$	81.73 $\pm 0.09$	81.86 $\pm 0.12$	81.95 $\pm 0.11$	82.09 $\pm 0.09$	82.37 $\pm 0.05$
90ep	83.39 $\pm 0.15$	83.39 $\pm 0.09$	83.65 $\pm 0.14$	83.58 $\pm 0.17$	83.59 $\pm 0.07$	83.50 $\pm 0.08$
150ep	84.57 $\pm 0.14$	84.65 $\pm 0.17$	84.56 $\pm 0.05$	84.47 $\pm 0.08$	84.33 $\pm 0.06$	84.28 $\pm 0.07$
300ep	85.32 $\pm 0.15$	85.27 $\pm 0.03$	85.06 $\pm 0.01$	84.81 $\pm 0.09$	84.59 $\pm 0.06$	84.44 $\pm 0.08$

Appendix

Table 8: Top-1 val. accuracy (ReaL), PyTorch Inception crop. As the training budget increases from 30 epochs to 300 epochs, the optimal crop scale lower bound

a_{\mathrm{min}}

decreases (stronger augmentation).

$a_{min}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	5.1e-4	8.0e-4	6.6e-3	0.013	3.3e-3	7.5e-3
60ep	2.6e-4	1.e-4	7.3e-3	0.074	0.029	0.024
90ep	4.0e-4	1.2e-3	0.022	0.16	0.29	0.93
150ep	6.6e-5	0.034	0.021	0.25	0.91	0.68
300ep	2.5e-3	0.86	0.84	0.013	0.36	0.14

Appendix

Table 9: T-test p-values of the differences in top-1 val. accuracies (original) between the TensorFlow and PyTorch Inception crop, two-sided, equal variance, color-coded ( p <0.05 , p <0.01 , p <0.001 ).

$a_{min}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	2.3e-4	8.8e-4	0.01	0.024	1.6e-3	0.011
60ep	1.0e-3	5.7e-5	7.6e-3	0.027	0.034	2.4e-3
90ep	1.1e-3	7.2e-3	0.022	0.88	0.47	0.63
150ep	0.015	0.31	0.04	0.59	0.18	0.33
300ep	0.79	0.091	0.036	0.035	0.1	0.95

Appendix

Table 10: T-test p-values of the differences in top-1 val. accuracies (ReaL) between the TensorFlow and PyTorch Inception crop, two-sided, equal variance, color-coded ( p <0.05 , p <0.01 , p <0.001 ).

$a_{min}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	1.8e-4	1	2.1e-3	2.6e-5	3.e-7	2.5e-7
60ep	9.8e-3	1	1.7e-3	7.1e-5	6.2e-5	4.4e-5
90ep	1.9e-4	1	9.4e-4	1.5e-4	2.7e-4	7.0e-4
150ep	1.8e-3	1	2.9e-3	9.9e-3	9.8e-3	0.31
300ep	5.e-3	1	0.3	0.037	1.3e-3	1.2e-3

Appendix

Table 11: T-test p-values of the top-1 val. accuracies (original) between the default

a_{min}=0.05

and other values with various training budgets, TensorFlow Inception crop, two-sided, equal variance, color-coded ( p <0.05 , p <0.01 , p <0.001 ).

$a_{min}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	1.5e-3	1	3.4e-3	2.1e-4	7.4e-6	8.5e-6
60ep	0.013	1	3.7e-3	3.4e-5	2.3e-5	5.2e-7
90ep	1.8e-4	1	0.016	2.8e-4	3.1e-4	8.5e-3
150ep	0.013	1	0.071	0.96	0.26	0.13
300ep	0.27	1	0.16	4.6e-3	7.3e-4	1.4e-3

Appendix

Table 12: T-test p-values of the top-1 val. accuracies (ReaL) between the default

a_{min}=0.05

and other values with various training budget, TensorFlow Inception crop, two-sided, equal variance, color-coded ( p <0.05 , p <0.01 , p <0.001 ).

$a_{min}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	0.43	1	0.016	3.4e-3	7.3e-4	4.3e-4
60ep	2.7e-3	1	0.38	0.081	2.1e-3	5.3e-4
90ep	0.39	1	0.1	0.083	0.019	0.066
150ep	0.62	1	0.34	0.6	0.42	0.17
300ep	0.45	1	0.21	1.5e-3	1.2e-3	2.2e-4

Appendix

Table 13: T-test p-values of the top-1 val. accuracies (original) between

a_{min}=0.05

and other values with various training budgets, PyTorch Inception crop, two-sided, equal variance, color-coded ( p <0.05 , p <0.01 , p <0.001 ).

$a_{min}$	0.025	0.05	0.1	0.15	0.2	0.25
30ep	0.27	1	0.019	3.1e-3	7.4e-4	3.8e-4
60ep	0.023	1	0.18	0.055	6.9e-3	4.3e-4
90ep	0.98	1	0.057	0.17	0.042	0.19
150ep	0.6	1	0.47	0.18	0.039	0.027
300ep	0.56	1	2.6e-4	1.1e-3	4.9e-5	8.6e-5

Appendix

Table 14: T-test p-values of the top-1 val. accuracies (ReaL) between

a_{min}=0.05

and other values with various training budget, PyTorch Inception crop, two-sided, equal variance, color-coded ( p <0.05 , p <0.01 , p <0.001 ).

Panel of Figure 16 — Figure 16: Top-1 val. accuracy of PyTorch Inception crop (solid line) vs. TensorFlow Inception crop (dashed line), original label (left) and ReaL (right), plotted with the 5th percentile crop scale as the x-axis. TensorFlow Inception crop generally underperforms except 300ep where it matches PyTorch Inception crop even with such unified augmentation strength measure.

Figure 17: Top-1 val. accuracy (ReaL), Beta crop with $\alpha=1$ . Solid line: $\beta=3$ , Dashed line: $\beta=5$

Figure 18: Top-1 val. accuracy (ReaL), Beta crop with $\beta=1$ . Top-1 val. accuracy vs. $\alpha$ , $a_{min}=0.05$ is plotted on the right. For $\alpha=\beta=1$ , we reuse the data points of the PyTorch Inception crop experiments.

$\alpha$	1.149	1.287	1.551	1.818	2.099	2.403
30ep	67.47 $\pm 0.18$	67.92 $\pm 0.20$	68.67 $\pm 0.15$	69.22 $\pm 0.25$	69.61 $\pm 0.19$	69.87 $\pm 0.04$
60ep	74.59 $\pm 0.13$	74.79 $\pm 0.14$	75.24 $\pm 0.17$	75.32 $\pm 0.12$	75.48 $\pm 0.15$	75.51 $\pm 0.04$
90ep	76.89 $\pm 0.04$	76.99 $\pm 0.23$	77.39 $\pm 0.11$	77.25 $\pm 0.15$	77.15 $\pm 0.14$	77.29 $\pm 0.21$
150ep	78.63 $\pm 0.21$	78.49 $\pm 0.07$	78.73 $\pm 0.06$	78.59 $\pm 0.24$	78.63 $\pm 0.09$	78.59 $\pm 0.12$
300ep	79.57 $\pm 0.10$	79.60 $\pm 0.11$	79.42 $\pm 0.10$	79.27 $\pm 0.15$	78.95 $\pm 0.23$	79.04 $\pm 0.12$

Appendix

Table 15: Top-1 val. accuracy of Beta crop, matching strength (original label). As the training budget increases from 30 epochs to 300 epochs, the optimal

\alpha

decreases (stronger augmentation).

\alpha=1.149

may be an exception possibly due to too many crops near zero area. In future iterations we may want small but nonzero

a_{\min}

(say

0<a_{\min}\leq 0.025

) instead while keeping the same 5th percentile crop scale.

$\alpha$	1.149	1.287	1.551	1.818	2.099	2.403
30ep	75.16 $\pm 0.15$	75.58 $\pm 0.17$	76.44 $\pm 0.12$	76.97 $\pm 0.23$	77.30 $\pm 0.08$	77.58 $\pm 0.10$
60ep	81.72 $\pm 0.05$	81.85 $\pm 0.06$	82.20 $\pm 0.10$	82.17 $\pm 0.16$	82.35 $\pm 0.15$	82.35 $\pm 0.05$
90ep	83.48 $\pm 0.04$	83.56 $\pm 0.11$	83.72 $\pm 0.09$	83.59 $\pm 0.06$	83.54 $\pm 0.17$	83.72 $\pm 0.15$
150ep	84.72 $\pm 0.16$	84.48 $\pm 0.08$	84.63 $\pm 0.03$	84.48 $\pm 0.10$	84.41 $\pm 0.08$	84.42 $\pm 0.06$
300ep	85.15 $\pm 0.16$	85.14 $\pm 0.03$	84.90 $\pm 0.10$	84.77 $\pm 0.16$	84.58 $\pm 0.11$	84.55 $\pm 0.14$

Appendix

Table 16: Rop-1 val. accuracy of Beta crop, matching strength (ReaL). As the training budget increases from 30 epochs to 300 epochs, the optimal

\alpha

decreases (stronger augmentation).

Figure 19: Sample saliency maps given by the rectified gradient for the 1st images of the first 25 classes $c\in[0\ldotp\ldotp 24]$ in torchvision.datasets.ImageNet validation split. We use a model trained with 300ep training budget and the TensorFlow Inception crop with $a_{\mathrm{min}}=0.05$ . Though conceptually the simplest, the saliency maps given by the rectified gradient are the least interpretable of the 3.

Figure 20: Mean gradient entropy $\overline{S_{\mathrm{grad}}}$ defined in eq. 2 vs. top-1 val. accuracy (ReaL) for all 450 Inception crop and Beta crop experiments with AdamW optimizer, color-coded by $a_{\mathrm{min}}$ .

Panel of Figure 21 — Figure 21: Distribution of the gradient entropy $S_{\mathrm{grad}}$ defined in eq. 2 for the 50000 validation set images given by models trained with the TensorFlow Inception crop (left) and PyTorch Inception crop (right) with $a_{\mathrm{min}}=0.05$ , color-coded by the training budget.

Figure 22: Sample saliency maps given by Grad-CAM at the last layer for the 1st images of the first 25 classes $c\in[0\ldotp\ldotp 24]$ in torchvision.datasets.ImageNet validation split. We use a model trained with 300ep training budget and the TensorFlow Inception crop with $a_{\mathrm{min}}=0.05$ .

Panel of Figure 23 — Figure 23: Distribution of the Grad-CAM entropy $S_{\mathrm{grad-cam}}$ for the 50000 validation set images given by models trained with the TensorFlow Inception crop (left) and PyTorch Inception crop (right) with $a_{\mathrm{min}}=0.05$ , color-coded by the training budget.

Figure 24: Sample saliency maps given by LRP at the last layer for the 1st images of the first 25 classes $c\in[0\ldotp\ldotp 24]$ in torchvision.datasets.ImageNet validation split. We use a model trained with 300ep training budget and the TensorFlow Inception crop with $a_{\mathrm{min}}=0.05$ .

Panel of Figure 25 — Figure 25: Distribution of the relevance entropy $S_{\mathrm{rel}}$ for the 50000 validation set images given by models trained with the TensorFlow Inception crop (left) and PyTorch Inception crop (right) with $a_{\mathrm{min}}=0.05$ , color-coded by the training budget.

	Sign	Spectral	Bias
Norm	$d_{\mathrm{in}}\max_{i,j}\|A_{i,j}\|$	$\sqrt{\frac{d_{\mathrm{in}}}{d_{\mathrm{out}}}}\\|{\bm{A}}\\|_{\mathcal{S}_{\infty}}$	$\mathrm{RMS}$
LMO	${\bm{A}}\mapsto-\frac{\sign({\bm{A}})}{d_{\mathrm{in}}}$	${\bm{A}}\mapsto-\sqrt{\frac{d_{\mathrm{out}}}{d_{\mathrm{in}}}}{\bm{U}}{\bm{V}}^{\top}$	${\bm{b}}_{\ell}\mapsto-\tfrac{{\bm{b}}_{\ell}}{\\|{\bm{b}}_{\ell}\\|_{\mathrm{RMS}}}$

Appendix

Table 17: Norms and the associated

\operatorname{lmo}

s as normalized in our experiments. Sign and Spectral assume matrix weight

\bm{\theta}_{l}={\bm{A}}\in\mathbb{R}^{d_{\mathrm{out}}\times d_{\mathrm{in}}}

while Bias assumes vector weight

\bm{\theta}_{l}={\bm{b}}_{\ell}\in\mathbb{R}^{d_{\mathrm{out}}}

{\bm{U}}{\bm{V}}^{\top}

refers to the reduced SVD of the input matrix with unitary matrices

{\bm{U}}

and

{\bm{V}}^{\top}

from the full SVD

{\bm{A}}={\bm{U}}\text{diag}(\bm{\sigma}){\bm{V}}^{\top}

while

\|{\bm{A}}\|_{\mathcal{S}_{\infty}}=\max(\bm{\sigma})

is the spectral norm of the matrix.

	Original label				ReaL
$\lambda_{l}$	0.04	0.08	0.12	0.16	0.04	0.08	0.12	0.16
30ep	72.08 $\pm 0.19$	73.31 $\pm 0.09$	73.24 $\pm 0.21$	72.50 $\pm 0.22$	79.46 $\pm 0.18$	80.83 $\pm 0.07$	80.81 $\pm 0.13$	80.18 $\pm 0.20$
60ep	76.98 $\pm 0.09$	77.44 $\pm 0.09$	76.88 $\pm 0.14$	76.07 $\pm 0.16$	83.58 $\pm 0.08$	84.09 $\pm 0.10$	83.75 $\pm 0.15$	83.26 $\pm 0.18$
90ep	78.42 $\pm 0.19$	78.68 $\pm 0.09$	78.32 $\pm 0.04$	77.31 $\pm 0.12$	84.51 $\pm 0.08$	84.94 $\pm 0.05$	84.88 $\pm 0.05$	84.11 $\pm 0.10$
150ep	79.27 $\pm 0.08$	79.65 $\pm 0.07$	79.05 $\pm 0.15$	78.31 $\pm 0.07$	84.94 $\pm 0.09$	85.41 $\pm 0.05$	85.20 $\pm 0.14$	84.69 $\pm 0.08$
300ep	79.67 $\pm 0.06$	80.10 $\pm 0.14$	79.98 $\pm 0.10$	78.64 $\pm 0.32$	85.04 $\pm 0.10$	85.66 $\pm 0.21$	85.66 $\pm 0.09$	84.54 $\pm 0.35$

Appendix

Table 18: Top-1 val. accuracy of the Scion optimizer WD sweep, original label (left) and ReaL (right).

Panel of Figure 26 — Figure 26: Mean gradient entropy $\overline{S_{\mathrm{grad}}}$ vs. top-1 val. accuracy, original label (left) and ReaL (right), for all 72 experiments with Scion optimizer, color-coded by $\lambda_{l}$ .

Data Augmentation

Data augmentation expands or transforms training examples to improve learning, data efficiency, and robustness to distribution shifts.

See all 249 papers

InstructMixup: Instruction-Guided Salient Patch Editing for Robust Data Augmentation
Jul 21, 2026Khawar Islam, Arif Mahmood, Xin Jin +1Synthetic Data Augmentation Data Augmentation
The University of Melbourne, Melbourne, Australia · The Information Technology University, Lahore, Pakistan · Westlake University, China
Amplified Patch-Level Differential Privacy for Free via Random Cropping
Mar 25, 2026Kaan Durmaz, Jan Schuchardt, Sebastian Schmidt +1Personalized Differential Privacy Privacy-Preserving ML
Technical University of Munich · Machine Learning Research, Morgan Stanley
$S^{2}$ -FracMix: Label-Preserving Self-Saliency Mixup Augmentation
Jun 24, 2026Khawar Islam, Arif Mahmood, Xin Jin +1Data Augmentation Mixup
The University of Melbourne, Australia · Information Technology University, Pakistan · Westlake University, China

Jul 21, 2026J/K move · Enter open · S save

InstructMixup: Instruction-Guided Salient Patch Editing for Robust Data Augmentation

Khawar Islam, Arif Mahmood, Xin Jin, Naveed Akhtar

The University of Melbourne, Melbourne, Australia · The Information Technology University, Lahore, Pakistan · Westlake University, China

Synthetic Data Augmentation Data Augmentation

Open paper AskPDF arXiv

In image and video technologies, data augmentation is widely used to improve the generalization of deep visual models, and mixup-based strategies that interpolate between samples have become the dominant approach. However, computing informative mixing regions adds substantial overhead, and blending content across different images frequently disrupts the semantic integrity of the resulting sample. We propose \our{}, a data augmentation method that constructs challenging yet label-consistent training samples entirely within a single visual sample. \our{} first extracts multi-scale salient patches from the sample using a lightweight saliency detector, refines each patch with an instruction-guided generative model, and blends the edited patch back into the non-salient regions of the same sample; because the generative edits are computed once and cached offline, this step adds negligible training cost. To further diversify the learned representation, \our{} injects self-similar fractal structure into the same salient regions at an adaptive ratio, so each training sample carries both fractal and non-fractal structure. We derive a second-order approximation of the resulting vicinal risk, showing that the method simultaneously enforces invariance to the generative edit and suppresses curvature along the perturbed salient directions, and we verify both predictions empirically. We evaluate on small to large backbones for instance Convolutional Neural Networks (CNNs), Vision Transformers (ViTs) and Vision-Language Foundational Models (VLMs) across seven benchmarks covering coarse- and fine-grained classification, robustness to corruption and occlusion, calibration, and transfer and self-supervised learning, InstructMixup outperforms nine competing augmentation methods, surpassing the strongest baseline across all benchmarks.

Abstract

Data Augmentation

Explore similar work

InstructMixup: Instruction-Guided Salient Patch Editing for Robust Data Augmentation

Amplified Patch-Level Differential Privacy for Free via Random Cropping

S2S^{2}S2-FracMix: Label-Preserving Self-Saliency Mixup Augmentation

$S^{2}$ -FracMix: Label-Preserving Self-Saliency Mixup Augmentation