Pairwise AUC Optimization Needs Corrective Power: A Unified View

Jia Chen; Zhiyong Yang; Shilong Bao; Qianqian Xu; Qingming Huang

cs.LGOct 15, 2026

Pairwise AUC Optimization Needs Corrective Power: A Unified View

Authors: Jia Chen, Zhiyong Yang, Shilong Bao, Qianqian Xu, Qingming Huang

Organizations: School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

Abstract

Direct pairwise AUC-surrogate training is attractive because it trains toward the ranking metric used at test time, especially under class imbalance. This creates a direct alignment between the training objective and the evaluation metric, but such metric alignment alone does not guarantee that stochastic training remains corrective. We study this gap through Corrective Power: whether pairwise mini-batch updates produce stronger and more reliable improvements for more severe positive-negative ranking errors. Our main observation is that direct pairwise training can enter regions where severe errors remain common but no longer produce useful corrective motion. In such regions, even the mini-batch noise that would normally help SGD move along the steepest severe-error correction direction can become much weaker. In this paper, we seek a theoretical explanation for this by exploring the degenerate property of the U-statistics formed from the pairwise gradients therein. Our results provide a mechanistic explanation for several empirical patterns: (a) squared-hinge loss is often an effective AUC surrogate; (b) cross-entropy warm-up can help models that enter the AUC phase from a poor starting region; and (c) parameter-efficient fine-tuning from a pretrained model can reduce that need by starting closer to a favorable ranking geometry. Experiments on 8 CV tasks and 8 NLP tasks, using ResNet50, DenseNet121, DistilBERT-base, and Qwen2.5-1.5B-Instruct, support these predictions.

Figures & tables

Panel of Figure 1 — Figure 1: AUC dynamics comparison under different initialization. The background color shows AUC on a fixed two-dimensional parameter slice. Curves are simulated training paths; stars and large circles mark starts and final iterates. Panels (a) , (b) , and (c) use bad, slightly better, and good initializations. Details are in Appendix B.1 .

Panel of Figure 2 — Figure 2: Noise scale and surrogate weights illustrate two ingredients of corrective motion. (a) A one-dimensional projected escape simulation comparing a first-order $B^{-1}$ variance regime with a collapsed $B^{-2}$ variance regime. The left potential and the right heatmaps use the same state axis $z$ . (b) Update weights for five standard pairwise surrogates as functions of the pairwise margin $x$ . Simulation details are in Appendix B.2 .

Task	Mode	CE	Square	Hinge	Exp.	Logistic	Sq.-hinge
DenseNet121
Apple Disease	Direct	60.64 ±2.63	61.10 ±1.85	58.81 ±2.46	60.18 ±1.21	60.46 ±3.09	62.69 ±4.86
	Warm-up	–	61.10 ±1.85	59.91 ±2.60	61.03 ±1.92	60.54 ±2.02	62.69 ±4.86
Rice Disease	Direct	72.45 ±6.57	71.43 ±7.23	70.92 ±7.66	70.61 ±6.81	71.63 ±7.30	71.53 ±7.30
	Warm-up	–	74.90 ±6.31	73.98 ±6.26	74.90 ±6.41	71.63 ±7.30	75.71 ±8.61
APTOS Sev./Mod.	Direct	82.75 ±0.95	83.83 ±1.41	85.10 ±0.76	83.68 ±1.66	84.18 ±1.37	84.20 ±1.12

Table 1: Held-out AUC (%) for direct and CE-warm-started pairwise training. Representative DenseNet121 and Qwen2.5-1.5B-Instruct results are shown; ResNet50 and DistilBERT-base are reported in Table B.3 . Direct starts pairwise training immediately; Warm-up first runs CE and then switches to the indicated pairwise loss. Values are mean AUC (%) with sample standard deviations as subscripts; bold and underline mark the best and second-best results within each task.

Panel of Figure 3 — Figure 3: Real-run trajectory measures support the Corrective Power explanation. Four training-trajectory quantities computed from experiment runs. Panels (a,b) show the scaled first-order Hoeffding variance term on SNLI E/N along the steepest severe-error correction direction; shaded bands show one standard error after the same smoothing used for the displayed mean. $x$ marks denote the CE warm-up interval before the pairwise variance term is recorded. Panels (c,d) show ANLI R1 E/N trajectories and mark the CE-to-AUC switch. Definitions are in Appendix B.5 .

Panel of Figure 4 — Figure 4: Aggregate real-run measures support the Corrective Power explanation. Four aggregate quantities computed from the same experiment runs. Panels (a,b) report warm-up gains by loss and initialization, respectively. Panels (c,d) report relative weight proportions over wrong-margin intervals. Definitions are in Appendix B.5 .

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Source	Binary construction
ImageNet-LT Dogs	ImageNet-LT split files with ImageNet images	Tail dog classes versus sibling dog classes.
Dog Breeds	Stanford Dogs	Spitz, husky, and malamute versus Eskimo dog and Samoyed.
CRJ700/CRJ900	FGVC-Aircraft	Bombardier CRJ700 versus Bombardier CRJ900.
APTOS Sev./Mod.	APTOS diabetic-retinopathy fundus images	Severe-or-worse retinopathy versus moderate retinopathy.
DeepWeeds P/S	DeepWeeds	Parthenium weed versus Siam weed, with low-shot training construction.
Apple Disease	D-KAP apple leaf disease archive	Apple scab versus apple rot, with low-shot training construction.

Appendix

Table 3: CV task construction. Each row gives the source dataset and the binary ranking task used in the experiments.

Task	Source	Binary construction
GoEmotions Annoyance	GoEmotions, simplified	Annoyance label versus all other examples.
GoEmotions Fear	GoEmotions, simplified	Fear label versus all other examples.
ANLI R1 Ent./Neu.	ANLI Round 1 prepared split	Entailment versus neutral.
ANLI R3 Con./Neu.	ANLI Round 3 prepared split	Contradiction versus neutral.
ANLI R3 Ent./Neu.	ANLI Round 3 prepared split	Entailment versus neutral.
SNLI High-Overlap	SNLI prepared high-overlap split	Entailment versus neutral on a high-overlap slice.

Appendix

Table 4: NLP task construction. DistilBERT and Qwen use the same binary task definitions.

CV task	Train ( $N$ ; pos./neg.)	Val. ( $N$ ; pos./neg.)	Test ( $N$ ; pos./neg.)
ImageNet-LT Dogs	$6788$ ( $202/6586$ )	$1080$ ( $360/720$ )	$2700$ ( $900/1800$ )
Dog Breeds	$517$ ( $259/258$ )	$111$ ( $56/55$ )	$110$ ( $55/55$ )
CRJ700/CRJ900	$50$ ( $25/25$ )	$30$ ( $15/15$ )	$66$ ( $33/33$ )
APTOS Sev./Mod.	$1041$ ( $342/699$ )	$223$ ( $73/150$ )	$223$ ( $73/150$ )
DeepWeeds P/S	$50$ ( $25/25$ )	$30$ ( $15/15$ )	$80$ ( $40/40$ )
Apple Disease	$50$ ( $25/25$ )	$30$ ( $15/15$ )	$80$ ( $40/40$ )

Appendix

Table 5: CV split sizes. In the split columns,

N

is the total number of examples and pos./neg. gives the positive/negative counts.

NLP task	Train ( $N$ ; pos./neg.)	Val. ( $N$ ; pos./neg.)	Test ( $N$ ; pos./neg.)
GoEmotions Annoy.	$39069$ ( $2223/36846$ )	$4341$ ( $247/4094$ )	$5427$ ( $320/5107$ )
GoEmotions Fear	$39069$ ( $536/38533$ )	$4341$ ( $60/4281$ )	$5427$ ( $78/5349$ )
ANLI R1 E/N	$12423$ ( $5371/7052$ )	$667$ ( $334/333$ )	$667$ ( $334/333$ )
ANLI R3 C/N	$20000$ ( $10000/10000$ )	$798$ ( $396/402$ )	$798$ ( $396/402$ )
ANLI R3 E/N	$20000$ ( $10000/10000$ )	$804$ ( $402/402$ )	$804$ ( $402/402$ )
SNLI High-Overlap	$20000$ ( $10000/10000$ )	$4000$ ( $2000/2000$ )	$4000$ ( $2000/2000$ )

Appendix

Table 6: NLP split sizes. In the split columns,

N

is the total number of examples and pos./neg. gives the positive/negative counts.

Regime	Models/tasks	Optimization settings
CV, ImageNet-LT	ResNet50 on ImageNet-LT Dogs	$20$ epochs, batch/eval batch $128/256$ , SGD with momentum $0.9$ , learning rate $0.05$ , weight decay $10^{-4}$ , cosine annealing, positive batch fraction $0.25$ , CE-to-AUC switch LR scale $1.0$ .
CV, other ResNet50	Dog Breeds, CRJ700/CRJ900, APTOS Sev./Mod., DeepWeeds P/S	$10$ epochs, batch/eval batch $64/128$ , AdamW [ 85 ] , learning rate $10^{-4}$ , weight decay $10^{-4}$ , cosine annealing, positive batch fraction $0.25$ , CE-to-AUC switch LR scale $0.5$ .
CV, DenseNet121	Apple Disease, Rice Disease, APTOS Sev./Mod., HAM10000 M/N	$10$ epochs, batch/eval batch $48/128$ , AdamW, learning rate $10^{-4}$ , weight decay $10^{-4}$ , cosine annealing, positive batch fraction $0.25$ , CE-to-AUC switch LR scale $0.5$ .
DistilBERT NLP	Eight NLP tasks in Appendix B.4	$4$ epochs, batch/eval batch $32/64$ , gradient accumulation $1$ , AdamW, learning rate $2\cdot 10^{-4}$ , weight decay $0.01$ , cosine schedule with warm-up steps equal to $5\%$ of total optimizer updates, maximum length $128$ , positive batch fraction $0.25$ , max gradient norm $1.0$ , CE-to-AUC switch LR scale $0.5$ .
Qwen NLP	Same eight NLP tasks	$4$ epochs, batch/eval batch $24/48$ , gradient accumulation $1$ , AdamW, learning rate $8\cdot 10^{-5}$ , weight decay $0.01$ , cosine schedule with warm-up steps equal to $5\%$ of total optimizer updates, maximum length $256$ , positive batch fraction $0.25$ , max gradient norm $1.0$ , CE-to-AUC switch LR scale $0.5$ .

Appendix

Table 7: Base training settings for the experimental regimes.

Regime	Grid
CV	Warm-up ratios $0,0.01,0.025,0.03,0.05,0.075,0.10,0.15,0.20,0.25,0.30$ . Margins $0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9$ .
DistilBERT	Warm-up ratios $0,0.01,0.025,0.05,0.075,0.10,0.15,0.20,0.25,0.30$ . Margins $0.1,0.2,0.4,0.6,0.7$ .
Qwen	Warm-up ratios $0,0.02,0.04,0.06,0.08,0.10$ . Margins $0.1,0.2,0.4,0.6$ .

Appendix

Table 8: Warm-up-ratio and pairwise-margin grids used for pairwise AUC-surrogate runs.

Task / surrogate	Region	Severe projected drift	Log-variance–log- $B$ slope
APTOS / direct square	Good	$+0.273$ to $+1.043$	$-1.102$ to $-1.070$
APTOS / direct square	Bad	$-0.187$ to $+0.105$	$-1.250$ to $-1.185$
APTOS / direct exponential	Good	$+0.574$ to $+1.304$	$-1.099$ to $-1.081$
APTOS / direct exponential	Bad	$-0.104$ to $+0.164$	$-1.470$ to $-1.351$
APTOS / direct logistic	Good	$+0.552$ to $+0.687$	$-1.091$ to $-1.060$
APTOS / direct logistic	Bad	$-0.003$ to $+0.016$	$-1.562$ to $-1.412$

Appendix

Table 9: Fixed-checkpoint batch-size scaling of the severe-truncated projected variance. Good and bad regions are determined by severe projected drift. Entries report ranges over three independent reference/logging-pool splits.

Figure 5: Misranking trajectories for all five pairwise losses on DistilBERT GoEmotions Annoyance and SNLI E/N. Black curves are direct pairwise training; colored curves are CE warm-up followed by AUC training, with loss colors shown in the legend. Vertical dashed lines mark the CE-to-AUC switch.

Figure 6: Scaled first-order Hoeffding variance trajectories for all five pairwise losses on DistilBERT ANLI R3 C/N and ANLI R1 E/N. Black curves are direct pairwise training; colored curves are CE warm-up followed by AUC training, with loss colors shown in the legend. Shaded bands show one standard error across matching runs after the same trajectory smoothing used for the displayed mean.

x

marks denote the CE warm-up interval before the pairwise variance term is recorded, and vertical dashed lines mark the CE-to-AUC switch.

Task	$n$	Directional R/H/N	Isotropic R/H/N	Mean $\Delta$ AUC D/I	Mean $\Delta S_{\rm sev}$ D/I
ImageNet-LT dog-tail	16	2/6/8	0/0/16	$-2.541/-0.001$	$+6.036/-0.046$
APTOS	24	6/13/5	1/1/22	$-1.316/+0.035$	$+1.317/-0.033$
Total	40	8/19/13	1/1/38	—	—

Appendix

Table 10: Matched directional-intervention data at fixed direct-hinge checkpoints. R/H/N denotes rescue/harm/no clear effect. Endpoint changes are measured at the final intervention epoch relative to the shared no-noise baseline and are reported in percentage points; D/I denotes directional/isotropic.

Multiclass task	CE	Direct hinge	Warm-up hinge	Direct squared-hinge	Warm-up squared-hinge
Aircraft Airbus-6 LT	60.92	61.43	61.61	61.68	60.91
CIFAR-100 Medium-Mammals-5 LT	81.08	81.13	82.91	83.62	84.44
CIFAR-100 Vehicles-10 LT	86.47	88.30	88.64	87.97	89.08

Appendix

Table 11: Results on three long-tailed multiclass tasks (AUC%).

Multilabel task	BCE	Direct hinge	Warm-up hinge	Direct squared-hinge	Warm-up squared-hinge
Competition-5	80.86	82.75	83.40	81.51	83.09
Cardiac/Fluid-4	80.76	81.14	82.13	79.96	81.44
Consolidation/Cardiac-2	51.52	57.28	56.09	57.08	59.69

Appendix

Table 12: Results on three CheXpert multilabel tasks (AUC%).

Dataset	Metric	CE	Hinge direct	Hinge warm-up	Sq.-hinge direct	Sq.-hinge warm-up
iNaturalist	Macro-OVO-AUC	96.64	92.45	99.70	89.18	99.74
ImageNet-LT	Macro-OVO-AUC	98.50	97.87	98.45	97.69	98.91
iNaturalist	Macro-OVR-AUC	97.70	88.72	97.31	79.94	98.13
ImageNet-LT	Macro-OVR-AUC	94.80	93.21	94.81	92.95	96.08

Appendix

Table 13: Large-scale multiclass results on iNaturalist 2018 and ImageNet-LT (AUC%).

Task–model setting	Deep-AUC	Ours
Apple Disease–DenseNet121	55.56	62.69
Rice Disease–DenseNet121	77.76	75.71
APTOS Sev./Mod.–DenseNet121	82.12	84.20
HAM10000 M/N–DenseNet121	97.16	97.70
ImageNet-LT Dogs–ResNet50	94.56	95.83
Dog Breeds–ResNet50	85.74	87.00

Appendix

Table 14: Deep-AUC and our results on

17

task–model settings (AUC%).

U-Statistics

See all 11 papers

Active Learners as Efficient PRP Rerankers
May 14, 2026Jeremías Figueiredo Paschmann, Juan Kaplan, Francisco Nattero +3Pairwise Preference Learning Pairwise Comparison
ELIAS Lab, Departamento de Ingeniería, Universidad de San Andrés
Pairwise Learning Increases Data Efficiency in Deep Neural Networks
Oct 20, 2026Robert Bain, Marjan Rashidi, Elizabeth ChrastilData-Efficient Learning Small-Data Learning
Department of Neurobiology & Behavior, University of California, Irvine, Irvine, CA 92697, USA · Department of Cognitive Sciences, University of California, Irvine, Irvine, CA 92697, USA
Perturbation Sensitivity at Convergence: A Simple Signal for Identifying Spuriously Correlated Samples
Aug 5, 2026Nilesh KumarTraining Data Selection Classification

May 14, 2026J/K move · Enter open · S save

Active Learners as Efficient PRP Rerankers

Jeremías Figueiredo Paschmann, Juan Kaplan, Francisco Nattero, Santiago Barron+2

ELIAS Lab, Departamento de Ingeniería, Universidad de San Andrés

Pairwise Preference Learning Pairwise Comparison Pairwise Preference Evaluation Learning to Rank Active Learning LLM Reranking+3

Open paper AskPDF arXiv

Pairwise Ranking Prompting (PRP) elicits pairwise preference judgments from an LLM, which are then aggregated into a ranking, usually via classical sorting algorithms. However, judgments are noisy, order-sensitive, and sometimes intransitive, so sorting assumptions do not match the setting. Because sorting aims to recover a full permutation, truncating it to meet a call budget does not produce a dependable top-K. We thus reframe PRP reranking as active learning from noisy pairwise comparisons and show that active rankers are drop-in replacements that improve NDCG@10 per call in the call-constrained regime. Our noise-robust framework also introduces a randomized-direction oracle that uses a single LLM call per pair. This approach converts systematic position bias into zero-mean noise, enabling unbiased aggregate ranking without the cost of bidirectional calls.

Abstract

U-Statistics

Explore similar work

Active Learners as Efficient PRP Rerankers

Pairwise Learning Increases Data Efficiency in Deep Neural Networks

Perturbation Sensitivity at Convergence: A Simple Signal for Identifying Spuriously Correlated Samples