Consistent Learning-to-Defer with Expert-Conditional Advice

Yannis Montreuil; Leïna Montreuil; Axel Carlier; Lai Xing Ng; Wei Tsang Ooi

stat.MLMar 15, 2026

Consistent Learning-to-Defer with Expert-Conditional Advice

Authors: Yannis Montreuil, Leïna Montreuil, Axel Carlier, Lai Xing Ng, Wei Tsang Ooi

Organizations: School of Computing National University of Singapore Singapore, 117418 · Département de Mathématiques Sorbonne University Paris, 75005, France · Fédération ENAC ISAE-SUPAERO ONERA Université de Toulouse Toulouse, 31555, France · Agency for Science, Technology and Research Institute for Infocomm Research Singapore, 138634

Abstract

We introduce learning-to-defer with advice, a new setting in which a system learns both which expert should handle an input and which information that expert should receive. The policy acts before observing advice and minimizes prediction loss plus expert and acquisition fees. Training may reveal all advice sources or only a subset on each record. We show that composing routing and advice losses can be inconsistent and that augmented training can incur avoidable estimation variance. We propose reference conditional classification, which jointly learns a fixed expert's error probability and each action's error conditional on that expert's success or failure. The reference outcome is observed on every labeled record, so records with missing advice still inform the estimates. We prove a consistency bound for bounded task losses, show that limited training preserves the full population objective, and establish variance reductions for the analyzed fits. On FEVER and Sensitive, full training has the lowest mean dev loss at every evaluated fee, while limited training improves on full training at most matched acquisition budgets.

Figures & tables

Figure 1: All-source training. Mean dev decision loss; bands show one sample standard deviation over four seeds. Sensitive shows

\lambda\leq 2

on a

\log(1+\lambda/0.04)

scale (full sweep: Appendix Table 4 ). Lower is better.

Figure 2: Equal paid source observations. Mean dev decision loss over eight seeds; bands show one sample standard deviation. FEVER uses

\lambda=2

and Sensitive uses

\lambda=0.12

. Lower is better.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Meaning
$X,Y,A=(A^{1},\ldots,A^{K})$	Input, label, and advice sources.
$J,K,[K]_{0},\Pi$	Numbers of experts and paid sources; $[K]_{0}=\{0,\ldots,K\}$ includes no advice; $\Pi=[J]\times[K]_{0}$ .
$e_{j},\widetilde{A}^{(k)},\psi$	Expert $j$ , advice vector revealing only source $k$ , and task loss.
$B,C_{0},C$	Bounds on task loss, no-advice total cost, and all pair costs, respectively.
$\beta_{j},\gamma_{k}$	Expert and source fees.
$D_{j,k},c_{j,k},\bar{c}_{j,k}(x)$	Task loss, total cost, and conditional expected total cost of a pair; subscript $i$ abbreviates $(j,k)$ .

Appendix

Table 1: Notation shared across the paper. A star marks a population target or an optimal value; a hat marks a fitted estimate or decoded decision.

Panel of Figure 3 — Figure 3: The deferral-advice protocol for the proposed decoder. The model uses only $x$ to estimate every pair’s marginal task cost. It adds the known fees, selects one pair, and only then reveals the selected source to the selected expert. The no-advice choice $\widehat{k}=0$ reveals $\widetilde{a}^{(0)}$ .

Figure 4: Advice effects depend on the expert. Dev task-error reduction relative to no advice, in percentage points, for each fixed expert–source pair. Fees are excluded. Positive values indicate that the source reduces the expert’s error.

Task	Method	Learning rate	Weight decay	$\alpha$	Reference
Sensitive	RC full	$10^{-3}$	$0$	$1$	$(1,0)$
Sensitive	RC limited	$10^{-3}$	$0$	$1$	$(1,0)$
FEVER	RC full	$10^{-3}$	$10^{-4}$	$1/4$	$(2,0)$
FEVER	RC limited	$10^{-3}$	$0$	$1$	$(1,0)$

Appendix

Table 2: Reference conditional configurations selected during development and retained for every reported fee and acquisition budget.

Fee	Reference conditional full	Augmented surrogate	Learning-to-Defer
0	0.555 $\pm$ 0.004	0.563 $\pm$ 0.002	0.627 $\pm$ 0.002
0.5	0.560 $\pm$ 0.003	0.565 $\pm$ 0.006	0.627 $\pm$ 0.002
1	0.566 $\pm$ 0.006	0.572 $\pm$ 0.002	0.627 $\pm$ 0.002
2	0.576 $\pm$ 0.005	0.584 $\pm$ 0.004	0.627 $\pm$ 0.002
4	0.592 $\pm$ 0.003	0.600 $\pm$ 0.003	0.627 $\pm$ 0.002
6	0.603 $\pm$ 0.004	0.614 $\pm$ 0.006	0.627 $\pm$ 0.002

Appendix

Table 3: All-source dev results on FEVER (mean

\pm

sample standard deviation over seeds 101–104). Each method uses its stated dev checkpoint rule. Lower is better.

Fee	Reference conditional full	Augmented surrogate	Learning-to-Defer
0	0.244 $\pm$ 0.002	0.248 $\pm$ 0.001	0.273 $\pm$ 0.001
0.04	0.247 $\pm$ 0.002	0.252 $\pm$ 0.001	0.273 $\pm$ 0.001
0.12	0.253 $\pm$ 0.001	0.258 $\pm$ 0.002	0.273 $\pm$ 0.001
0.2	0.256 $\pm$ 0.000	0.262 $\pm$ 0.001	0.273 $\pm$ 0.001
1	0.266 $\pm$ 0.001	0.277 $\pm$ 0.001	0.273 $\pm$ 0.001
2	0.266 $\pm$ 0.001	0.273 $\pm$ 0.002	0.273 $\pm$ 0.001

Appendix

Table 4: All-source dev results on Sensitive (mean

\pm

sample standard deviation over seeds 101–104). Each method uses its stated dev checkpoint rule. Lower is better.

Fee	Decision loss	Task loss	Expert cost	Advice cost	Advice rate
0.2	0.2561	0.2127	0.0329	0.0105	36.17%
1	0.2660	0.2273	0.0299	0.0088	7.81%
2	0.2659	0.2366	0.0292	0.0001	0.03%
5	0.2659	0.2364	0.0295	0.0000	0.00%

Appendix

Table 5: Cost decomposition for reference conditional full on Sensitive dev (means over seeds 101–104). Decision loss is the sum of task loss, expert cost, and advice cost. Advice rate is the fraction of inputs using a paid source.

Figure 5: Advice use across fees. Mean fraction of evaluated examples on which each method requests a paid source; bands show one sample standard deviation over seeds 101–104. Both tasks use dev data. Learning-to-Defer has no advice action.

Resource	Reference conditional full	Reference conditional limited
Training labels used	$Q/4$	$\min\{Q,n\}$
Paid source requests	$Q$	$Q$
Expert outcomes under paid sources	$3Q$	$3Q$
No-advice expert outcomes	$3Q/4$	$3\min\{Q,n\}$
Dev source requests	$4n_{\rm dev}$	$4n_{\rm dev}$
Optimizer updates	$U(Q)$	$U(Q)$

Appendix

Table 6: Resources used at acquisition budget

Q

with three experts and four paid sources. The dev table has

n_{\rm dev}=3{,}750

records on FEVER and

4{,}000

on Sensitive and is outside

Q

Budget $Q$	RC full	RC limited	Augmented	Limited $-$ full	Limited lower
1404	0.685 $\pm$ 0.013	0.674 $\pm$ 0.024	0.651 $\pm$ 0.007	-0.0116	6/8
2812	0.668 $\pm$ 0.021	0.656 $\pm$ 0.011	0.644 $\pm$ 0.005	-0.0125	6/8
5624	0.643 $\pm$ 0.004	0.623 $\pm$ 0.008	0.639 $\pm$ 0.006	-0.0201	8/8
11248	0.627 $\pm$ 0.006	0.601 $\pm$ 0.007	0.624 $\pm$ 0.006	-0.0267	8/8
16872	0.607 $\pm$ 0.007	0.589 $\pm$ 0.006	0.614 $\pm$ 0.006	-0.0181	8/8
22500	0.597 $\pm$ 0.005	0.579 $\pm$ 0.005	0.606 $\pm$ 0.004	-0.0180	8/8

Appendix

Table 7: Reference conditional full, reference conditional limited and augmented training on FEVER (dev) at

\lambda=2

with the same number

Q

of paid source observations and optimizer updates (mean

\pm

sample standard deviation over seeds 101–108). Lower is better.

Budget $Q$	RC full	RC limited	Augmented	Limited $-$ full	Limited lower
1000	0.277 $\pm$ 0.007	0.273 $\pm$ 0.004	0.285 $\pm$ 0.005	-0.0037	6/8
2000	0.272 $\pm$ 0.005	0.269 $\pm$ 0.003	0.279 $\pm$ 0.003	-0.0029	6/8
4000	0.266 $\pm$ 0.002	0.264 $\pm$ 0.002	0.274 $\pm$ 0.002	-0.0022	6/8
8000	0.263 $\pm$ 0.004	0.256 $\pm$ 0.002	0.270 $\pm$ 0.003	-0.0065	8/8
12000	0.259 $\pm$ 0.003	0.254 $\pm$ 0.002	0.266 $\pm$ 0.002	-0.0043	8/8
16000	0.257 $\pm$ 0.002	0.254 $\pm$ 0.001	0.264 $\pm$ 0.002	-0.0031	7/8

Appendix

Table 8: Reference conditional full, reference conditional limited and augmented training on Sensitive (dev) at

\lambda=0.12

with the same number

Q

of paid source observations and optimizer updates (mean

\pm

sample standard deviation over seeds 101–108). Lower is better.

Figure 6: Paired difference at equal acquisition budgets. Mean within-seed decision-loss difference between reference conditional limited and reference conditional full; bands show one sample standard deviation over seeds 101–108 on the FEVER and Sensitive dev sets. Values below zero favor the limited method.

Figure 7: Reference conditioning reduces acquisition variance. Markers show measured

n\widehat{\operatorname{Var}}

n=8192

; error bars show one Monte Carlo standard error over 50,000 datasets. Lines show the exact augmented variance and the asymptotic variances of the acquired-only and RC estimates. Conditioning reduces variance beyond acquisition-count normalization when the reference is informative (left). The acquired-only and RC curves nearly coincide in the control (right). All three estimates coincide at

\rho=1

Informative reference: $(a,b)=(0.1,0.9)$
$n$	$1/\rho$	Augmented HT	Acquired-only mean	RC
512	1	$0.249\pm 0.002$	$0.249\pm 0.002$	$0.249\pm 0.002$
512	2	$0.850\pm 0.005$	$0.498\pm 0.003$	$0.341\pm 0.002$
512	4	$2.087\pm 0.013$	$1.004\pm 0.006$	$0.521\pm 0.003$
512	8	$4.503\pm 0.029$	$2.010\pm 0.013$	$0.897\pm 0.006$
512	16	$9.425\pm 0.061$	$4.116\pm 0.026$	$1.688\pm 0.011$

Appendix

Table 9: Scaled estimation variance

n\widehat{\operatorname{Var}}

over 50,000 independent datasets per setting, with one Monte Carlo standard error after

\pm

. All prespecified sample sizes and acquisition probabilities are included.

Learning to Defer

Learning to defer trains AI systems to decide when to predict autonomously and when to route cases to human or machine experts.

See all 26 papers

Online Learning-to-Defer with Varying Experts
May 12, 2026Dang Hoang Duy, Yannis Montreuil, Maxime Meyer +3Learning to Defer Bandit Learning
School of Computing, National University of Singapore, Singapore · IPAL, IRL 2955, Singapore · Department of Mathematics, National University of Singapore, Singapore, 117543 +2
A Query Is Not a Commitment: Learning to Correct Expert Answers in Online Deferral
Oct 5, 2026Yannis Montreuil, Axel Carlier, Lai Xing Ng +1Cost-Sensitive Learning Learning to Defer
School of Computing, National University of Singapore · IRIT, Toulouse INP, France · Institute for Infocomm Research, A*STAR, Singapore
Learning-to-Defer in Non-Stationary Time Series via Switching State-Space Models
Jan 30, 2026Yannis Montreuil, Letian Yu, Axel Carlier +2Cost-Sensitive Learning Adaptive Model Routing
School of Computing National University of Singapore · Université de Toulouse Fédération ENAC ISAE-SUPAERO ONERA · Institute for Infocomm Research A*STAR, Singapore

May 12, 2026J/K move · Enter open · S save

Online Learning-to-Defer with Varying Experts

Dang Hoang Duy, Yannis Montreuil, Maxime Meyer, Axel Carlier+2

School of Computing, National University of Singapore, Singapore · IPAL, IRL 2955, Singapore · Department of Mathematics, National University of Singapore, Singapore, 117543+2

Learning to Defer Bandit Learning Multiclass Classification Online Learning

Open paper AskPDF arXiv

Learning-to-Defer (L2D) methods route each query either to a predictive model or to external experts. While existing work studies this problem in batch settings, real-world deployments require handling streaming data, changing expert availability, and shifting expert distribution. We introduce the first online L2D algorithm for multiclass classification with bandit feedback and a dynamically varying pool of experts. Our method achieves regret guarantees of

O((n+n_e)T^{2/3})

in general and

O((n+n_e)\sqrt{T})

under a low-noise condition, where

T

is the time horizon,

n

is the number of labels, and

n_e

is the number of distinct experts observed across rounds. The analysis builds on novel

\mathcal{H}

-consistency bounds for the online framework, combined with first-order methods for online convex optimization. Experiments on synthetic and real-world datasets demonstrate that our approach effectively extends standard Learning-to-Defer to settings with varying expert availability and reliability.

Abstract

Learning to Defer

Explore similar work

Online Learning-to-Defer with Varying Experts

A Query Is Not a Commitment: Learning to Correct Expert Answers in Online Deferral

Learning-to-Defer in Non-Stationary Time Series via Switching State-Space Models