Learning to defer (L2D) enables human-AI cooperation by deciding when an AI system should act autonomously or defer to a human expert. Existing L2D methods, however, assume static human performance, contradicting well-established findings on fatigue-induced degradation. We propose Fatigue-Aware Learning to Defer via Constrained Optimisation (FALCON), which explicitly models workload-varying human performance using psychologically grounded fatigue curves. FALCON formulates L2D as a Constrained Markov Decision Process (CMDP) whose state includes both task features and cumulative human workload, and optimises accuracy under human-AI cooperation budgets via PPO-Lagrangian training. We further introduce FA-L2D, a benchmark that systematically varies fatigue dynamics from near-static to rapidly degrading regimes. Experiments across multiple datasets show that FALCON consistently outperforms state-of-the-art L2D methods across coverage levels, generalises zero-shot to unseen experts with different fatigue patterns, and demonstrates the advantage of adaptive human-AI collaboration over AI-only or human-only decision-making when coverage lies strictly between 0 and 1.
Figures & tables
Figure 1: Example of an L2D scenario illustrating workload-variant human performance in human–AI task allocation within a single episode. FALCON adapts deferral decisions based on both task difficulty and accumulated human fatigue. At t=1 , an easy task is handled by the AI while the human expert remains fresh. At t=2 , a challenging case is deferred to the human expert who has sufficient cognitive capacity. By t=3 , another hard task is still assigned to the human despite mild fatigue accumulation. At the final time step t=T , severe human fatigue leads to AI handling the task to prevent performance degradation.
Figure 2: (a): Examples of w(ρ) . The values of parameters (w0,wpeak,wbase,k,ρˉ,ρ^) in Example 1,2 and 3 are (0.9,1,0.7,0.1,0.375,0.05) , (0.8,0.95,0.5,0.09,0.5,0.025) and (0.8,0.9,0.6,0.2,0.6,0.1) . (b): The architecture of FALCON with workload-variant human performance. A backbone model extracts visual features from the input xt , while the cumulative human workload ρt is passed through an embedding layer. The visual and workload features are concatenated and processed by a Resettable S5 layers ( Lu et al., 2023 ) to capture temporal dependencies and output the policy π(at∣st) alongside value estimates.
Algorithm 1 FALCON training procedure
Algorithm 2 FALCON testing procedure
Params
Value
Activation
Relu
Clipping_Coefficient ϵ
0.2
Entropy_Coefficient
0.001
Lagrangian_LR
0.035
Lagrangian_INIT λ
0.001
GAE_LAMBDA
0.95
Table 1: PPO parameters
Figure 3: Human performance-Cumulative Workload curves on various datasets. The blue and red lines denote the upper and lower bound of human performance under cumulative workload accumulation.
Params
Range
Description
w0
U(0.7,0.9)
initial performance
wbase
U(0.4,0.5)
minimum performance
wpeak
U(0.8,1.0)
maximum performance
ρ^
U(0.025,0.1)
relative workload at the peak performance
ρˉ
U(0.25,0.5)
relative workload at the inflection point of the decay phase
k
U(0.05,0.1)
steepness of performance decline
Table 2: The range of parameters of human performance variation in Eq. 2 on Cifar100 dataset.
Params
Range
Description
w0
U(0.8,0.9)
initial performance
wbase
U(0.6,0.7)
minimum performance
wpeak
U(0.9,1.0)
maximum performance
ρ^
U(0.025,0.1)
relative workload at the peak performance
ρˉ
U(0.25,0.5)
relative workload at the inflection point of the decay phase
k
U(0.05,0.1)
steepness of performance decline
Table 3: The range of parameters of human performance variation in Eq. 2 on Chaoyang dataset.
Params
Range
Description
w0
U(0.65,0.9)
initial performance
wbase
U(0.3,0.4)
minimum performance
wpeak
U(0.8,1.0)
maximum performance
ρ^
U(0.025,0.1)
relative workload at the peak performance
ρˉ
U(0.25,0.5)
relative workload at the inflection point of the decay phase
k
U(0.05,0.1)
steepness of performance decline
Table 4: The range of parameters of human performance variation in Eq. 2 on FLickr10K dataset.
Params
Range
Description
w0
U(0.8,0.9)
initial performance
wbase
U(0.6,0.7)
minimum performance
wpeak
U(0.9,1.0)
maximum performance
ρ^
U(0.025,0.1)
relative workload at the peak performance
ρˉ
U(0.25,0.5)
relative workload at the inflection point of the decay phase
k
U(0.05,0.1)
steepness of performance decline
Table 5: The range of parameters of human performance variation in Eq. 2 on Micebone dataset.
Figure 4: Training time of FALCON and competing methods on Cifar100 (1e7 iterations).
Figure 5: Inference time of FALCON and competing methods on Cifar100 (50 episodes).
Figure 6: Accuracy-Coverage curves of several L2D strategies and FALCON on various datasets.
Cifar100
Chaoyang
FLickr10K
MiceBone
OneStage L2D
70.87 ± 0.13
83.24 ± 0.14
63.06 ± 0.12
84.61 ± 0.12
TwoStage L2D
70.50 ± 0.15
83.15 ± 0.08
61.77 ± 0.13
84.58 ± 0.15
L2D-Pop
71.01 ± 0.11
82.65 ± 0.17
62.72 ± 0.18
83.96 ± 0.14
EA-L2D
66.26 ± 0.39
82.39 ± 0.08
63.26 ± 0.23
84.59 ± 0.12
Ours
74.01 ± 0.09
84.13 ± 0.11
64.40 ± 0.08
86.08 ± 0.13
Table 6: Quantitative comparison in terms of AUACC (×100) ( Nadeem et al., 2009 ) of the SOTA L2D ( Mozannar and Sontag, 2020a ; Madras et al., 2018 ; Tailor et al., 2024 ; Strong et al., 2025 ) on the L2D datasets. The results consist of the mean and standard deviations obtained from three experiments using models trained with different random seeds. The best result per benchmark is marked in bold.
Figure 7: Different human performance during testing (left column) and corresponding results with fine-tuning (middle column) and zero-shot (right column) testing on Cifar100 across all methods.
Sustained High Performance
Normal Fatigue
Rapid Fatigue
Fine-tuning
Zero-shot
Fine-tuning
Zero-shot
Fine-tuning
Zero-shot
OneStage L2D
78.23 ± 0.17
77.25 ± 0.13
72.67 ± 0.10
70.83 ± 0.16
66.75 ± 0.09
63.85 ± 0.11
TwoStage L2D
78.78 ± 0.14
75.22 ± 0.16
73.10 ± 0.09
71.56 ± 0.07
67.36 ± 0.14
67.49 ± 0.13
L2D-Pop
77.63 ± 0.08
76.38 ± 0.12
72.07 ± 0.11
70.35 ± 0.13
64.82 ± 0.12
64.40 ± 0.08
EA-L2D
73.46 ± 0.10
72.23 ± 0.14
67.87 ± 0.15
66.51 ± 0.12
63.97 ± 0.17
62.31 ± 0.14
Ours
79.58 ± 0.10
79.70 ± 0.12
76.93 ± 0.07
76.20 ± 0.09
72.36 ± 0.15
71.68 ± 0.07
Table 7: Quantitative comparison in terms of the Area Under Accuracy-Coverage Curve (AUACC) (×100) ( Nadeem et al., 2009 ) of the SOTA L2D ( Mozannar and Sontag, 2020a ; Madras et al., 2018 ; Tailor et al., 2024 ; Strong et al., 2025 ) with three different human performance curves on the Cifar100 dataset. The results consist of the mean value obtained from three experiments using models trained with different random seeds. The best result per benchmark is marked in bold.
Figure 8: Accuracy-Coverage curves of CMDP ablation on Cifar100 dataset.
Figure 9: Validation against clinical human performance data. (a) Comparison between simulated human performance in FALCON and real-world human data ( Cowley and Gale, 1997 ) . The orange line represents the radiologist’s correct recall rate, which shows a significant decline from 78% to 66% after the vigilance threshold. The green shaded area indicates the range of our simulated experts on the Chaoyang dataset, demonstrating high ecological validity. (b) Zero-shot testing results on the Chaoyang dataset.
Modern AI systems increasingly combine large language models (LLMs) with human experts, using the LLM to handle routine queries while deferring uncertain cases for expert review. When expert capacity is limited, however, deferral decisions must account not only for the reliability of the LLM but also for congestion in the expert queue. We formulate this as an online sequential decision-making problem in which, for each query, the system observes the confidence signal of the LLM's answer and must decide whether to output the answer or defer the query to a capacity-constrained expert. The reliability of the confidence signal is initially unknown and must be learned from expert feedback, which is both selectively observed (only deferred queries are labeled) and delayed by queuing. We propose KL-PBD, a learning algorithm that uses Kullback--Leibler lower confidence bounds on the unknown reliability parameters to guide deferral decisions. For K confidence levels and horizon T, KL-PBD achieves O(KlogT) gap-dependent regret when the accuracy at every confidence level is bounded away from the deferral boundary, and O(KT) worst-case regret. We establish matching lower bounds, showing that these rates are optimal up to logarithmic factors. Finally, we validate our approach on synthetic instances and real-world question-answering datasets, namely GSM8K, NuminaMath, and MedMCQA, using Qwen LLMs.
Vasileios Saketos, William Reveillard, Alexandre Proutiere +2
KTH Royal Institute of Technology, Stockholm, Sweden
Investigating the impact of fatigue on human physiological function and motor behavior is crucial for developing biomechanics and medical applications aimed at mitigating fatigue, reducing injury risk, and creating sophisticated ergonomic designs, as well as for producing physically-plausible 3D animation sequences. While the former has a prominent position in state-of-the-art literature, fatigue-driven motion generation is still an underexplored area. In this study, we present FatigueFusion, a deep-learning architecture for the fusion of fatigue features within a latent representation space, enabling the creation of a variation of novel fatigued movements, intermediate fatigued states, and progressively fatigued motions. Unlike existing approaches that focus on imitating the effects of fatigue accumulation in motion patterns, our framework incorporates algorithmic and data-driven modules to impose subject-specific temporal and spatial fatigue features on nonfatigued motions, while leveraging PINN-based techniques to simulate fatigue intensity. Since all motion modulation tasks are taking place in latent space, FatigueFusion offers an end-to-end architecture that operates directly on non-fatigued joint angle sequences and control parameters, allowing seamless integration into any motion synthesis pipeline, without relying on fatigue input data. Overall, our framework can be employed for various fatigue-driven synthesis tasks, such as fatigue profile transfer and fusion, while it also provides a solution for accurate rendering of the human fatigue state in both animation and simulation pipelines.
Iliana Loi, Konstantinos Moustakas
Department of Electrical and Computer Engineering, University of Patras, Patras, 26504 Greece
A learning-to-defer (L2D) system decides, for each input, whether to predict on its own or to hand it to one of several available experts. The very well established recipe trains classifier and router jointly by treating the K classes and J experts as competing actions in one shared (K+J)-action geometry. Subsequent work has proposed a series of incremental fixes within this geometry; we show that each still suffers, to varying severity, from an optimization-level pathology (target distortion, gradient amplification, winner-take-all starvation, set-mass collapse, or class-expert coupling) even under statistical consistency. We step outside the augmented-action family entirely and propose a decoupled surrogate: a softmax classifier head and an independent sigmoid head per expert, mirroring the two natural objects of the problem. We show that per-sample updates are then coordinatewise and the class-expert Hessian block is identically zero, and prove an excess-risk bound with calibration constant max{22,2J/λ} -- to our knowledge the first multi-expert L2D guarantee whose constant does not grow with the expert pool when the per-expert weight is held fixed. On controlled synthetic studies and on CIFAR-10, CIFAR-10H, and Covertype, it is the only method in our comparison that remains stable as the expert pool grows, preserves rare specialists, and improves over a standalone classifier on every real-data benchmark.
Yannis Montreuil, Axel Carlier, Lai Xing Ng +1
School of Computing, National University of Singapore, Singapore, 117418 · Fédération ENAC ISAE-SUPAERO ONERA, Université de Toulouse, Toulouse, 31555, France · Agency for Science, Technology and Research, Institute for Infocomm Research, Singapore, 138634