cs.AIOct 18, 2026
SaveTowards Human-AI Complementarity for AI Safety
Organizations: Sampura Research · University of California, Berkeley · University of Illinois Urbana-Champaign · Queen Mary University of London ·
Abstract
Safely overseeing AI is increasingly challenging for the humans doing it, as they must evaluate longer and more complex outputs with every new model generation. To mitigate this challenge, we could leverage complementarity: where humans and models make different mistakes, combining their judgements could yield an oversight signal stronger than either produces alone. We study this opportunity in realistic, safety-relevant settings, contributing the largest known suite of such tasks (3,957 instances across 8 datasets and 6 domains) as well as a platform and data pipeline for measuring human, AI, and human-AI performance on matched items. We collected 35,613 judgements from 416 human raters, finding a considerable opportunity for a complementarity effect. We also found that with the assistance methods we tested, human raters corrected the AI's errors less often, even when overall human accuracy rose, and the hybridisation methods we tested added little over the better single source on average. While complementarity is an important and feasible engineering target for safe oversight, achieving it remains an open challenge for future work.
Figures & tables
Figure 1: Overview of our approach . While we find a significant opportunity for Human-AI complementarity, the methods we test do not consistently realise this opportunity.
| Dataset | Domain | Interaction | Split | Items | Ratings/item |
| CulturalBench | Factual | Single-turn | held out | 1227 | 3.00 |
| Attune | Alignment / values | Multi-turn | 50/50 | 800 | 3.00 |
| SafeAgent | Unsafe agent action | Agentic | 20/80 | 640 | 3.00 |
| Liar’s Bench | Deceptive alignment | Single/multi-turn | 20/80 | 464 | 3.00 |
| BBEH-Safety | Reasoning | Single-turn | 20/80 | 300 | 3.00 |
| FACTS | Factual | Single-turn | 20/80 | 300 | 3.00 |
Table 1: Evaluation datasets. Items are defined as questions with human ratings. Ratings/item is the minimum number of ratings per item across the three assistance conditions. Split gives the training/test percentages; CulturalBench and SHADE-Arena are held out entirely.
Figure 2: Human rating interface. (a) The question, answer field, and confidence control. (b) Top- provides candidate answers with rationales. (c) ISD presents subtasks with model confidence and editable pre-filled answers; raters report their own confidence. The ISD screenshot shows the first two of three subtasks. See Section 4.1 for the protocols.
Figure 3: Complementarity leaderboard (test split, ). Accuracy (%) with 95% question-level binomial intervals. For each dataset and model, we show assisted and unassisted humans, the best of the hybrid for each human arm, and the higher arm-specific oracle. Intervals do not adjust for test-set selection; All is question-weighted. The full per-policy leaderboard, with paired differences against each column’s comparator, is in Table 4 .
Figure 4: Joint human and model outcomes. Left: Top- paired with Sonnet and ISD with Gemini, each with its unassisted baseline. Right: unassisted, by domain. The human-right/model-wrong component is the opportunity : items on which selecting the human answer would correct a model error. An oracle succeeds on every component except “both wrong.” Domain differences describe this evaluation suite and do not isolate the causes of human or model performance.
Figure 5: Human accuracy by model correctness. Committee accuracy on matched questions, pooling training and test. Left: Top- versus benchmarked Sonnet; right: ISD versus benchmarked Gemini. Error bars show 95% question-bootstrap intervals. These comparators are standalone model answers, not session-specific advice.
Figure 6: Hybridisation with API-answerer confidence signals. Left: gains of the best test-performing routing policy over the better single source across 260 configurations, and the oracle gap available against the gain realised for each model–assistance pairing. Middle and right: calibration error (ECE) and discrimination (AUROC) of each confidence signal under each rescaling. Calibration uses 1,936 matched questions per model, excluding CulturalBench; unavailable combinations are omitted.
Figure 7: Appropriate reliance before and after hybridisation. Change in individual rater accuracy relative to unassisted raters, on items grouped by whether the benchmarked model’s verdict was right or wrong. Grey: the rater’s own answer. Teal: the hybrid answer under the fit-free compare-confidence rule, which keeps the model’s answer unless the rater’s Platt-scaled confidence exceeds the model’s. ISD is scored against Gemini and Top- against Sonnet on the 1,936 test items with a calibrated confidence for both sides; means over 7 datasets with 95% bootstrap intervals resampling items within dataset.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Qwen | Human only | SWAG-LoRA | Linear probe | Confidence LoRA | Oracle-trained probe (question) | Oracle | |
| Attune | 412 | 41.0 [36.4, 45.6] | 46.1 [41.3, 51.0] | 42.7 [37.9, 47.6] | 48.3 [43.4, 53.2] | 41.0 [36.4, 45.6] | 45.1 [40.3, 50.0] | 62.1 [57.3, 66.7] |
| BBEH-Safety | 238 | 40.3 [34.5, 46.6] | 39.9 [34.0, 46.2] | 40.3 † [34.5, 46.6] | 40.3 [34.5, 46.6] | 41.6 [35.3, 47.9] | 44.1 [37.8, 50.4] | 56.7 [50.4, 63.0] |
| CulturalBench | 1,212 | 61.5 [58.7, 64.3] | 71.2 [68.6, 73.8] | 65.3 [62.5, 68.0] | 61.5 † [58.7, 64.3] | 71.4 [68.8, 73.9] | 71.2 [68.6, 73.8] | 81.4 [79.1, 83.5] |
| FACTS | 240 | 7.9 [4.6, 11.2] | 80.4 [75.4, 85.4] | 7.9 † [4.6, 11.2] | 80.4 [75.4, 85.4] | 75.8 [70.4, 81.2] | 80.4 [75.4, 85.4] | 81.7 [76.7, 86.7] |
| Liar’s Bench | 371 | 36.9 [32.1, 42.0] | 46.9 [41.8, 52.0] | 44.7 [39.6, 49.9] | 47.7 [42.6, 52.8] | 46.9 [41.8, 52.0] | 47.7 [42.6, 52.8] | 53.9 [48.8, 59.0] |
| Diff-Aware | 58 | 48.3 [36.2, 60.3] | 37.9 [25.9, 50.0] | 48.3 [36.2, 60.3] | 46.6 [34.5, 58.6] | 39.7 [27.6, 51.7] | 44.8 [32.8, 56.9] | 65.5 [53.4, 77.6] |
Table 2: Trained routing on Qwen3.5-9B. Accuracy (%) with 95% question-level percentile bootstrap intervals (10,000 replicates, seed 42). counts matched test questions. Human answers are unassisted committee plurality votes. The oracle-trained probe uses question-only hidden states; the other trained methods supply confidence estimates to compare-confidence routing. Bold marks routing accuracies strictly above both Qwen and Human only; it does not indicate statistical significance. The oracle succeeds whenever either source is correct and is not bolded. Missing method scores fall back to Qwen. Daggers mark cells consisting entirely of this fallback. Intervals describe each accuracy separately, not paired differences between methods.
Figure 8: Content-based LLM routing compared with AI-only and human-only performance. Bars give accuracy on held-out test items for each dataset under four policies: AI alone, unassisted human committees, and the zero-shot and few-shot content-based routers. Error bars are S.E.M. The dashed line marks the oracle ceiling, which counts an item as correct whenever either source answers it correctly.
| Dataset | Zero-shot vs AI | Zero-shot vs Human | Few-shot vs AI | Few-shot vs Human | |
| Attune | 411 † | +4.1 (0.150) | -1.2 (0.252) | +4.7 (0.117) | -0.2 (0.739) |
| BBEH-Safety | 238 | -1.3 (0.656) | +6.3 (0.022)* | +0.0 (1.000) | +7.6 (0.003)* |
| CulturalBench | 1212 | -6.9 ( 0.001)* | +0.6 (0.127) | -6.8 ( 0.001)* | +0.7 (0.012)* |
| FACTS | 240 | -4.2 (0.123) | +0.4 (0.318) | -4.2 (0.123) | +0.4 (0.318) |
| Liar’s Bench | 371 | +4.6 (0.002)* | -7.0 ( 0.001)* | +5.4 ( 0.001)* | -6.2 (0.002)* |
| Diff-Aware | 58 | -6.9 (0.044)* | +24.1 (0.012)* | -10.3 (0.109) | +20.7 (0.009)* |
Table 3: Per-dataset comparison of content-based routing against AI and unassisted humans. Entries are percentage-point differences (routing policy minus comparator) with two-tailed p-values in parentheses; marks p 0.05, uncorrected for multiplicity. Positive values favour the router. counts router-covered items. † Attune has 411 covered items for zero-shot and 404 for few-shot, excluding one and eight unparseable outputs, respectively.
Figure 9: Left: distribution of the best-test-policy gain over the better single source across all 260 routing configurations. Right: oracle gap available against the mean gain realised, by model–assistance pairing. Routing gains remain small relative to the oracle gap.
Figure 10: Calibration and discrimination on 1,936 matched test questions per model, excluding CulturalBench, which lacks calibrated scores. Unavailable combinations are omitted.
Figure 11: Error structure across answerers. Left: correctness correlation ( ) between Sonnet, Gemini, and unassisted human committees on test items. Right: oracle accuracy for each model paired with the other model or with the human committee. Human answers aggregate three ratings; model answers aggregate five samples.
Figure 12: Opportunity under assistance, comparing Top- against Sonnet and ISD against Gemini. Opportunity is lower with assistance than without it in both pairings: assisted human committees correct fewer errors made by the corresponding model.
Figure 13: Changes in individual-rater accuracy relative to the unassisted condition, grouped by whether the corresponding benchmarked model answers correctly. ISD is compared with Gemini 3 Flash and Top- with Claude Sonnet 4.6. Both methods have negative estimated effects on assistant-wrong items and positive effects on assistant-correct items. Results pool all 3,957 training and test items and average eight datasets equally.
| Method | Assistant | assistant right | assistant wrong | Acc. right |
| ISD | Gemini 3 Flash | +17.5 [+15.7, +19.3] | -10.8 [-13.7, -8.0] | 82.4% |
| Top- | Claude Sonnet 4.6 | +10.7 [+8.8, +12.5] | -12.6 [-15.8, -9.3] | 73.6% |
Table 6: Reliance decomposition, each assistance method against the corresponding benchmarked model’s verdict. is assisted minus unassisted individual-rater accuracy in percentage points. The rightmost column is assisted accuracy on items the assistant got right. Results pool all 3,957 training and test items, averaging eight datasets equally; brackets give 95% within-dataset item-bootstrap intervals.
Figure 14: Changes in median completion time and human accuracy relative to the unassisted condition. Each point represents a dataset and assistance method. Time changes are percentages; accuracy changes are percentage points. Large increases in completion time do not consistently accompany accuracy gains.
Figure 15: Supplementary confidence analysis on Qwen3.5-9B using the earlier dataset collection. Model-confidence signals are evaluated by mean within-dataset AUROC for predicting model correctness. The repeated-sample oracle uses correctness labels from other samples of the same item and is unavailable at deployment. The human-difficulty head predicts human correctness, a separate target.
| Signal | Knowledge | Safety | Overall | % of ceiling |
| Reliability ceiling (oracle) | 0.895 | 0.848 | 0.860 | 100% |
| Confidence LoRA (ours) | 0.738 | 0.730 | 0.728 | 63% |
| Self-consistency ( ) | 0.746 | 0.616 | 0.698 | 55% |
| LoRA without reasoning trace | 0.605 | 0.707 | 0.588 | 25% |
| Human-difficulty head | 0.519 | 0.648 | 0.606 | 29% |
| Per-dataset base rate | 0.500 | 0.500 | 0.500 | 0% |
Table 7: Confidence signals on the Qwen3.5-9B answerer, fall-wave test items. Mean within-dataset AUROC over the five non-degenerate cells; chance . The ceiling is a leave-one-out oracle over the model’s other four samples, not a deployable signal.
Figure 16: Calibration performance of six model-confidence signals before and after Platt scaling. We report expected calibration error (ECE; left) and Brier score (right), where lower values indicate better probabilistic predictions. Before calibration, the MLP probe achieves the lowest ECE, while the linear probe achieves the lowest Brier score. After Platt scaling, computed confidence achieves the lowest ECE, whereas the linear probe achieves the lowest Brier score. Overall, the hidden-state probes are competitive with the output-based confidence signals, although their relative performance depends on the evaluation metric.