Human-AI Complementarity

Follow this topic Ask about this topic

Latest papers

16 papers3 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Oct 18, 2026

Towards Human-AI Complementarity for AI Safety

Safely overseeing AI is increasingly challenging for the humans doing it, as they must evaluate longer and more complex outputs with every new model generation. To mitigate this challenge, we could leverage complementarity: where humans and models make different mistakes, combining their judgements could yield an oversight signal stronger than either produces alone. We study this opportunity in realistic, safety-relevant settings, contributing the largest known suite of such tasks (3,957 instances across 8 datasets and 6 domains) as well as a platform and data pipeline for measuring human, AI, and human-AI performance on matched items. We collected 35,613 judgements from 416 human raters, finding a considerable opportunity for a complementarity effect. We also found that with the assistance methods we tested, human raters corrected the AI's errors less often, even when overall human accuracy rose, and the hybridisation methods we tested added little over the better single source on average. While complementarity is an important and feasible engineering target for safe oversight, achieving it remains an open challenge for future work.

Sander Lanbo Shi, Miranda Anna Christ, Eric Modesitt +20

Sampura Research · University of California, Berkeley · University of Illinois Urbana-Champaign +6

Human-AI Complementarity Human Oversight of AI AI Safety Evaluation Human-in-the-Loop Evaluation

Oct 16, 2026

Indirect Influences of Artificial Intelligence Advice on Complex Sepsis Treatment Decisions

Sepsis is a major public health concern, and many sepsis patients die due to ineffective hemodynamic management. Artificial intelligence could support hemodynamic decisions, particularly in complex cases where conflicting information and comorbidities drive uncertainty and practice variation. However, these systems' real-world value depends on not just accuracy, but how they influence the quality of clinician decisions. We first retrospectively evaluated an AI system's recommendations in 6,928 sepsis patients, finding that they improved upon the original providers' decisions overall but had significantly worse performance in complex cases. We then conducted a pre-registered experiment with 228 US critical care physicians to test their use of this imperfect advice in six complex treatment decisions, using one of three advice types designed to vary reasoning autonomy. Participants given simple model recommendations overcame poor advice, choosing optimal treatments in 56.3% of decisions compared to 31.8% if they had always followed the recommendations. They also made significantly better decisions than those receiving a less prescriptive advice type (42.9%). Physicians selectively adapted and improved weak advice, likely because it prompted them to reconsider their assessment. Our findings suggest that improving sepsis care decisions with AI may require accounting for these indirect influences resulting from human-AI collaborative reasoning.

Venkatesh Sivaraman, Joel Levin, Eric P. Mason +6

Department of Epidemiology and Biostatistics, University of California, San Francisco, San Francisco, California, United States · Rady School of Management, University of California, San Diego, La Jolla, California, United States · Human-Computer Interaction Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania, United States +3

AI-Assisted Decision Making Human-AI Decision Making Clinical Decision-Making Human-AI Complementarity

Oct 12, 2026

Better together? A performance theory of human-machine skill policies under generalization difficulty

Organizations pairing a human and a machine on a task decide, often by default, which executes each instance and what happens when both do. We account for what this attains under the squared rule, with performance distributed over instances to capture generalization difficulty. A gross stake lies between the skills' mean and best per instance. It splits at the best single skill into a systematic part, known from separate assessments and secured by commitment, and a residual, the expected value of perfect information on which skill each instance favors, known only from paired observation and split further by classes. Skills set the stake; workflows decide the take, via two channels. Routing directs work, weighting or selecting responses; with binary truth it is capped by the best skill per instance. Reading responses as joint evidence is not routing; what they jointly reveal caps it. Synergy lets skills interact before responding; bounded by no skill either way, its lift over independent responses must clear a break-even and can fall below one. As algorithms, arrangements form a ladder stating what each reads, holds and records, and a procedure whose checks keep earlier gains when class means are known and lose little on finite records given a margin. The theory extends to many skills, each worth what it adds where it beats all others held, and to competing skills, where at parity a single failure's allowed cost decides. In a public chest-radiograph study with 227 radiologists, they are near parity with the model and the stake is mostly residual; assisted reading, no better on average originally, returned less than averaging both responses; recalibrating the model alone beats any deployable arrangement of raw responses. Organizations can learn from their records what is at stake, which arrangement collects it and what evidence each step needs; pricing costs and errors is future work.

Riccardo Zanardelli

Human-AI Complementarity Human-AI Decision Making Human-AI Collaboration

Sep 15, 2026

Available but Unclaimed: An Empirical Study of Human-AI Synergy

People increasingly reason with large language models (LLMs), yet complementary capabilities do not guarantee outperforming both components. In a between-subjects study, participants (N=535) solved a 40-item battery of matrix reasoning, mental rotation, syllogisms, and letter-string analogies, unaided or with GPT-5.6-Luna, Claude Opus 4.8, Gemini 3.6 Flash, or Kimi K3. Each assisted trial required consultation with the model. Each model answered every item alone 100 times under matched elicitation. The assisted-unaided accuracy difference increased with item-level LLM competence. Deference varied across tasks and increased with competence within tasks. Post-advice confidence distinguished correct from incorrect answers less strongly than unaided confidence. In a reference comparison, about half the increase in LLM accuracy carried through to assisted accuracy. How much of that accuracy gain reached participants differed across the models. These findings motivate evaluating LLMs in interaction with humans and designing support for selective deference that preserves independent reasoning.

Robin Welsch, Michelle Rausch, Pascal Knierim +4

Aalto University, Espoo, Finland · University of Innsbruck, Innsbruck, Austria · Humboldt-Universität zu Berlin, Berlin, Germany +1

Human-AI Complementarity AI-Assisted Decision Making Human-AI Collaboration LLM Reasoning

Aug 19, 2026

CentaurBench: Benchmarking LLM Capabilities on Augmenting vs. Automating Real-World Work Tasks

Most LLM benchmarks rank models on their ability to automate work tasks. In practice, however, models are often used to assist other (human or LLM) agents. The question that drives model selection is therefore not only which model produces the best output, but which model most improves the work of another (weaker) agent. We introduce a unified framework that evaluates the capability of models to automate and augment another agent's performance. Across seven economically grounded real-world tasks, an assistant model writes assistance text for a standardized lower-capacity worker model, which produces the deliverable. In automation mode, the assistant produces the output directly. Outputs are scored through blind pairwise comparisons by an LLM judge panel with task-specific rubrics, replicated across ten runs. Rankings across the two regimes are only modestly correlated, and the automation winner loses augmentation on five of seven tasks. Assistance is not reliably positive. The unaided worker outranks every assisted condition on three tasks, and only one model's guidance beats no guidance on average. These results suggest that automation ability is an incomplete proxy for assistance quality, motivating benchmarks that evaluate models according to the roles they play in human-AI and multi-agent systems.

Pattaraphon Kenny Wongchamcharoen, Kris Gulati, Min Min Fong +1

Data Innovation and AI Lab (DIAL) at the Haas School of Business, UC Berkeley · UC Berkeley · UC Berkeley and NBER

LLM Agent Evaluation Human-AI Collaboration Human-AI Complementarity AI and Labor Multi-LLM Collaboration

Jul 7, 2026

Robust Human-AI Complementarity under Uncertainty

Machine learning models are often intended to augment rather than replace human decision makers, by providing information that is complementary to human judgement. Yet, in practice, human decision makers routinely fail to realize such complementary gains, even when models provide useful signal. In this work, we study how asymmetric information about the quality of information available to a human decision maker vs. an AI impacts the ability of a decision maker to extract complementary value from AI predictions. We show that a key factor is the error correlation structure between human and AI predictions. In particular, when the AI's prediction errors are negatively correlated with those of the human, the decision maker can construct robust strategies which guarantee improvements in expected utility. We empirically investigate whether these conditions for complementarity arise in practice, using real-world forecasting benchmarks.

Yewon Byun, Bryan Wilder

Machine Learning Department, Carnegie Mellon University.

Human-AI Complementarity Human-AI Decision Making Decision-Making under Uncertainty

Jul 2, 2026

Human Capital, Not Model Benchmarks, Predicts Hybrid Intelligence in Forecasting

Whether pairing people with AI helps or hurts is usually reported as a single average effect. Using a real-money prediction market (Polymarket) as an objective, externally resolved benchmark, this pilot shows that the value of human-AI collaboration depends on a specific, measurable form of human capital. Analyzed at the level of the individual forecaster, hybrid performance is trimodal: most people either deferred to the model (matching it) or used it to rubber-stamp a prior guess (performing worse than the model alone), while a minority engaged in genuine complementary reasoning and reached accuracy matching or even exceeding (i.e., lower error than) the market itself. Collaborative traits (perspective-taking, intellectual humility, and curiosity) rather than raw cognitive ability or model benchmarks, distinguished who reached that mode. The results are preliminary but statistically robust, and motivate a pre-registered replication now in preparation.

Vivienne Ming

The Human Trust; Possibility Science; UCL Global Business School for Health

Human-AI Complementarity Probabilistic Forecasting Human-AI Collaboration

Jun 3, 2026

Tree-Based Formalization of Multi-Agent Complementarity in Human-AI Interactions

Complementarity is the case in which a human--AI interaction (HAI) outperforms the best prediction benchmark available among its members. Although this idea is central in HAI research, formal work on complementarity remains limited. Existing frameworks do not model how agents' predictions compose into workflow-sensitive multi-agent protocols. We close this gap by introducing a tree-based formalization of complementarity in multi-agent HAI. An HAI protocol is represented by an ordered agent-role configuration together with a rooted planar binary tree whose leaves are decorated by prediction vectors. A local binary composition rule is evaluated recursively along the tree, yielding a tree-relative complementarity functional relative to a pointwise-min oracle benchmark. We prove four results. First, selector-based HAIs, including self- or AI-reliance, cannot achieve complementarity regardless of task, loss, or prediction quality. Second, in regression under squared loss, complementarity is equivalent to Euclidean distance minimization from the ground-truth vector; for

N=2

, the optimal linear-pooling weight has a closed form and a residual-correction interpretation. Third, under linear local composition, every protocol tree defines a barycentric coordinate chart on the simplex of leaf weights; Tamari-cover reparameterizations of protocol trees preserve complementarity, and for

N=4

, they satisfy the pentagon identity. Fourth, in binary classification, no internal local composition can achieve complementarity under endpoint-monotone losses, including standard Bregman and many finite Bernoulli

f

-divergence losses; an analogous obstruction holds for multiclass aggregation under cross-entropy. In summary, our framework shows that complementarity is attainable in multi-agent regression, but obstructed in classification under natural conditions on local aggregation and loss functions.

Andrea Ferrario

Institute of Biomedical Ethics and History of Medicine, University of Zurich, Zurich, Switzerland · SUPSI, Dalle Molle Institute for Artificial Intelligence (IDSIA), Lugano, Switzerland · ETH Zurich, Zurich, Switzerland

Human-AI Complementarity Ensemble Learning Human-AI Collaboration

May 28, 2026

Rationalize: Shared Semantic Reasoning for Human-AI Alignment

We introduce Rationalize, a role-pair framework for shared semantic reasoning between humans and AI models in data-driven sensemaking. Building on ideas in human-machine teaming and critical thinking, we conceptualize human-AI interaction as a series of complementary role pairs (Explorer-Guide, Investigator-Informant, Teacher-Student, Judge-Advocate) operating in a shared reasoning space. In this space, human analysts and AI models (such as LLMs) make purposes, questions, assumptions, evidence, inferences, and implications explicit, facilitating alignment not only at the output level but at the level of rationalization of intent and action by each side. We relate these role pairs to the bidirectional human-AI alignment framework, illustrating how "aligning AI to humans" and "aligning humans to AI" differ by role, and sketch a collaborative research agenda for alignment design and assessment using element-level and role-specific approaches.

Aritra Dasgupta, Naga Datha Saikiran Battula, Avina Nakarmi +3

New Jersey Institute of Technology Newark, USA

AI Alignment Human-AI Interaction Human-AI Collaboration Human-AI Complementarity

May 18, 2026

Generative AI and the Productivity Divide: Human-AI Complementarities in Education

Generative Artificial Intelligence (GenAI) is transforming how firms create, process, and apply knowledge, yet little is known about the heterogeneity of its productivity effects across users. We report results from a randomized controlled experiment in which participants-analogs of early-career knowledge workers-were assigned to self-study a technical domain using either traditional resources or large-language-model (LLM) assistance. On average, GenAI access significantly increased task performance, but the distribution of gains was highly uneven. Improvements were not predicted by GPA or prior knowledge, but by AI Interaction Competence (AIC) -- the ability to elicit, filter, and verify model outputs. High-AIC participants realized outsized gains; low-AIC participants saw limited or even negative marginal returns. A scaffolding intervention (conceptual maps) reduced outcome variance, indicating that standardized workflows can mitigate inequality in AI-mediated performance. We interpret these findings through the lens of human-AI complementarities: GenAI raises mean productivity while introducing a new axis of capability inequality. Managerially, firms should pair GenAI access with short AIC micro-training and simple standard operating procedures to capture value consistently and avoid uneven adoption outcomes.

Lihi Idan, Bharat Anand

Industrial and Systems Engineering Department, Texas A&M University · Leonard N. Stern School of Business, New York University

Human-AI Complementarity AI in Education AI Literacy Generative AI in Education Human-AI Collaboration

May 9, 2026

When Can Human-AI Teams Outperform Individuals? Tight Bounds with Impossibility Guarantees

Human-AI teams fail to outperform their best member in 70% of studies, yet no theory specifies when complementarity is achievable. We derive tight bounds for the broad class of confidence-based aggregation rules by integrating signal detection theory with information-theoretic analysis, yielding four results: (1) a complementarity theorem (teams outperform individuals iff error correlation

ρ_{HM} < ρ^*

, with

ρ^* \approx a

in the symmetric near-chance regime); (2) minimax bounds showing gains scale as

Θ(\sqrt{Δd})

with metacognitive sensitivity difference; (3) an impossibility result proving no confidence-based aggregation rule achieves complementarity when

ρ_{HM} \geq ρ^*

; and (4) multi-class generalization

ρ^*_K \approx ρ^*/\sqrt{K-1}

. Predictions match observed team accuracy (

R = 0.94

on ImageNet-16H,

R = 0.91

on CIFAR-10H) and the multi-class threshold scaling holds on human data (

R = 0.93

,

K = 16

), with robustness under non-Gaussian distributions. The framework explains why complementarity is rare and provides actionable design formulas; results apply to aggregation, not to interactive deliberation that generates novel answers.

Dongxin Guo, Jikun Wu, Siu-Ming Yiu

The University of Hong Kong, Hong Kong, China · Stellaris AI Limited, Hong Kong, China · Brain Investing Limited, Hong Kong, China

Human-AI Complementarity Ensemble Learning Human-AI Decision Making Human-AI Collaboration

Apr 13, 2026

Toward Human-AI Complementarity Across Diverse Tasks

Human-AI complementarity, the idea that combining human and AI judgments can outperform either alone, offers a promising pathway toward robust oversight of advanced AI systems. However, whether human-AI complementarity can be achieved on realistic tasks remains an open question. We investigate this through two approaches: hybridization and two AI assistance methods (top-2 assistance and subtask delegation), evaluated on a multi-domain dataset of 1,886 samples spanning knowledge, factuality, long-context reasoning, and deception detection. We find only modest complementarity gains. Baseline hybridization yields just +0.4 percentage points (pp) over AI alone (69.3% vs 68.9%), limited both by a small complementarity region (only 8.9% of items where AI errs but humans do not) and the inability of confidence-based routing to identify it, since the model's confidence is similarly distributed across correct and incorrect predictions. Applied when AI has low confidence, top-2 assistance increases human accuracy from 28.4% to 38.3%, surpassing AI alone (37.7%) -- but primarily because humans adopt correct AI suggestions, not because they successfully override AI errors. These findings suggest that the primary bottleneck is not human task accuracy per se, but the ability to route decisions to humans when it matters and to design assistance methods that enable humans to catch AI mistakes. Our quantitative and qualitative analyses pinpoint where and why each method succeeds or fails, offering concrete targets for future work. We will release our dataset and code upon request to support progress toward more effective human-AI collaboration for AI oversight.

Yuzheng Xu, Annya Dahmani, Matthew D. Blanchard +13

The University of Tokyo · UC Berkeley · University of Sydney +6

Human-AI Complementarity Human-AI Decision Making Human-AI Collaboration Human-in-the-Loop AI Human Oversight of AI

Mar 30, 2026

Self++: Co-Determined Agency for Human--AI Symbiosis in Extended Reality

Self++ is a design blueprint for human-AI symbiosis in extended reality (XR) that preserves human authorship while still benefiting from increasingly capable AI agents. Because XR can shape both perceptual evidence and action, apparently 'helpful' assistance can drift into over-reliance, covert persuasion, and blurred responsibility. Self++ grounds interaction in two complementary theories: Self-Determination Theory (autonomy, competence, relatedness) and the Free Energy Principle (predictive stability under uncertainty). It operationalises these foundations through co-determination, treating the human and the AI as a coupled system that must keep intent and limits legible, tune support over time, and preserve the user's right to endorse, contest, and override. These requirements are summarised as the co-determination principles (T.A.N.): Transparency, Adaptivity, and Negotiability. Self++ organises augmentation into three concurrently activatable overlays spanning sensorimotor competence support (Self: competence overlay), deliberative autonomy support (Self+: autonomy overlay), and social and long-horizon relatedness and purpose support (Self++: relatedness and purpose overlay). Across the overlays, it specifies nine role patterns (Tutor, Skill Builder, Coach; Choice Architect, Advisor, Agentic Worker; Contextual Interpreter, Social Facilitator, Purpose Amplifier) that can be implemented as interaction patterns, not personas. The contribution is a role-based map for designing and evaluating XR-AI systems that grow capability without replacing judgment, enabling symbiotic agency in work, learning, and social life and resilient human development.

Thammathip Piumsomboon

School of Product Design, University of Canterbury, Christchurch, New Zealand

Human-AI Collaboration Human-AI Complementarity Human-Centered AI

Feb 23, 2026

Align When They Want, Complement When They Need! Human-Centered Ensembles for Adaptive Human-AI Collaboration

In human-AI decision making, designing AI that complements human expertise has been a natural strategy to enhance human-AI collaboration, yet it often comes at the cost of decreased AI performance in areas of human strengths. This can inadvertently erode human trust and cause them to ignore AI advice precisely when it is most needed. Conversely, an aligned AI fosters trust yet risks reinforcing suboptimal human behavior and lowering human-AI team performance. In this paper, we start by identifying this fundamental tension between performance-boosting (i.e., complementarity) and trust-building (i.e., alignment) as an inherent limitation of the traditional approach for training a single AI model to assist human decision making. To overcome this, we introduce a novel human-centered adaptive AI ensemble that strategically toggles between two specialist AI models - the aligned model and the complementary model - based on contextual cues, using an elegantly simple yet provably near-optimal Rational Routing Shortcut mechanism. Comprehensive theoretical analyses elucidate why the adaptive AI ensemble is effective and when it yields maximum benefits. Moreover, experiments on both simulated and real-world data show that when humans are assisted by the adaptive AI ensemble in decision making, they can achieve significantly higher performance than when they are assisted by single AI models that are trained to either optimize for their independent performance or even the human-AI team performance.

Hasan Amin, Ming Yin, Rajiv Khanna

Purdue University, USA

Human-AI Complementarity Human-AI Collaboration Human-AI Decision Making Ensemble Learning

Feb 19, 2026

Multi-Round Human-AI Collaboration with User-Specified Requirements

As humans increasingly rely on multiround conversational AI for high stakes decisions, principled frameworks are needed to ensure such interactions reliably improve decision quality. We adopt a human centric view governed by two principles: counterfactual harm, ensuring the AI does not undermine human strengths, and complementarity, ensuring it adds value where the human is prone to err. We formalize these concepts via user defined rules, allowing users to specify exactly what harm and complementarity mean for their specific task. We then introduce an online, distribution free algorithm with finite sample guarantees that enforces the user-specified constraints over the collaboration dynamics. We evaluate our framework across two interactive settings: LLM simulated collaboration on a medical diagnostic task and a human crowdsourcing study on a pictorial reasoning task. We show that our online procedure maintains prescribed counterfactual harm and complementarity violation rates even under nonstationary interaction dynamics. Moreover, tightening or loosening these constraints produces predictable shifts in downstream human accuracy, confirming that the two principles serve as practical levers for steering multi-round collaboration toward better decision quality without the need to model or constrain human behavior.

Sima Noorani, Shayan Kiyani, Hamed Hassani +1

University of Pennsylvania

Human-AI Complementarity Human-AI Collaboration Human-AI Decision Making Online Learning

Oct 30, 2025

Human-AI Complementarity: A Goal for Amplified Oversight

Human feedback is critical for aligning AI systems to human values. As AI capabilities improve and AI is used to tackle more challenging tasks, verifying quality and safety becomes increasingly challenging. This paper explores how we can leverage AI to improve the quality of human oversight. We focus on an important safety problem that is already challenging for humans: fact-verification of AI outputs. We find that combining AI ratings and human ratings based on AI rater confidence is better than relying on either alone. Giving humans an AI fact-verification assistant further improves their accuracy, but the type of assistance matters. Displaying AI explanation, confidence, and labels leads to over-reliance, but just showing search results and evidence fosters more appropriate trust. These results have implications for Amplified Oversight -- the challenge of combining humans and AI to supervise AI systems even as they surpass human expert performance.

Rishub Jain, Sophie Bridgers, Lili Janzer +3

1Google DeepMind · 2Work done while previously at Google DeepMind

Human-AI Complementarity Claim Verification Trust in AI Human Oversight of AI