cs.AIFeb 23, 2026

Align When They Want, Complement When They Need! Human-Centered Ensembles for Adaptive Human-AI Collaboration

Authors: Hasan Amin, Ming Yin, Rajiv Khanna

Organizations: Purdue University, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

In human-AI decision making, designing AI that complements human expertise has been a natural strategy to enhance human-AI collaboration, yet it often comes at the cost of decreased AI performance in areas of human strengths. This can inadvertently erode human trust and cause them to ignore AI advice precisely when it is most needed. Conversely, an aligned AI fosters trust yet risks reinforcing suboptimal human behavior and lowering human-AI team performance. In this paper, we start by identifying this fundamental tension between performance-boosting (i.e., complementarity) and trust-building (i.e., alignment) as an inherent limitation of the traditional approach for training a single AI model to assist human decision making. To overcome this, we introduce a novel human-centered adaptive AI ensemble that strategically toggles between two specialist AI models - the aligned model and the complementary model - based on contextual cues, using an elegantly simple yet provably near-optimal Rational Routing Shortcut mechanism. Comprehensive theoretical analyses elucidate why the adaptive AI ensemble is effective and when it yields maximum benefits. Moreover, experiments on both simulated and real-world data show that when humans are assisted by the adaptive AI ensemble in decision making, they can achieve significantly higher performance than when they are assisted by single AI models that are trained to either optimize for their independent performance or even the human-AI team performance.

Figures & tables

Figure 1: Summary comparison of human-AI team performance across AI design paradigms. Standard AI optimizes independent accuracy; aligned and complementary AIs specialize in trust-building and error-correction, respectively; behavior-aware AI optimizes team loss under our human interaction model. Our proposed adaptive AI ensemble, including the more practical one using Rational Routing Shortcut (RRS), strategically toggles between aligned and complementary AI, and achieves the highest team accuracy. Results shown here correspond to the behavior-grounded evaluation on WoofNette data.
Figure 2: Complementarity-alignment tradeoff increases sharply with specialization, exacerbated by imperfect human DM in alignment region ( α\alpha ), illustrating that a single model cannot simultaneously optimize for trust and performance.
Figure 3: Accuracy gain ( Γteam\Gamma_{\text{team}} ) of adaptive AI ensemble over single AI on the College Admissions data. The plots (left to right) empirically validate that: (i) gain increases with specialist divergence ( ∥θma∗−θmc∗∥\|\theta_{m_{a}}^{*}-\theta_{m_{c}}^{*}\| ); (ii) gain scales with human accuracy ( α\alpha ) in alignment region (i.e., the aligned group’s feature regime), reflecting the reliability factor κ\kappa ; (iii) gain peaks when the task mixture is balanced (i.e., aligned group fraction p≈0.5p\approx 0.5 ), matching the concave p(1−p)p(1-p) dependence; and (iv) gain increases linearly with group certainty ( 1−H/2log⁡21-\mathcal{H}/2\log 2 ), demonstrating robustness to routing uncertainty.
ParadigmAI AccuracyTeam Accuracy
Standard AI69.87±0.4469.87_{\pm 0.44}69.13±0.2869.13_{\pm 0.28}
Aligned AI61.71±0.5661.71_{\pm 0.56}60.73±0.2460.73_{\pm 0.24}
Complementary AI61.01±0.7761.01_{\pm 0.77}69.96±0.5069.96_{\pm 0.50}
Behavior-Aware AI64.99±0.9764.99_{\pm 0.97}70.90±0.3670.90_{\pm 0.36}
Adaptive AI (Oracle)80.37±0.3180.37_{\pm 0.31}74.75±0.3474.75_{\pm 0.34}
Adaptive AI (RRS)82.64±0.3582.64_{\pm 0.35}75.13±0.3275.13_{\pm 0.32}
Table 1: Accuracy (%) of independent AI and Human-AI team across different AI paradigms on the WoofNette benchmark. Human accuracy is 65.10±0.2765.10_{\pm 0.27} .
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure A1: Samples from College Admission dataset with p=0.5p=0.5 and δ=0.25\delta=0.25 .
Figure A2: Visualization of decision boundary of standard and specialist logistic regression models on College Admission dataset with p=0.5p=0.5 and δ=0.25\delta=0.25 .
Figure A3: Adaptive AI gains extend beyond logistic regression and grow quadratically with specialist divergence across varied models.
Figure A4: Adaptive AI gains persist across the full family of fixed-weight single-model surrogates. Top: accuracy gains over standard AI. Pure specialists ( w∈{0,1}w{\in}\{0,1\} ) can outperform intermediate weights, yet adaptive AI consistently dominates. Bottom: log-loss improvement (positive values indicate lower loss). Fixed-weight deviations do not improve upon the population-consistent objective, confirming w=pw=p as the appropriate single-model baseline; adaptive AI achieves further improvement by routing to specialists.
Figure A5: Sample images from the WoofNette dataset.

Explore similar work

CardsList

  1. AI, Take the Wheel: What Drives Delegation and Trust in Human-Computer Cooperative Question Answering?

    May 27, 2026Maharshi Gor, Yoo Yeon Sung, Yu Hou +4Human-AI AgreementTrust in AI

    University of Maryland College Park, MD, USA · University of California Berkeley, CA, USA · Phasechange.ai Lakewood, CO, USA +1

  2. Learning to Decide with AI Assistance under Human-Alignment

    May 12, 2026Nina Corvelo Benz, Eleni Straitouri, Manuel Gomez-RodriguezAI-Assisted Decision MakingHuman-AI Decision Making

    §Max Planck Institute for Biochemistry, Martinsried, Germany · †Max Planck Institute for Software Systems, Kaiserslautern, Germany

  3. Human-Centric Reflective Architecture for Human-AI Collaborative Decision-Making

    Jul 3, 2026Andreas Kouridakis, Dimitrios Patiniotis Spyropoulos, George VourosHuman Preference ModelingHuman-in-the-Loop AI

    University of Piraeus Piraeus, Greece