Overview

The field examines how people and AI can combine judgment, divide tasks, or transfer decision authority, especially when decisions are uncertain or consequential. Common approaches include calibrated advice, selective prediction and deferral, adaptive routing and ensembles, preference elicitation, and explanations designed to support human review. Researchers test these methods in behavioral experiments and applied settings such as clinical diagnosis, finance, negotiation, and public services, measuring both decision outcomes and interaction behavior. A central tension is that AI assistance can improve performance yet also encourage overreliance, weaken human skills, or shift influence and responsibility; gains depend on task context, complementary strengths, and whether people can detect when the AI is wrong.

Latest papers

199 papers20 papers in the last four weeks, down 29% on the four weeks before. 0.2% of all new papers.

Feb 19, 2026

Multi-Round Human-AI Collaboration with User-Specified Requirements

As humans increasingly rely on multiround conversational AI for high stakes decisions, principled frameworks are needed to ensure such interactions reliably improve decision quality. We adopt a human centric view governed by two principles: counterfactual harm, ensuring the AI does not undermine human strengths, and complementarity, ensuring it adds value where the human is prone to err. We formalize these concepts via user defined rules, allowing users to specify exactly what harm and complementarity mean for their specific task. We then introduce an online, distribution free algorithm with finite sample guarantees that enforces the user-specified constraints over the collaboration dynamics. We evaluate our framework across two interactive settings: LLM simulated collaboration on a medical diagnostic task and a human crowdsourcing study on a pictorial reasoning task. We show that our online procedure maintains prescribed counterfactual harm and complementarity violation rates even under nonstationary interaction dynamics. Moreover, tightening or loosening these constraints produces predictable shifts in downstream human accuracy, confirming that the two principles serve as practical levers for steering multi-round collaboration toward better decision quality without the need to model or constrain human behavior.
Feb 19, 2026

The Bots of Persuasion: Examining How Conversational Agents' Linguistic Expressions of Personality Affect User Perceptions and Decisions

Large Language Model-powered conversational agents (CAs) are increasingly capable of projecting sophisticated personalities through language, but how these projections affect users is unclear. We thus examine how CA personalities expressed linguistically affect user decisions and perceptions in the context of charitable giving. In a crowdsourced study, 360 participants interacted with one of eight CAs, each projecting a personality composed of three linguistic aspects: attitude (optimistic/pessimistic), authority (authoritative/submissive), and reasoning (emotional/rational). While the CA's composite personality did not affect participants' decisions, it did affect their perceptions and emotional responses. Particularly, participants interacting with pessimistic CAs felt lower emotional state and lower affinity towards the cause, perceived the CA as less trustworthy and less competent, and yet tended to donate more toward the charity. Perceptions of trust, competence, and situational empathy significantly predicted donation decisions. Our findings emphasize the risks CAs pose as instruments of manipulation, subtly influencing user perceptions and decisions.
Feb 18, 2026

Human-AI Collaboration in Large Language Model-Integrated Building Energy Management Systems: The Role of User Domain Knowledge and AI Literacy

This study aimed to comprehend how user domain knowledge and artificial intelligence (AI) literacy impact the effective use of human-AI interactive building energy management system (BEMS). While prior studies have investigated the potential of integrating large language models (LLMs) into BEMS or building energy modeling, very few studies have examined how user interact with such systems. We conducted a systematic role-playing experiment, where 85 human subjects interacted with an advanced generative pre-trained transformer (OpenAI GPT-4o). Participants were tasked with identifying the top five behavioral changes that could reduce home energy use with the GPT model that functioned as an LLM-integrated BEMS. Then, the collected prompt-response data and participant conclusions were analyzed using an analytical framework that hierarchically assessed and scored human-AI interactions and their home energy analysis approaches. Also, participants were classified into four groups based on their self-evaluated domain knowledge of building energy use and AI literacy, and Kruskal-Wallis H tests with post-hoc pairwise comparisons were conducted across 20 quantifiable metrics. Key takeaways include: most participants employed concise prompts (median: 16.2 words) and relied heavily on GPT's analytical capabilities; and notably, only 1 of 20 metrics, appliance identification rate, showed statistically significant group differences (p=0.037), driven by AI literacy rather than domain knowledge, suggesting an equalizing effect of LLMs across expertise levels. This study provides foundational insights into human-AI collaboration dynamics and promising development directions in the context of LLM-integrated BEMS and contributes to realizing human-centric LLM-integrated energy systems.
Feb 16, 2026

Mind the (DH) Gap! A Contrast in Risky Choices Between Reasoning and Conversational LLMs

The use of large language models either as decision support systems, or in agentic workflows, is rapidly transforming the digital ecosystem. However, the understanding of LLM decision-making under uncertainty remains limited. We study LLM risky choices along two dimensions: (1) prospect representation (based on an explicit representation or outcome history) and (2) decision rationale (explanation). Our study, which involves 20 frontier and open LLMs, is complemented by a matched human subjects experiment, which provides one reference point, while an expected payoff maximizing rational agent model provides another. We find that LLMs cluster into two categories: reasoning models (RMs) and conversational models (CMs). RMs tend towards rational behavior, are insensitive to the order of prospects, gain/loss framing, and explanations, and behave similarly whether prospects are explicit or presented via a history of outcomes. CMs are significantly less rational, slightly more human-like, sensitive to prospect ordering, framing, and explanation, and exhibit a large description-history gap. Paired comparisons of open LLMs suggest that a key factor differentiating RMs and CMs is training for mathematical reasoning.
Feb 12, 2026

Choose Your Agent: Tradeoffs in Adopting AI Advisors, Coaches, and Delegates in Multi-Party Negotiation

As AI usage becomes more prevalent in social contexts, understanding agent-user interaction is critical to designing systems that imp rove both individual and group outcomes. We present an online behavioral experiment (N=243) in which participants play three multi-tu rn bargaining games in groups of three. Each game, presented in randomized order, grants access to a single LLM assistance modality: proactive recommendations from an Advisor, reactive feedback from a Coach, or autonomous execution by a Delegate. All three modalitie s are powered by an LLM with super-human performance within this negotiation setting. On each turn, participants privately decide whe ther to act manually or use the AI modality available in that game. We document a preference-performance misalignment: participants s trongly prefer the higher-control Advisor (44%) over the Delegate (19%), yet groups only significantly increase collective surplus un der Delegate access. Adjusting for voluntary non-compliance, delegating to the AI yields suggestive individual welfare gains, roughly 1.5x the intent-to-treat estimate. A mechanism analysis traces this gap to a human filter: AI-generated proposals create more joint surplus than manual proposals across all conditions, but in the Advisor and Coach modes users modify, override, or ignore the AI's su ggestions, reverting toward human-baseline trade patterns. The Delegate advantage arises not from a different AI capability but from bypassing this filtering step altogether. Realizing these welfare gains depends not only on model capability, but on the interaction structure through which that capability is delivered. We argue that assistance modalities should be designed as mechanisms with endog enous participation; adoption-compatible interaction rules are a prerequisite to improving welfare with automated assistance.
Feb 12, 2026

Using predictive multiplicity to measure individual performance within the AI Act

When building AI systems for decision support, one often encounters the phenomenon of predictive multiplicity: a single best model does not exist; instead, one can construct many models with similar overall accuracy that differ in their predictions for individual cases. Especially when decisions have a direct impact on humans, this can be highly unsatisfactory. For a person subject to high disagreement between models, one could as well have chosen a different model of similar overall accuracy that would have decided the person's case differently. We argue that this arbitrariness conflicts with the EU AI Act, which requires providers of high-risk AI systems to report performance not only at the dataset level but also for specific persons. The goal of this paper is to put predictive multiplicity in context with the EU AI Act's provisions on accuracy and to subsequently derive concrete suggestions on how to evaluate and report predictive multiplicity in practice. Specifically: (1) We introduce the AI Act's accuracy provisions and argue that incorporating information about predictive multiplicity could serve compliance with specific provisions for providers. (2) Based on this legally rigorous analysis, we suggest individual conflict ratios and δδ-ambiguity as tools to quantify the disagreement between models on individual cases and to help detect individuals subject to conflicting predictions. (3) Based on computational insights, we derive easy-to-implement rules on how model providers could evaluate predictive multiplicity in practice. (4) Ultimately, we suggest that information about predictive multiplicity should be made available to deployers under the AI Act, enabling them to judge whether system outputs for specific individuals are reliable enough for their use case.
Feb 6, 2026

Language Model Goal Selection Differs from Humans' in a Self-Directed Learning Task

Whether in agentic workflows, social studies, or chat settings, large language models (LLMs) are increasingly being asked to replace humans in choosing which goals to pursue, rather than completing predefined tasks. However, the assumption that LLMs accurately reflect human preferences for goal setting remains largely untested. We assess the validity of LLMs as proxies for human goal selection in a controlled, self-directed learning task borrowed from cognitive science. Across five models (GPT-5, Gemini 2.5 Pro, Claude Sonnet 4.5, Qwen3 32B, and Centaur), we find substantial divergence from human behavior. While people gradually explore and learn to achieve goals with diversity across individuals, most models exploit a single identified solution or show surprisingly low performance, with distinct patterns across models and little variability across instances of the same model. Chain-of-thought reasoning and persona steering provide limited improvements, and our conclusions hold across experimental settings. While they await confirmation in applied settings, these findings highlight the uniqueness of human goal selection and caution against its replacement with current models.
Jan 31, 2026

Supporting Calibrated Reliance in AI-Assisted Decision-Making: Different Strategies for Different Tasks

As AI systems increasingly support human decision-making, a central challenge is determining what information helps people recognize when to rely on AI predictions and when to question or override them. Across three controlled human-subject studies spanning abstract visual reasoning with RAVEN matrices and deductive logical reasoning with LSAT problems, we examine how different forms of AI support affect performance in AI-assisted decision-making. A multi-stage reveal study shows that AI predictions and explanations can affect objective accuracy and subjective confidence differently. In visual reasoning, LLM explanations do not improve accuracy beyond the predicted answer alone, and no additional support format significantly outperforms prediction-only support; predicted probabilities show the highest descriptive accuracy and error recovery, while a derived selective-automation policy provides a higher-performing reference benchmark. In language-based logical reasoning, by contrast, LLM explanations yield the highest accuracy and error recovery, outperforming expert-written explanations and probability-based support. These results suggest that the effectiveness of AI support strategies depends on the task setting. Human--AI interfaces should instead be designed to support calibrated reliance and effective error recovery by matching the form of assistance to the task and the evidence available to users.
Jan 16, 2026

Predicting Biased Human Decision-Making with Large Language Models in Conversational Settings

We examine whether large language models (LLMs) can predict biased decision-making in conversational settings, and whether their predictions capture not only human cognitive biases but also how those effects change under cognitive load. In a pre-registered study (N = 1,648), participants completed six classic decision-making tasks via a chatbot with dialogues of varying complexity. Participants exhibited two well-documented cognitive biases: the Framing Effect and the Status Quo Bias. Increased dialogue complexity resulted in participants reporting higher mental demand. This increase in cognitive load selectively, but significantly, increased the effect of the biases, demonstrating the load-bias interaction. We then evaluated whether LLMs (GPT-4, GPT-5, and open-source models) could predict individual decisions given demographic information and prior dialogue. While results were mixed across choice problems, LLM predictions that incorporated dialogue context were significantly more accurate in several key scenarios. Importantly, their predictions reproduced the same bias patterns and load-bias interactions observed in humans. Across all models tested, the GPT-4 family consistently aligned with human behavior, outperforming GPT-5 and open-source models in both predictive accuracy and fidelity to human-like bias patterns. These findings advance our understanding of LLMs as tools for simulating human decision-making and inform the design of conversational agents that adapt to user biases.
Jan 14, 2026

Do people apply different norms to humans and large language models acting on their behalf? Evidence from norm elicitations in two canonical economic games

While delegating tasks to large language models (LLMs) can save people time, there is growing evidence that offloading tasks to such models produces social costs. We elicit appropriateness of behavior in two canonical economic games to study whether people have different norms for humans and LLMs acting on their behalf. More specifically, we study the social appropriateness of a spectrum of possible behaviors: when LLMs divide resources on our behalf (Dictator Game and Ultimatum Game) and when they evaluate the fairness of resource splits (Ultimatum Game). We use the Krupka-Weber norm elicitation task to detect possible shifts in social appropriateness ratings. Results of two pre-registered and incentivized experimental studies using representative samples from the UK and US (N = 2,658) show three key findings. First, in the Dictator Game, the most socially appropriate allocation remains the equal split when the Dictator is a human or an LLM acting on behalf of a human, but self-serving allocations are judged less appropriate when implemented by an LLM. Second, in the Ultimatum Game, rejecting offers from an LLM Proposer is judged as more appropriate than rejecting identical offers from a human Proposer, with this difference concentrated among relatively generous offers. Lastly, rejection decisions implemented by an LLM Responder are judged no less appropriate than identical rejection decisions implemented by a human Responder. Post hoc, these patterns are consistent with the notion that people grant LLMs bounded agency in the investigated resource-allocation scenarios. Our findings have implications for the design of AI systems.
Dec 12, 2025

Generative AI for Analysts

We study how generative artificial intelligence (GenAI) reshapes financial analysts' information production. Using the 2023 integration of GenAI into FACTSET as a plausibly exogenous change in AI access, we find that FACTSET-associated reports become markedly richer--featuring 26% more distinct information sources, 24% broader topical coverage, and 21% more analytical methods--while also improving timeliness. However, these gains do not uniformly improve decision quality: relative forecast accuracy declines when analysts face greater information-processing demands. Yet, a machine-learning benchmark processing the same observable inputs shows no analogous deterioration, pointing to a human processing constraint rather than poorer underlying information. Placebo tests using other data vendors make a common platform-wide technology trend unlikely. Overall, GenAI relaxes information-acquisition constraints while making human attention a more important bottleneck.
Sep 26, 2025

It's My Agent, Not Me: Personal Privacy Boundaries in AI-Delegated Information Sharing

Aligning AI systems with human privacy preferences requires understanding individuals' nuanced disclosure behaviors beyond general privacy norms. Yet eliciting these boundaries remains challenging because they involve context-dependent trade-offs. We present an elicitation approach that probes individuals' privacy boundaries through a discriminative task. Through a between-subjects study with varying communication roles and delegation conditions, we collected 1,650 privacy boundaries from 169 participants across 60 scenarios. We examined how communication roles, AI delegation, and individual differences influence boundaries. Results show that communication roles significantly influence sensitivity to disclosure granularity or identifiability, depending on whose private information is shared; AI delegation and individuals' need for privacy heighten sensitivity to disclosed identifiers. Our analysis further reveals how AI delegation shifts privacy preferences across contexts, highlighting new complexities in AI-mediated communication. We advocate eliciting nuanced privacy boundaries within real-world data flows by considering roles and AI delegation, and enabling user involvement in guiding privacy-aware AI behavior.
Sep 25, 2025

Understanding Role Switching in Human-AI Collaboration through Multimodal Behavioral Signals

Human-AI collaboration often requires dividing complex tasks into complementary subtasks. As a task unfolds, users may want to shift which subtasks they perform and which their AI partner performs, in response to evolving task demands and perceptions of the AI's capabilities. In this work, we investigate whether behavioral signals can reveal such changes during a sequential decision-making task. We conducted a study using hand-and-brain chess, where, on each turn, participants chose either to select the piece type (brain) while their AI partner chose the move (hand), or to choose the move after the AI selected the piece type. Across 21 chess players, this yielded more than 1,100 decisions to retain their role from the previous turn or switch to the other role. Players generally retained their current roles across turns. When participants did switch, they exhibited more exploratory gaze patterns and role switches were associated with lower subsequent move quality. Using these behavioral and task-specific signals, we trained a classifier to distinguish switch from stay decisions, achieving a PR-AUC of 0.56 (compared to a random baseline of 0.39). Feature-set ablations showed that gaze and task-specific features contributed most to model performance. Role switching was not a simple choice between controlling and delegating the task as both roles required participants to perform one subtask and delegate the other. However, interviews showed that many participants perceived selecting the piece type as giving them greater control. Perceived AI ability, relative subtask difficulty, and desired influence over the direction of play shaped these role preferences. These findings suggest that behavioral cues could help intelligent systems recognize when users want to reallocate complementary responsibilities during collaboration.
Aug 11, 2025

Unequal Uncertainty: Rethinking Algorithmic Interventions for Mitigating Discrimination from AI

Uncertainty in artificial intelligence (AI) predictions raises pressing legal and ethical questions for AI-assisted decision-making. This article examines two uncertainty-based algorithmic interventions that act as guardrails for human-AI interaction: selective abstention, which withholds high-uncertainty predictions from human decision-makers, and selective friction, which presents such predictions together with salient warnings about the model's uncertainty. Prior work suggests that uncertainty-based abstention can exacerbate disparities where under-represented groups are more likely to receive uncertain predictions. We provide, to our knowledge, the first doctrinal analysis of uncertainty-based algorithmic interventions under laws from the United Kingdom and examine their consequences through two AI-assisted case studies: consumer credit and risk of reoffending. We show that the use of uncertainty thresholds, though formally neutral, can generate discriminatory effects. We argue that both interventions pose risks of unlawful discrimination, but that selective friction is legally preferable. It preserves access to the prediction and is more likely to satisfy proportionality under the Equality Act 2010. Whether selective friction also improves decision quality in practice is uncertain. We identify conditions under which it may improve or worsen decision quality.
Aug 8, 2025

Leveraging generative AI for human understanding: meta-requirements and design principles for Explanatory AI as an emerging paradigm

Artificial intelligence (AI) systems increasingly support decision-making across critical domains, yet explainable AI (XAI) approaches prioritise algorithmic transparency over human comprehension. While XAI reveals computational processes for model validation, end users require explanations integrating domain knowledge, contextual reasoning, and professional frameworks. This disconnect reveals a design challenge: existing approaches fail to address how practitioners need to understand and act upon AI recommendations. This paper introduces Explanatory AI as an emerging complementary paradigm where AI systems leverage generative and multimodal capabilities to serve as explanatory partners for human understanding. Unlike traditional XAI, which answers 'How did the algorithm decide?' for validation purposes, Explanatory AI addresses 'Why does this make sense?' for practitioners making informed decisions. Through theory-informed design, we synthesise multidisciplinary perspectives from cognitive science, communication research, and education with empirical evidence from healthcare and AI expert interviews. Our analysis identifies five dimensions distinguishing Explanatory AI from XAI: explanatory purpose, communication mode, epistemic stance, adaptivity, and cognitive design. We derive five meta-requirements and formulate ten design principles prescribing how to build such systems, illustrated through five UI mockups exemplifying implementation. Explanatory AI operates standalone or combined with XAI, serving distinct purposes in creating systems that genuinely support human reasoning and decision-making.
Jul 1, 2025

Rethinking Group Recommender Systems in the Era of Generative AI: From One-Shot Recommendations to Conversational Group Decision Support

More than twenty-five years ago, early ideas emerged on how to design systems that provide recommendations to groups of users rather than individuals. Since then, a wide range of algorithmic approaches has been proposed to elicit individual preferences, aggregate them, and generate group recommendations. However, despite this substantial body of work, relatively few examples of real-world group recommender systems are in use. This observation invites reflection on the assumptions of the academic research, particularly regarding how inter-group communication unfolds and how recommendation-supported decisions should be made. In this essay, we suggest that some of these assumptions and corresponding system designs may not fully align with users' needs or expectations. We therefore outline a forward-looking perspective that explores how insights from group-decision making research and recent advances in Generative AI, including systems such as ChatGPT, could better inform the design of group recommender systems. As one possible direction, we consider settings in which human group members interact and exchange information through a chat based interface, while an AI-based agent supports the decision-making process. Such an approach may contribute to design more natural group and system interactions, and could help facilitate a broader adoption of group recommender systems in practice.
May 22, 2025

Serious Games: Human-AI Interaction, Evolution, and Coevolution

The serious games between humans and AI have only just begun. Evolutionary Game Theory (EGT) models the competitive and cooperative strategies of biological entities. EGT could help predict the potential evolutionary equilibrium of humans and AI. The objective of this work was to examine EGT models relevant to human-AI interaction, evolution, and co-evolution. Of thirteen EGT models considered, three were examined: the Hawk-Dove Game, Iterated Prisoner's Dilemma, and the War of Attrition. This selection was based on the widespread acceptance and clear relevance of these models to potential human-AI evolutionary dynamics and co-evolutionary trajectories. The Hawk-Dove Game predicts balanced mixed-strategy equilibria based on the costs of conflict. Iterated Prisoner's Dilemma suggests that repeated interaction may lead to cognitive co-evolution. The War of Attrition suggests that competition for resources may result in strategic co-evolution, asymmetric equilibria, and conventions on sharing resources. Each model was examined from the perspective of human and AI decision-making, from psychological and biological perspectives, and from an AI viewpoint. AI is being shaped by human input and is evolving in response to it. So too, neuroplasticity allows the human brain to evolve in response to stimuli. If humans and AI converge in future, what might be the result of human neuroplasticity combined with an ever-evolving AI? There are profound ethical and cognitive implications. EGT may provide a suitable framework to understand and predict human-AI interaction, evolution, and co-evolution. However, future research should extend beyond EGT and explore additional frameworks, empirical validation methods, and interdisciplinary perspectives. In the spirit of further exploration, an illustrative computational simulation is provided.
Jan 30, 2024

Human Responses to Algorithmic Correction: Evidence from Centre Court

We study how algorithmic review changes error composition using Hawk-Eye records from professional tennis. Within 20 mm of the line, mistaken "in" call rates increase by 8.6 percentage points, while mistaken "out" call rates decrease by 4.1 points. Within matches, calls shift toward avoiding previously overturned errors. We find no evidence that initial accuracy deteriorates within 100 mm of the line, despite concerns that workers may rely on correction as a safety net. We derive an investment framework incorporating human responses alongside correction benefits and remedy costs, showing that holding human decisions fixed can misstate the organizational returns to review.
Nov 16, 2021

Impact of Model Interpretability and Outcome Feedback on Trust in AI

This paper bridges the gap in Human-Computer Interaction (HCI) research by comparatively assessing the effects of interpretability and outcome feedback on user trust and collaborative performance with AI. Through novel pre-registered experiments (N=1,511 total participants) using an interactive prediction task, we analyzed how interpretability and outcome feedback influence users' task performance and trust in AI. The results counter the widespread belief that interpretability drives trust, showing that interpretability led to no robust improvements in trust and that outcome feedback had a significantly greater and more reliable effect. However, both factors had modest effects on participants' task performance. These findings suggest that (1) interpretability may be less effective at increasing trust than factors like outcome feedback, and (2) augmenting human performance via AI systems may not be a simple matter of increasing trust in AI, as increased trust is not always associated with equally sizable performance improvements. Our exploratory analyses further delve into the mechanisms underlying this trust-performance paradox. These findings present an opportunity for research to focus not only on methods for generating interpretations but also on techniques that ensure interpretations impact trust and performance in practice.