Trust in Autonomous Human--Robot Collaboration: Effects of Responsive Interaction Policies
Authors: Shauna Heron, Meng Cheng Lau
Organizations: School of Social Sciences, Laurentian University, Sudbury, ON, P3E 2C6 · School of Engineering and Computer Science, Laurentian University, Sudbury, ON, P3E 2C6
Trust plays a central role in human--robot collaboration, yet its formation is rarely examined under the constraints of fully autonomous interaction. This pilot study investigated how interaction policy influences trust during in-person collaboration with a social robot operating without Wizard-of-Oz control or scripted repair. Participants completed a multi-stage collaborative task with a mobile robot that autonomously managed spoken-language dialogue, affect inference, and task progression. Two interaction policies were compared: a responsive policy, in which the robot proactively adapted its dialogue and assistance based on inferred interaction state, and a neutral, reactive policy, in which the robot provided only direct, task-relevant responses when prompted. Responsive interaction was associated with significantly higher post-interaction trust under viable communication conditions, despite no reliable differences in overall task accuracy. Sensitivity analyses indicated that affective and experiential components of trust were more sensitive to communication breakdown than evaluative judgments of reliability, and that as language-mediated interaction degraded, the trust advantage associated with responsiveness attenuated and ratings became less clearly interpretable as calibrated evaluations of collaborative competence. These findings suggest that trust in autonomous human--robot interaction emerges from process-level interaction dynamics and operates within constraints imposed by communication viability, highlighting the importance of evaluating trust under real autonomy conditions when designing interactive robotic systems.
Figures & tables
Figure 1: Experimental setup showing the autonomous robot and participant-facing task interface used during in-person sessions. Participants entered task responses and navigated between task stages using the interface, while the robot autonomously tracked task state and adapted its interaction based on participant input.
Figure 2: Task 1 interface including the 6 × 4 grid of 24 candidates. Participants could track those eliminated by clicking on subjects which would grey them out. A box was provided to input their final answer and a button included to move to the next task.
Characteristic
N
CONTROL N = 13 1
RESPONSIVE N = 16 1
p-value 2
Gender
27
0.84
Woman
6 / 13 (46%)
7 / 14 (50%)
Man
7 / 13 (54%)
7 / 14 (50%)
Age Group
27
0.35
18-24
5 / 13 (38%)
7 / 14 (50%)
25-34
4 / 13 (31%)
2 / 14 (14%)
Table 1: Baseline participant characteristics by interaction policy (full sample)
Characteristic
CONTROL N = 10 1
RESPONSIVE N = 14 1
p-value 2
Experienced Trust (TI-HRC)
39 (22)
67 (21)
0.004
Subscales
Reliability
40 (24)
65 (18)
0.012
Trust Perception
42 (23)
60 (22)
0.075
Affective Trust
50 (31)
79 (22)
0.018
Perceived Trust (TPS-HRI)
59 (17)
77 (18)
0.022
Table 2: Post-Interaction Trust and Task Accuracy (eligible sample)
Figure 3: Distribution of Trust Perception by interaction policy. Points represent individual observations; violins depict score distributions. Red points indicate group means with 95% confidence intervals. Statistical comparisons are reported in the Results section.
Characteristic
CONTROL N = 10 1
RESPONSIVE N = 14 1
p-value 2
Objective Measure Averages
Dialogue Turns
34 (9)
33 (5)
0.45
Session Duration (min)
13.24 (3.06)
15.26 (2.12)
0.084
Response Time (ms)
14.37 (3.76)
17.24 (2.52)
<0.001
Silent Periods
5.60 (1.96)
4.71 (2.05)
0.29
Engaged Responses
2.00 (2.21)
3.50 (1.95)
0.040
Table 3: Post-Interaction Objective Measure Averages by Policy in the Eligible Sample
Figure 4: Posterior distributions of fixed effects from the final Bayesian mixed-effects models (eligible sample: n=24) predicting post-interaction trust. Fixed effects include policy, Negative Attitudes Towards Robots (NARS) and language (binary). Half-eye densities show posterior distributions; points indicate posterior medians; thick and thin intervals denote 80% and 95% credible intervals, respectively. The dashed vertical line marks a null effect, and positive values indicate higher trust scores on the respective scale (points on a 0-100 scale).
Figure 5: Predicted trust as a function of communication breakdown by interaction policy. Panels show experienced trust (left) and perceived trust (right). Lines represent posterior medians and shaded bands indicate 80% credible intervals. Communication breakdown is defined as the proportion of interaction turns flagged as breakdowns; predictions are shown at mean baseline negative attitudes toward robots.
Characteristic
CONTROL N = 10 1
RESPONSIVE N = 14 1
p-value 2
% Task Accuracy
0.60 (0.21)
0.66 (0.23)
0.47
Suspect ID Accuracy (robot dependent)
3 / 10 (30%)
9 / 14 (64%)
0.10
Building ID Accuracy
7 / 10 (70%)
11 / 14 (79%)
0.67
Zone ID Accuracy
5 / 10 (50%)
4 / 14 (29%)
0.40
Floor ID Accuracy
7 / 10 (70%)
13 / 14 (93%)
0.27
Table 4: Post-Interaction Task Performance by Policy in the Eligible Sample
Figure 6: Correlation Matrix of Key Study Variables
Figure 7: A selection of facial expressions the language model could choose from each turn. Retrieved from [1] N. T. White, B. Cagiltay, J. E. Michaelis, and B. Mutlu, “Designing Emotionally Expressive Social Commentary to Facilitate Child-Robot Interaction,” in Interaction Design and Children, Athens Greece: ACM, Jun. 2021, pp. 314–325. doi: 10.1145/3459990.3460714.
Variable
Type
Description
task_outcome
categorical
Final task status ( completed , timeout , skipped , partial , abandoned ).
task_completed_without_help
binary
Task was completed without any help requests to the robot.
Table 12
Variable
Type
Description
human_help_request
binary
Participant explicitly or implicitly asks the robot for help or guidance.
human_reasoning
binary
Participant reasons out loud with the robot toward problem-solving.
human_confirmation_seeking
binary
Participant seeks confirmation of a tentative belief or solution.
human_sentence_fragment
binary
Unintelligable sentence fragment
Table 13
Variable
Type
Description
robot_helpful_guidance
binary
Robot provides accurate, task-relevant information or guidance.
robot_unhelpful
binary
Robot provides misleading or incorrect guidance.
robot_stt_failure
binary
Robot response reflects a speech-to-text or input understanding failure.
robot_clarification_request
binary
Robot asks the participant for information or to repeat or clarify their input.
robot_reasoning
binary
Robot reasons out loud with the participant to assist problem-solving
Table 14
Variable
Type
Description
robot_empathy_expression
binary
Robot expresses empathy, encouragement, or reassurance.
robot_emotion_acknowledgement
binary
Robot explicitly references an inferred participant emotional state.
robot_collaborative_language
binary
Robot uses language like “we”, “lets”, “together”.
Table 15
Variable
Type
Description
human_affective_engagement
binary
Participant responds in a socially warm or engaged manner (e.g., “It’s up to us, Misty!” and/or mirrors or responds to the robot’s affective expression and/or treats the robot as a social agent.
This paper presents an empirically grounded agent-based model capturing trust dynamics, workload distribution, and collaborative performance in human-robot teams. The model, implemented in NetLogo 6.4.0, simulates teams of 2--10 agents performing tasks of varying complexity. We validate against Hancock et al.'s (2021) meta-analysis, achieving interval validity for 4 of 8 trust antecedent categories and strong ordinal validity (Spearman \r{ho}=0.833ρ= 0.833 \r{ho}=0.833). Sensitivity analysis using OFAT and full factorial designs (n=50n = 50 n=50 replications per condition) reveals robot reliability exhibits the strongest effect on trust (η2=0.35η^2 = 0.35 η2=0.35) and dominates task success (η2=0.93η^2 = 0.93 η2=0.93) and productivity (η2=0.89η^2 = 0.89 η2=0.89), consistent with meta-analytic findings. Trust asymmetry ratios ranged from 0.07 to 0.55 -- below the meta-analytic benchmark of 1.50 -- revealing that per-event asymmetry does not guarantee cumulative asymmetry when trust repair mechanisms remain active. Scenario analysis uncovered trust-performance decoupling: the Trust Recovery scenario achieved the highest productivity (4.29) despite the lowest trust (38.2), while the Unreliable Robot scenario produced the highest trust (73.2) despite the lowest task success (33.4%), establishing calibration error as a critical diagnostic distinct from trust magnitude. Factorial ANOVA confirmed significant main effects for reliability, transparency, communication, and collaboration (p<.001p < .001 p<.001), explaining 45.4% of trust variance. The open-source implementation provides an evidence-based tool for identifying overtrust and undertrust conditions prior to deployment.
Ravi Kalluri
College of Professional Studies, Northeastern University – San Jose, California, United States
AI systems are fallible, and humans can make mistakes in deciding whether to trust AI over their own judgment. Thus, improving human-AI collaboration requires understanding when, why, and how humans decide to rely on AI. We study two distinct reliance decisions: the delegation choice -- deciding when to let AI act autonomously without knowing its output, and the adoption choice -- evaluating AI suggestions and deciding how to use them. Both of these decoupled reliance patterns shape collaboration, but prior work rarely studies them together in realistic settings with the same users. We address this gap by studying collaborative human--AI teams competing in a question-answering game in which humans can choose when and how to work with AI agents to win. Our 24 matches pair 23 expert humans with 16 AI agents, capturing 387 delegation and 1440 adoption decisions. While human--AI collaboration performs better than either AI or humans alone, humans make suboptimal collaboration decisions, both under-relying on correct AI suggestions (3.9% of opportunities missed) and over-relying when AI misleads them (1.7%). Both parties contribute wrong answers: reported model confidence is near chance when humans and AI disagree, while confirmation bias drives higher under-reliance (64.5%) when an AI suggestion agrees with humans' initial incorrect answer. To close this gap, we recommend calibrated confidence, evidence-grounded explanations, and mechanisms that help users refine trust.
Maharshi Gor, Yoo Yeon Sung, Yu Hou +4
University of Maryland College Park, MD, USA · University of California Berkeley, CA, USA · Phasechange.ai Lakewood, CO, USA +1
Robots are entering human spaces faster than we can establish when they deserve trust. We propose a framework for trustworthy physical AI that integrates Safety, Behavioral Intelligibility, and Perceptual Alignment across embodiment, control, cognition, and design. Trustworthiness emerges from aligning physical capabilities, observable behavior, and expectations people form during interaction.
Niccolò Pagliarani, Maximilian Stölzle, Cosima du Pasquier +6
The BioRobotics Institute, Scuola Superiore Sant’Anna, Pontedera, Italy. · CSAIL, Massachusetts Institute of Technology, Cambridge, MA, USA. · CHARM Lab, Stanford University, Stanford, CA, USA. +3