Human-Robot Interaction

Also known as HRI

Follow this topic Ask about this topic

Human-robot interaction examines how people and robots understand one another and coordinate in settings such as collaborative work, social navigation, healthcare, and the home. Researchers combine multimodal perception and language models with shared-control policies, adaptive planning, haptics, and interfaces such as augmented reality; they study these systems through user experiments, field deployments, and benchmarks for tasks like handover and social navigation. The field also draws on design research and qualitative methods to understand trust, accessibility, privacy, agency, and how interactions affect different people and groups. A central challenge is translating strong task performance in controlled tests into safe, reliable, and socially appropriate behavior in real-world settings without increasing workload or reducing human control.

Latest papers

453 papers44 papers in the last four weeks, down 31% on the four weeks before. 0.3% of all new papers.

Mar 11, 2026

Safe Probabilistic Planning for Human-Robot Interaction using Conformal Risk Control

In this paper, we present a novel probabilistic safe control framework for human-robot interaction that combines control barrier functions (CBFs) with conformal risk control to provide formal safety guarantees while considering complex human behavior. The approach uses conformal risk control to quantify and control the prediction errors in CBF safety values and establishes formal guarantees on the probability of constraint satisfaction during interaction. We introduce an algorithm that dynamically adjusts the safety margins produced by conformal risk control based on the current interaction context. Through experiments on human-robot navigation scenarios, we demonstrate that our approach significantly reduces collision rates and safety violations as compared to baseline methods while maintaining high success rates in goal-reaching tasks and efficient control. The code, simulations, and other supplementary material can be found on the project website: https://jakeagonzales.github.io/crc-cbf-website/.

Jake Gonzales, Kazuki Mizuta, Karen Leung +1

University of Washington, Department of Electrical and Computer Engineering · University of Washington, Department of Aeronautics and Astronautics

Human-Robot Interaction Safe Robot Navigation Robot Collision Avoidance Control Barrier Functions Conformal Risk Control

Mar 10, 2026

Design of a Robot-Assisted Chemical Dialysis System

Scientists perform diverse manual procedures that are tedious and laborious. Such procedures are considered a bottleneck for modern experimental science, as they consume time and increase burdens in fields including material science and medicine. We employ a user-centered approach to designing a robot-assisted system for dialysis, a common multi-day purification method used in polymer and protein synthesis. Through two usability studies, we obtain participant feedback and revise design requirements to develop the final system that satisfies scientists' needs and has the potential for applications in other experimental workflows. We anticipate that integration of this system into real synthesis procedures in a chemical wet lab will decrease workload on scientists during long experimental procedures and provide an effective approach to designing more systems that have the potential to accelerate scientific discovery and liberate scientists from tedious labor.

Diane Jung, Caleb Escobedo, Noah Liska +4

University of Colorado at Boulder, Boulder, CO, USA · University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

Laboratory Automation Scientific Workflow Automation Human-Robot Interaction

Mar 10, 2026

TATIC: Task-Aware Temporal Learning for Human Intent Inference from Physical Corrections in Human-Robot Collaboration

In human-robot collaboration (HRC), robots must adapt online to dynamic task constraints and evolving human intent. While physical corrections provide a natural, low-latency channel for operators to convey motion-level adjustments, extracting task-level semantic intent from such brief interactions remains challenging. Existing foundation-model-based approaches primarily rely on vision and language inputs and lack mechanisms to interpret physical feedback. Meanwhile, traditional physical human-robot interaction (pHRI) methods leverage physical corrections for trajectory guidance but struggle to infer task-level semantics. To bridge this gap, we propose TATIC, a unified framework that utilizes torque-based contact force estimation and a task-aware Temporal Convolutional Network (TCN) to jointly infer discrete task-level intent and estimate continuous motion-level parameters from brief physical corrections. Task-aligned feature canonicalization ensures robust generalization across diverse layouts, while an intent-driven adaptation scheme translates inferred human intent into robot motion adaptations. Experiments achieve a 0.904 Macro-F1 score in intent recognition and demonstrate successful hardware validation in collaborative disassembly (see experimental video at https://youtu.be/xF8A52qwEc8).

Jiurun Song, Xiao Liang, Minghui Zheng

Department of Mechanical Engineering, Texas A&M University, College Station, TX 77843, USA · Department of Civil and Environmental Engineering, Texas A&M University, College Station, TX 77843, USA

Human-Robot Collaboration Human-Robot Interaction

Mar 10, 2026

Characterizing Healthy & Post-Stroke Neuromotor Behavior During 6D Upper-Limb Isometric Gaming: Implications for Design of End-Effector Rehabilitation Robot Interfaces

Successful robot-mediated rehabilitation requires designing games and robot interventions that promote healthy motor practice. However, the interplay between a given user's neuromotor behavior, the gaming interface, and the physical robot makes designing system elements -- and even characterizing what behaviors are "healthy" or pathological -- challenging. We leverage our OpenRobotRehab 1.0 open access data set to assess the characteristics of 13 healthy and 2 post-stroke users' force output, muscle activations, and game performance while executing isometric trajectory tracking tasks using an end-effector rehabilitation robot. We present an assessment of how subtle aspects of interface design impact user behavior; an analysis of how pathological neuromotor behaviors are reflected in end-effector force dynamics; and a novel hidden Markov model (HMM)-based neuromotor behavior classification method based on surface electromyography (sEMG) signals during cyclic motions. We demonstrate that task specification (including which axes are constrained and how users interpret tracking instructions) shapes user behavior; that pathology-related features are detectable in 6D end-effector force data during isometric task execution (with significant differences between healthy and post-stroke profiles in force error and average force production at

p=0.05

); and that healthy neuromotor strategies are heterogeneous and inherently difficult to characterize. We also show that our HMM-based models discriminate healthy and post-stroke neuromotor dynamics where synergy-based decompositions reflect no such differentiation. Lastly, we discuss these results' implications for the design of adaptive end-effector rehabilitation robots capable of promoting healthier movement strategies across diverse user populations.

Ajay Anand, Gabriel Parra, Chad A. Berghoff +1

The authors are with the Departments of Mechanical Engineering, Physical Medicine & Rehabilitation, and Mathematics, the Kahlert School of Computing, and the Robotics Center at the University of Utah, Salt Lake City, UT 84112, USA.

Surface Electromyography Assistive Robotics Human-Robot Interaction

Mar 10, 2026

Uncertainty, Vagueness, and Ambiguity in Human-Robot Interaction: Why Conceptualization Matters

Uncertainty, vagueness, and ambiguity are closely related and often confused concepts in human-robot interaction (HRI). In earlier studies, these concepts have been defined in contradictory ways and described using inconsistent terminology. This conceptual confusion and lack of terminological consistency undermine empirical comparability, thereby slowing the accumulation of theory. Consequently, consistent concepts that clarify these challenges, including their definitions, distinctions, and interrelationships, are needed in HRI. To address this lack of clarity, this paper proposes a consistent conceptual foundation for the challenges of uncertainty, vagueness, and ambiguity in HRI. First, we examine the meanings of these three terms in dictionaries. We then analyze the nature of their distinctions and interrelationships within the context of HRI. We further illustrate these characteristics through examples. Finally, we demonstrate how this consistent conceptual foundation facilitates the design of novel methods and the evaluation of existing methodologies for these phenomena.

Xiaowen Sun, Cornelius Weber, Matthias Kerzel +2

The Authors are with the Knowledge Technology Group, Department of Informatics, University of Hamburg, 22527 Hamburg, Germany.

Human-Robot Interaction

Mar 10, 2026

Embodied Human Simulation for Quantitative Design and Analysis of Interactive Robotics

Physical interactive robotics, ranging from wearable devices to collaborative humanoid robots, require close coordination between mechanical design and control. However, evaluating interactive dynamics is challenging due to complex human biomechanics and motor responses. Traditional experiments rely on indirect metrics without measuring human internal states, such as muscle forces or joint loads. To address this issue, we develop a scalable simulation-based framework for the quantitative analysis of physical human-robot interaction. At its core is a full-body musculoskeletal model serving as a predictive surrogate for the human dynamical system. Driven by a reinforcement learning controller, it generates adaptive, physiologically grounded motor behaviors. We employ a sequential training pipeline where the pre-trained human motion control policy acts as a consistent evaluator, making large-scale design space exploration computationally tractable. By simulating the coupled human-robot system, the framework provides access to internal biomechanical metrics, offering a systematic way to concurrently co-optimize a robot's structural parameters and control policy. We demonstrate its capability in optimizing human-exoskeleton interactions, showing improved joint alignment and reduced contact forces. This work establishes embodied human simulation as a scalable paradigm for interactive robotics design.

Chenhui Zuo, Jinhao Xu, Michael Qian Vergnolle +1

The authors are with the School of Aerospace Engineering, Tsinghua University, Beijing, China 100084.

Musculoskeletal Modeling Robotics Human-Robot Interaction Robotics Simulation Robot Morphology-Control Co-Design

Mar 9, 2026

Improving through Interaction: Searching Behavioral Representation Spaces with CMA-ES-IG

Robots that interact with humans must adapt to individual users' preferences to operate effectively in human-centered environments. An intuitive and effective technique to learn non-expert users' preferences is through rankings of robot behaviors, e.g., trajectories, gestures, or voices. Existing techniques primarily focus on generating queries that optimize preference learning outcomes, such as sample efficiency or final preference estimation accuracy. However, the focus on outcome overlooks key user expectations in the process of providing these rankings, which can negatively impact users' adoption of robotic systems. This work proposes the Covariance Matrix Adaptation Evolution Strategies with Information Gain (CMA-ES-IG) algorithm. CMA-ES-IG explicitly incorporates user experience considerations into the preference learning process by suggesting perceptually distinct and informative trajectories for users to rank. We demonstrate these benefits through both simulated studies and real-robot experiments. CMA-ES-IG, compared to state-of-the-art alternatives, (1) scales more effectively to higher-dimensional preference spaces, (2) maintains computational tractability for high-dimensional problems, (3) is robust to noisy or inconsistent user feedback, and (4) is preferred by non-expert users in identifying their preferred robot behaviors. This project's code is available at github.com/interaction-lab/CMA-ES-IG

Nathaniel Dennler, Zhonghao Shi, Yiran Tao +3

CSAIL/AeroAstro, Massachusetts Institute of Technology. · Department of Computer Science, University of Southern California.

Active Preference Learning Preference Learning Preference Elicitation Human-Robot Interaction Pairwise Preference Learning Covariance Matrix Adaptation Evolution Strategy

Mar 9, 2026

Decision-Aware Uncertainty Evaluation of Vision-Language Model-Based Early Action Anticipation for Human-Robot Interaction

Robots in shared workspaces must interpret human actions from partial, ambiguous observations, where overconfident early predictions can lead to unsafe or disruptive interaction. This challenge is amplified in egocentric views, where viewpoint changes and occlusions increase perceptual noise and ambiguity. As a result, downstream human-robot interaction modules require not only an action hypothesis but also a trustworthy estimate of confidence under partial observation. Recent vision-language model-based approaches have been proposed for short-term action recognition due to their open-vocabulary and context-aware reasoning, but their uncertainty reliability in the temporal-prefix regime is largely uncharacterized. We present the first systematic evaluation of uncertainty in vision-language model-based short-term action recognition for human-robot interaction. We introduce a temporal-prefix evaluation protocol and metrics for calibration and selective prediction. We also characterize miscalibration patterns and failure modes under partial observations. Our study provides the missing reliability evidence needed to use vision-language model predictions in confidence-gated human-robot interaction modules.

Zhaoda Du, Michael Bowman, Qiaojie Zheng +1

Colorado School of Mines, Golden, CO 80401, USA · Postdoctoral Fellow in the Cancer Biology Department at the University of Pennsylvania, Philadelphia, PA 19104, USA

Uncertainty Estimation for VLA Models VLMs for Robotics Human Behavior Prediction Human-Robot Interaction

Mar 9, 2026

Impact of Different Failures on a Robot's Perceived Reliability

Robots fail, potentially leading to a loss in the robot's perceived reliability (PR), a measure correlated with trustworthiness. In this study we examine how various kinds of failures affect the PR of the robot differently, and how this measure recovers without explicit social repair actions by the robot. In a preregistered and controlled online video study, participants were asked to predict a robot's success in a pick-and-place task. We examined manipulation failures (slips), freezing (lapses), and three types of incorrect picked objects or place goals (mistakes). Participants were shown one of 11 videos -- one of five types of failure, one of five types of failure followed by a successful execution in the same video, or a successful execution video. This was followed by two additional successful execution videos. Participants bet money either on the robot or on a coin toss after each video. People's betting patterns along with a qualitative analysis of their survey responses highlight that mistakes are less damaging to PR than slips or lapses, and some mistakes are even perceived as successes. We also see that successes immediately following a failure have the same effect on PR as successes without a preceding failure. Finally, we show that successful executions recover PR after a failure. Our findings highlight which robot failures are in higher need of repair in a human-robot interaction, and how trust could be recovered by robot successes.

Andrew Violette, Zhanxin Wu, Haruki Nishimura +5

Department of Computer Science, Cornell University · Toyota Research Institute (TRI). · Sibley School of Mechanical and Aerospace Engineering, Cornell University

Trust in HRI Robot Failure Recovery Human-Robot Interaction

Mar 9, 2026

Age-Related Differences in the Perception of Eye-Gaze from a Social Robot

There is an increasing interest in social robots assisting older adults during daily life tasks. In this context, non-verbal cues such as deictic gaze are important in natural communication in human-robot interaction. However, the sensibility to deictic-gaze declines naturally with age and results in a reduction in social perception. Therefore, this work explores the benefits of deictic gaze from social robots assisting older adults during daily life tasks, and how age-related differences may influence their social perception in contrast to younger populations. This may help on the design of adaptive age-related non-verbal cues in the Human-Robot Interaction context.

Lucas Morillo-Mendez, Martien G. S. Schrooten, Oscar Martinez Mozos

Örebro University, Fakultetsgatan 1, 702 81, Örebro, Sweden

Human-Robot Interaction Social Robotics

Mar 9, 2026

Human-Aware Robot Behaviour in Self-Driving Labs

Self-driving laboratories (SDLs) are rapidly transforming research in chemistry and materials science to accelerate new discoveries. Mobile robot chemists (MRCs) play a pivotal role by autonomously navigating the lab to transport samples, effectively connecting synthesis, analysis, and characterisation equipment. The instruments within an SDL are typically designed or retrofitted to be accessed by both human and robotic chemists, ensuring operational flexibility and integration between manual and automated workflows. In many scenarios, human and robotic chemists may need to use the same equipment simultaneously. Currently, MRCs rely on simple LiDAR-based obstruction detection, which forces the robot to passively wait if a human is present. This lack of situational awareness leads to unnecessary delays and inefficient coordination in time-critical automated workflows in human-robot shared labs. To address this, we present an initial study of an embodied, AI-driven perception method that facilitates proactive human-robot interaction in shared-access scenarios. Our method features a hierarchical human intention prediction model that allows the robot to distinguish between preparatory actions (waiting) and transient interactions (accessing the instrument). Our results demonstrate that the proposed approach enhances efficiency by enabling proactive human-robot interaction, streamlining coordination, and potentially increasing the efficiency of autonomous scientific labs.

Satheeshkumar Veeramani, Anna Kisil, Abigail Bentley +3

Department of Chemistry University of Liverpool United Kingdom · School of Computer Science and Informatics University of Liverpool United Kingdom · School of Computer Science and Informatics Department of Chemistry University of Liverpool United Kingdom

Human-Robot Collaboration Human-Robot Interaction Laboratory Automation

Mar 9, 2026

POIROT: Investigating Direct Tangible vs. Digitally Mediated Interaction and Attitude Moderation in Multi-party Murder Mystery Games

As social robots take on increasingly complex roles like game masters (GMs) in multi-party games, the expectation that physicality universally enhances user experience remains debated. This study challenges the "one-size-fits-all" view of tangible interaction by identifying a critical boundary condition: users' Negative Attitudes towards Robots (NARS). In a between-subjects experiment (N = 67), a custom-built robot GM facilitated a multi-party murder mystery game (MMG) by delivering clues either through direct tangible interaction or a digitally mediated interface. Baseline multivariate analysis (MANOVA) showed no significant main effect of delivery modality, confirming that tangibility alone does not guarantee superior engagement. However, primary analysis using multilevel linear models (MLM) revealed a reliable moderation: participants high in NARS experienced markedly lower narrative immersion under tangible delivery, whereas those with low NARS scores showed no such decrement. Qualitative findings further illuminate this divergence: tangibility provides novelty and engagement for some but imposes excessive proxemic friction for anxious users, for whom the digital interface acts as a protective social buffer. These results advance a conditional model of HRI and emphasize the necessity for adaptive systems that can tailor interaction modalities to user predispositions.

Wen Chen, Rongxi Chen, Shankai Chen +5

Tsinghua University Beijing, China · University of Pennsylvania Philadelphia, USA · Syracuse University Syracuse, USA +2

Human-Robot Interaction Human-Object Interaction Social Robotics

Mar 9, 2026

Identifying Influential Actions in Human-Robot Interactions

Human-robot interaction combines robotics, cognitive science, and human factors to study collaborative systems. This paper introduces a method for identifying influential robot actions using transfer entropy, a statistic that measures directed information transfer between time series. TE is effective for capturing complex, nonlinear interactions. We apply this method to analyze how robot actions affect human behavior during a conversation with a remotely controlled robot avatar. By focusing on the impact of proximity, our approach demonstrates TE's capability to identify key actions influencing human responses, highlighting its potential to improve the design and adaptability of robotic systems.

Haoyang Jiang, Chenfei Xu, Yuya Okadome +1

Guardian Robot Project, R-IH, Riken, Kyoto, Japan · Faculty of Engineering, Monash University, Melbourne, Australia · Graduate school of engineering science, Osaka University, Osaka, Japan +1

Human-Robot Interaction Social Robotics

Mar 8, 2026

Knowing When to Ask: Resolving Uncertainty in Human-Robot Joint Planning via Explicit Dialogue and Implicit Intent Cues

Effective human-robot collaboration in open-world environments requires joint planning under uncertainty about the task, the environment, and the human teammate. Communication is the most direct means of resolving such uncertainty, yet most existing systems support only one-way communication: robots listen and act, treating humans as passive supervisors rather than conversational teammates capable of two-way dialogue. We propose a unified human-robot joint planning system in which the robot actively resolves uncertainty through two complementary communication channels. When uncertainty is decision-critical, an uncertainty-mitigation joint planning module engages the human in clarification dialogue: it grounds ambiguous instructions via an LLM-assisted active elicitation mechanism, enumerates traversability hypotheses through a hypothesis-augmented A* search, and computes a cost-optimal querying policy via dynamic programming, so that the robot asks only the questions whose answers actually matter for the plan. When explicit dialogue is unnecessary or impractical, a real-time intent-aware collaboration module instead reads implicit, nonverbal cues, maintaining a probabilistic belief over the human's latent task intent from spatial and directional signals to enable coordination-aware task selection without any communication overhead. We validate the proposed system in both Gazebo simulations and real-world UAV deployments, integrated with a voice dialogue interface and a Vision-Language Model (VLM)-based 3D semantic perception pipeline. Experimental results show that cost-optimal clarification dialogue cuts the interaction cost by 51.9% while maintaining a 100% task success rate, and implicit intent reading reduces the cooperative task execution time by 25.4% compared to the baselines.

Zeyu Fang, Yuxin Lin, Cheng Liu +6

Department of Electrical and Computer Engineering, George Washington University · the Department of Electrical and Computer Engineering, Northeastern University · the Department of Mechanical and Aerospace Engineering, George Washington University

Human-Robot Collaboration Robot Task Planning Human-Robot Interaction

Mar 8, 2026

Directing the Robot: Scaffolding Creative Human-AI-Robot Interaction

Robots are moving beyond industrial settings into creative, educational, and public environments where interaction is open-ended and improvisational. Yet much of human-AI-robot interaction remains framed around performance and efficiency, positioning humans as supervisors rather than collaborators. We propose a re-framing of AI interaction with robots as scaffolding: infrastructure that enables humans to shape robotic behaviour over time while remaining meaningfully in control. Through scenarios from creative practice, learning-by-teaching, and embodied interaction, we illustrate how humans can act as executive directors, defining intent and steering revisions, while AI mediates between human expression and robotic execution. We outline design and evaluation implications that foreground creativity, agency, and flow. Finally, we discuss open challenges in social, scalable, and mission-critical contexts. We invite the community to rethink interacting with Robots and AI not as autonomy, but as sustained support for human creativity.

Jordan Aiko Deja, Isidro Butaslac, Nicko Reginio Caluya +1

De La Salle University Manila, Philippines · Nara Institute of Science and Technology Ikoma, Japan · Ritsumeikan University Osaka, Japan +1

Human-Robot Interaction Human-Robot Collaboration Human-AI Co-Creation Human-Centered AI

Mar 7, 2026

Morphology-Independent Facial Expression Imitation for Human-Face Robots

Accurate facial expression imitation on human-face robots is crucial for achieving natural human-robot interaction. Most existing methods have achieved photorealistic expression imitation through mapping 2D facial landmarks to a robot's actuator commands. Their imitation of landmark trajectories is susceptible to interference from facial morphology, which would lead to a performance drop. In this paper, we propose a morphology-independent expression imitation method that decouples expressions from facial morphology to eliminate morphological influence and produce more realistic expressions for human-face robots. Specifically, we construct an expression decoupling module to learn expression semantics by disentangling the expression representation from the morphology representation in a self-supervised manner. We devise an expression transfer module to map the representations to the robot's actuator commands through a learning objective of perceiving expression errors, producing accurate facial expressions based on the learned expression semantics. To support experimental validation, a custom-designed and highly expressive human-face robot, namely Pengrui, is developed to serve as an experimental platform for realistic expression imitation. Extensive experiments demonstrate that our method enables the human-face robot to reproduce a wide range of human-like expressions effectively. All code and implementation details of the robot will be released.

Xu Chen, Rui Gao, Che Sun +4

Robot Imitation Learning Humanoid Robotics Human-Robot Interaction

Mar 6, 2026

Material Driven HRI Design: Aesthetics as Explainability

Aesthetics - often treated as secondary to function-guides how people interpret robots' roles. A great deal of robot designs - both real and fictitious - use sleek industrial aesthetics. These feature hard glossy plastics, hiding as much of the underlying mechanical and electrical components as possible, resembling something akin to a nude humanoid figure. This leaves robots as something of a blank slate to which end-users apply coverings to, often based on media of fiction and non-fiction alike. We argue that designers can take cues from fashion to design interaction and set appropriate expectations. Rather than viewing appearance as decoration, we propose that color, texture, and material choices function as interaction signals. These signals can invite or discourage touch, clarify a robot's role, and help align user expectations with a robot's actual capabilities. When done thoughtfully, such cues can create familiarity and legibility; when done poorly, they can lead to wrong expectations. This preliminary paper proposes a framework describing how materials can create explainability by signaling expectations for interaction, task, and environment. We use this framework to do a content analysis of 6 robots.

Natalie Friedman, Kevin Weatherwax, Chengchao Zhu

BTP SAP Innovation Palo Alto, CA, USA

Human-Robot Interaction Social Robotics

Mar 6, 2026

RoboCritics: Enabling Reliable End-to-End LLM Robot Programming through Expert-Informed Critics

End-user robot programming grants users the flexibility to re-task robots in situ, yet it remains challenging for novices due to the need for specialized robotics knowledge. Large Language Models (LLMs) hold the potential to lower the barrier to robot programming by enabling task specification through natural language. However, current LLM-based approaches generate opaque, "black-box" code that is difficult to verify or debug, creating tangible safety and reliability risks in physical systems. We present RoboCritics, an approach that augments LLM-based robot programming with expert-informed motion-level critics. These critics encode robotics expertise to analyze motion-level execution traces for issues such as joint speed violations, collisions, and unsafe end-effector poses. When violations are detected, critics surface transparent feedback and offer one-click fixes that forward structured messages back to the LLM, enabling iterative refinement while keeping users in the loop. We instantiated RoboCritics in a web-based interface connected to a UR3e robot and evaluated it in a between-subjects user study (n=18). Compared to a baseline LLM interface, RoboCritics reduced safety violations, improved execution quality, and shaped how participants verified and refined their programs. Our findings demonstrate that RoboCritics enables more reliable and user-centered end-to-end robot programming with LLMs.

Callie Y. Kim, Nathan Thomas White, Evan He +2

Department of Computer Sciences, University of Wisconsin–Madison, Madison, Wisconsin, USA

Robot Safety Program Synthesis Human-Robot Interaction

Mar 6, 2026

Dual-Agent Multiple-Model Reinforcement Learning for Event-Triggered Human-Robot Co-Adaptation in Decoupled Task Spaces

This paper presents a shared-control rehabilitation policy for a custom 6-degree-of-freedom (6-DoF) upper-limb robot that decomposes complex reaching tasks into decoupled spatial axes. The patient governs the primary reaching direction using binary commands, while the robot autonomously manages orthogonal corrective motions. Because traditional fixed-frequency control often induces trajectory oscillations due to variable inverse-kinematics execution times, an event-driven progression strategy is proposed. This architecture triggers subsequent control actions only when the end-effector enters an admission sphere centred on the immediate target waypoint, and was validated in a semi-virtual setup linking a physical pressure sensor to a MuJoCo simulation. To optimise human--robot co-adaptation safely and efficiently, this study introduces Dual Agent Multiple Model Reinforcement Learning (DAMMRL). This framework discretises decision characteristics: the human agent selects the admission sphere radius to reflect their inherent speed--accuracy trade-off, while the robot agent dynamically adjusts its 3D Cartesian step magnitudes to complement the user's cognitive state. Trained in simulation and deployed across mixed environments, this event-triggered DAMMRL approach effectively suppresses waypoint chatter, balances spatial precision with temporal efficiency, and significantly improves success rates in object acquisition tasks.

Yaqi Li, Zhengqi Han, Huifang Liu +1

Faculty of Engineering and IT, University of Technology Sydney, Australia · College of Medical information and Artificial intelligence, Shandong First Medical University (Shandong Academy of Medical Science) 6699 Qingdao Road, Jinan, Shandong, China

Human-Robot Interaction RL for Robotics Shared Autonomy Assistive Robotics Event-Triggered Control

Mar 5, 2026

Vision-Language System using Open-Source LLMs for Gestures in Medical Interpreter Robots

Effective communication is vital in healthcare, especially across language barriers, where non-verbal cues and gestures are critical. This paper presents a privacy-preserving vision-language framework for medical interpreter robots that detects specific speech acts (consent and instruction) and generates corresponding robotic gestures. Built on locally deployed open-source models, the system utilizes a Large Language Model (LLM) with few-shot prompting for intent detection. We also introduce a novel dataset of clinical conversations annotated for speech acts and paired with gesture clips. Our identification module achieved 0.90 accuracy, 0.93 weighted precision, and a 0.91 weighted F1-Score. Our approach significantly improves computational efficiency and, in user studies, outperforms the speech-gesture generation baseline in human-likeness while maintaining comparable appropriateness.

Thanh-Tung Ngo, Emma Murphy, Robert J. Ross

Technological University Dublin Dublin, Ireland

Healthcare Robotics Human-Robot Interaction Robotics Co-Speech Gesture Generation VLMs for Robotics

Mar 5, 2026

RFM-HRI : A Multimodal Dataset of Medical Robot Failure, User Reaction and Recovery Preferences for Item Retrieval Tasks

While robots deployed in real-world environments inevitably experience interaction failures, understanding how users respond through verbal and non-verbal behaviors remains under-explored in human-robot interaction (HRI). This gap is particularly significant in healthcare-inspired settings, where interaction failures can directly affect task performance and user trust. We present the Robot Failures in Medical HRI (RFM-HRI) Dataset, a multimodal dataset capturing dyadic interactions between humans and robots embodied in crash carts, where communication failures are systematically induced during item retrieval tasks. Through Wizard-of-Oz studies with 41 participants across laboratory and hospital settings, we recorded responses to four failure types (speech, timing, comprehension, and search) derived from three years of crash-cart robot interaction data. The dataset contains 214 interaction samples including facial action units, head pose, speech transcripts, and post-interaction self-reports. Our analysis shows that failures significantly degrade affective valence and reduce perceived control compared to successful interactions. Failures are strongly associated with confusion, annoyance, and frustration, while successful interactions are characterized by surprise, relief, and confidence in task completion. Emotional responses also evolve across repeated failures, with confusion decreasing and frustration increasing over time. This work contributes (1) a publicly available multimodal dataset (RFM-HRI), (2) analysis of user responses to different failure types and preferred recovery strategies, and (3) a crash-cart retrieval scenario enabling systematic comparison of recovery strategies with implications for safety-critical failure recovery. Our findings provide a foundation for failure detection and recovery methods in embodied HRI.

Yashika Batra, Giuliano Pioldi, Promise Ekpo +5

Cornell University, Cornell Tech, United States of America · Weill Cornell Medicine, United States of America

Human-Robot Interaction Robot Failure Recovery Robotic Data Collection Affective Computing

Mar 5, 2026

NEFFY 2.0: A Breathing Companion Robot: User-Centered Design and Findings from a Study with Ukrainian Refugees

This paper presents the design of NEFFY 2.0, a social robot designed as a haptic slow-paced breathing companion for stress reduction, and reports findings from a mixed-methods user study with 14 refugees from Ukraine. Developed through a user-centered design process, NEFFY 2.0 builds on NEFFY 1.0 and integrates embodiment and multi-sensory interaction to provide low-threshold, accessible guidance of slow-paced breathing for stress relief, which may be particularly valuable for individuals experiencing prolonged periods of anxiety. To evaluate effectiveness, an experimental comparison of a robot-assisted breathing intervention versus an audio-only condition was conducted. Measures included subjective ratings and physiological indicators, such as heart rate (HR), heart rate variability (HRV) using RMSSD parameter, respiratory rate (RR), and galvanic skin response (GSR), alongside qualitative data from interviews exploring user experience and perceived support. Qualitative findings showed that NEFFY 2.0 was perceived as intuitive, calming and supportive. Survey results showed a substantially larger effect in significant reduction of perceived stress in the NEFFY 2.0 condition compared to audio-only. Physiological data reveled mixed results combined with large inter-personal variability. Three patterns of breathing practice with NEFFY 2.0 were identified using k-means clustering. Despite the small sample size, this study makes a novel contribution by providing empirical evidence of stress reduction in a vulnerable population through a direct comparison of robot-assisted and non-robot conditions. The findings position NEFFY 2.0 as a promising low-threshold tool that supports stress relief and contributes to the vision of HRI empowering society.

Ilona Buchem, Jessica Kazubski, Charly Goerke

Berlin University of Applied Sciences Berlin Germany

Human-Robot Interaction Social Robotics Mental Health Assistive Robotics

Mar 5, 2026

Rethinking the Role of Collaborative Robots in Rehabilitation

Current research on collaborative robots (cobots) in physical rehabilitation largely focuses on repeated motion training for people undergoing physical therapy (PuPT), even though these sessions include phases that could benefit from robotic collaboration and assistance. Meanwhile, access to physical therapy remains limited for people with disabilities and chronic illnesses. Cobots could support both PuPT and therapists, and improve access to therapy, yet their broader potential remains underexplored. We propose extending the scope of cobots by imagining their role in assisting therapists and PuPT before, during, and after a therapy session. We discuss how cobot assistance may lift access barriers by promoting ability-based therapy design and helping therapists manage their time and effort. Finally, we highlight challenges to realizing these roles, including advancing user-state understanding, ensuring safety, and integrating cobots into therapists' workflow. This view opens new research questions and opportunities to draw from the HRI community's advances in assistive robotics.

Vivek Gupte, Shalutha Rajapakshe, Emmanuel Senft

Idiap Research Institute & EPFL, Switzerland · Idiap Research Institute, Switzerland

Human-Robot Collaboration Assistive Robotics Human-Robot Interaction

Mar 4, 2026

Lightweight Visual Reasoning for Socially-Aware Robots

Robots operating in shared human environments must not only navigate, interact, and detect their surroundings, they must also interpret and respond to dynamic, and often unpredictable, human behaviours. Although recent advances have shown promise in enhancing robotic perception and instruction-following using Vision-Language Models (VLMs), they remain limited in addressing the complexities of multimodal human-robot interactions (HRI). Motivated by this challenge, we introduce a lightweight language-to-vision feedback module that closes the loop between an LLM and the vision encoder in VLMs. The module projects image-token hidden states through a gated Multi-Layer Perceptron (MLP) back into the encoder input, prompting a second pass that reinterprets the scene under text context. We evaluate this approach on three robotics-centred tasks: navigation in a simulated environment (Habitat), sequential scene description (Mementos-Robotics), and human-intention recognition (our HRI dataset). Results show that our method improves Qwen 2.5 (7B) by

3.3\%

(less distance),

+0.057

description score, and

+2.93\%

accuracy, with less than

3\%

extra parameters; Gemma 3 (4B) and LLaVA OV 1.5 (4B) show mixed navigation results but gains

+0.111,+0.055

and

+10.81\%,+4.79\%

on the latter two tasks. Code is available at https://github.com/alessioGalatolo/VLM-Reasoning-for-Robotics

Alessio Galatolo, Ronald Cumbal, Alexandros Rouchitsas +3

Department of Information Technology, Uppsala University, Sweden.

VLM Reasoning VLMs for Robotics Human-Robot Interaction

Mar 4, 2026

MRPoS: Mixed Reality-Based Robot Navigation Interface Using Spatial Pointing and Speech with Large Language Model

Recent advancements have made robot navigation more intuitive by transitioning from traditional 2D displays to spatially aware Mixed Reality (MR) systems. However, current MR interfaces often rely on manual "air tap" gestures for goal placement, which can be repetitive and physically demanding, especially for beginners. This paper proposes the Mixed Reality-Based Robot Navigation Interface using Spatial Pointing and Speech (MRPoS). This novel framework replaces complex hand gestures with a natural, multimodal interface combining spatial pointing with Large Language Model (LLM)-based speech interaction. By leveraging both information, the system translates verbal intent into navigation goals visualized by MR technology. Comprehensive experiments comparing MRPoS against conventional gesture-based systems demonstrate that our approach significantly reduces task completion time and workload, providing a more accessible and efficient interface. For additional material, please check: https://mertcookimg.github.io/mrpos

Eduardo Iglesius, Masato Kobayashi, Yuki Uranishi

The University of Osaka · Kobe University

Robot Navigation Human-Robot Interaction

Mar 4, 2026

MistyPilot: An Agentic Fast-Slow Thinking LLM Framework for Misty Social Robots

With the availability of open APIs in social robots, it has become easier to customize general-purpose tools to meet users' needs. However, interpreting high-level user instructions, selecting and configuring appropriate tools, and executing them reliably remain challenging for users without programming experience. To address these challenges, we introduce MistyPilot, an agentic LLM-driven framework for autonomous tool selection, orchestration, and parameter configuration. MistyPilot comprises two core components: a Physically Interactive Agent (PIA) and a Socially Intelligent Agent (SIA). The PIA enables robust sensor-triggered and tool-driven task execution, while the SIA generates socially intelligent and emotionally aligned dialogue. MistyPilot further integrates a fast-slow thinking paradigm to capture user preferences, reduce latency, and improve task efficiency. To comprehensively evaluate MistyPilot, we contribute five benchmark datasets. Extensive experiments demonstrate the effectiveness of our framework in routing correctness, task completeness, fast-slow thinking retrieval efficiency, tool scalability,and emotion alignment. All code, datasets, and experimental videos will be made publicly available on the project webpage.

Xiao Wang, Lu Dong, Jingchen Sun +3

State University of New York at Buffalo, Buffalo, NY 14260, USA

Agentic AI for Robotics LLM Agent Orchestration Large Language Model-Based Robot Planning Robotic Tool Use Social Robotics Human-Robot Interaction

Mar 3, 2026

Robotic Grasping and Placement Controlled by EEG-Based Hybrid Visual and Motor Imagery

We present a framework that integrates EEG-based visual and motor imagery (VI/MI) with robotic control to enable real-time, intention-driven grasping and placement. Motivated by the promise of BCI-driven robotics to enhance human-robot interaction, this system bridges neural signals with physical control by deploying offline-pretrained decoders in a zero-shot manner within an online streaming pipeline. This establishes a dual-channel intent interface that translates visual intent into robotic actions, with VI identifying objects for grasping and MI determining placement poses, enabling intuitive control over both what to grasp and where to place. The system operates solely on EEG via a cue-free imagery protocol, achieving integration and online validation. Implemented on a Base robotic platform and evaluated across diverse scenarios, including occluded targets or varying participant postures, the system achieves online decoding accuracies of 40.23% (VI) and 62.59% (MI), with an end-to-end task success rate of 20.88%. These results demonstrate that high-level visual cognition can be decoded in real time and translated into executable robot commands, bridging the gap between neural signals and physical interaction, and validating the flexibility of a purely imagery-based BCI paradigm for practical human-robot collaboration.

Yichang Liu, Tianyu Wang, Ziyi Ye +4

Institute of Science and Technology for Brain-inspired Intelligence (ISTBI), Fudan University · School of Data Science, Fudan University · Shanghai Innovation Institute +3

Human-Robot Interaction Robotic Grasping Robotic Manipulation EEG-Based Visual Decoding

Mar 3, 2026

Architectural HRI: Towards a Robotic Paradigm Shift in Human-Building Interaction

Recent advances in sensing, communication, interfaces, control, and robotics are expanding Human-Building Interaction (HBI) beyond adaptive building services and facades toward the physical actuation of architectural space. In parallel, research in robotic furniture, swarm robotics, and shape-changing spaces shows that architectural elements can now be robotically augmented to move, reconfigure, and adapt space. We propose that these advances promise a paradigm shift in HBI, in which multiple building layers physically adapt in synchrony to support occupant needs and sustainability goals more holistically. Conversely, we argue that this emerging paradigm also provides an ideal case for transferring HRI knowledge to unconventional robotic morphologies, including the interpretation of the robot as multiple architectural layers or even as a building. However, this research agenda remains challenged by the temporal, spatial, and social complexity of architectural HRI, and by fragmented knowledge across HCI, environmental psychology, cognitive science, and architecture. We therefore call for interdisciplinary research that unifies the why, what, and how of robotic actuation in architectural forms.

Alex Binh Vinh Duc Nguyen

University of Antwerp Antwerp, Belgium · KU Leuven Leuven, Belgium

Human-Robot Interaction

Mar 2, 2026

Learning Therapist Policy from Therapist-Exoskeleton-Patient Interaction

Post-stroke rehabilitation is often necessary for patients to regain proper walking gait. However, the typical therapy process can be exhausting and physically demanding for therapists, potentially reducing therapy intensity, duration, and consistency over time. We propose a Patient-Therapist Force Field (PTFF) to visualize therapist responses to patient kinematics and a Synthetic Therapist (ST) machine learning model to support the therapist in dyadic robot-mediated physical interaction therapy. The first encodes patient and therapist stride kinematics into a shared low-dimensional latent manifold using a Variational Autoencoder (VAE) and models their interaction through a Gaussian Mixture Model (GMM), which learns a probabilistic vector field mapping patient latent states to therapist responses. This representation visualizes patient-therapist interaction dynamics to inform therapy strategies and robot controller design. The latter is implemented as a Long Short-Term Memory (LSTM) network trained on patient-therapist interaction data to predict therapist-applied joint torques from patient kinematics. Trained and validated using leave-one-out cross-validation across eight post-stroke patients, the model was integrated into a ROS-based exoskeleton controller to generate real-time torque assistance based on predicted therapist responses. Offline results and preliminary testing indicate the potential of their use as an alternative approach to post-stroke exoskeleton therapy. The PTFF provides understanding of the therapist's actions while the ST frees the human therapist from the exoskeleton, allowing them to continuously monitor the patient's nuanced condition.

Grayson Snyder, Lorenzo Vianello, Levi Hargrove +2

Center for Robotics and Biosystems, Northwestern University, Evanston, IL, USA. · Legs and Walking Lab, Shirley Ryan AbilityLab, Chicago, IL, USA.

Assistive Robotics Human-Robot Interaction Policy Learning Robot Imitation Learning Ankle Exoskeleton Control

Mar 2, 2026

Non-verbal Real-time Human-AI Interaction in Constrained Robotic Environments

We study the ongoing debate regarding the statistical fidelity of AI-generated data compared to human-generated data in the context of non-verbal communication using full body motion. Concretely, we ask if contemporary generative models move beyond surface mimicry to participate in the silent, but expressive dialogue of body language. We tackle this question by introducing the first framework that generates a natural non-verbal interaction between Human and AI in real-time from 2D body keypoints. Our experiments utilize four lightweight architectures which run at up to 100 FPS on an NVIDIA Orin Nano, effectively closing the perception-action loop needed for natural Human-AI interaction. We trained on 437 human video clips and demonstrated that pretraining on synthetically-generated sequences reduces motion errors significantly, without sacrificing speed. Yet, a measurable reality gap persists. When the best model is evaluated on keypoints extracted from cutting-edge text-to-video systems, such as SORA and VEO, we observe that performance drops on SORA-generated clips. However, it degrades far less on VEO, suggesting that temporal coherence, not image fidelity, drives real-world performance. Our results demonstrate that statistically distinguishable differences persist between Human and AI motion.

Dragos Costea, Alina Marcu, Cristina Lazar +1

National University of Science and Technology Politehnica of Bucharest, 313 Splaiul Independentei, Bucharest, Romania · Institute of Mathematics of the Romanian Academy, Bucharest, Romania · NORCE - Norwegian Research Center, Norway

Human-AI Interaction Human Motion Generation Human-Robot Interaction