Human-Robot Collaboration

Ask about this topic

Overview

Human-robot collaboration studies how people and robots share tasks, information, and control in settings such as manufacturing, assembly, transport, healthcare, and remote operation. Common approaches combine compliant or shared control with perception of human motion, intent, fatigue, and attention; task planning and assistance may use reinforcement learning, motion prediction, or vision-language models. Researchers evaluate these systems in physical workcells, simulated tasks, teleoperation studies, and user experiments, measuring safety and task performance alongside workload, trust, agency, and coordination. A central challenge is balancing proactive autonomy and efficiency with predictable behavior, robust safety, and meaningful human oversight, especially when methods tested in simulation must transfer to real people and workplaces.

Latest papers 172

32 papers in the last four weeks, up 129% on the four weeks before. 0.2% of all new papers.

All topics
CardsList
Apr 4, 2026

Belief Dynamics for Detecting Behavioral Shifts in Safe Collaborative Manipulation

Robots operating in shared workspaces must maintain safe coordination with other agents whose behavior may change during task execution. When a collaborating agent switches strategy mid-episode, continuing under outdated assumptions can lead to unsafe actions and increased collision risk. Reliable detection of such behavioral regime changes is therefore critical. We study regime-switch detection under controlled non-stationarity in ManiSkill shared-workspace manipulation tasks. Across ten detection methods and five random seeds, enabling detection reduces post-switch collisions by 52%. However, average performance hides significant reliability differences: under a realistic tolerance of +-3 steps, detection ranges from 86% to 30%, while under +-5 steps all methods achieve 100%. We introduce UA-TOM, a lightweight belief-tracking module that augments frozen vision-language-action (VLA) control backbones using selective state-space dynamics, causal attention, and prediction-error signals. Across five seeds and 1200 episodes, UA-TOM achieves the highest detection rate among unassisted methods (85.7% at +-3) and the lowest close-range time (4.8 steps), outperforming an Oracle (5.3 steps). Analysis shows hidden-state update magnitude increases by 17x at regime switches and decays over roughly 10 timesteps, while the discretization step converges to a near-constant value (Delta_t approx 0.78), indicating sensitivity driven by learned dynamics rather than input-dependent gating. Cross-domain experiments in Overcooked show complementary roles of causal attention and prediction-error signals. UA-TOM introduces 7.4 ms inference overhead (14.8% of a 50 ms control budget), enabling reliable regime-switch detection without modifying the base policy.

Devashri Naik, Divake Kumar, Nastaran Darabi +1

University of Illinois Chicago

RoboticsHuman-Robot Collaboration
AskPDF
Apr 3, 2026

Joint Prediction of Human Motions and Actions in Human-Robot Collaboration

Fluent human--robot collaboration requires robots to continuously estimate human behaviour and anticipate future intentions. This entails reasoning jointly about continuous movements and discrete actions, which are still largely modelled in isolation. In this paper, we introduce MA-HERP, a hierarchical and recursive probabilistic framework for the joint estimation and prediction of human movements and actions. The model combines: (i) a hierarchical representation in which movements compose into actions through admissible Allen interval relations, (ii) a unified probabilistic factorisation coupling continuous dynamics, discrete labels, and durations, and (iii) a recursive inference scheme inspired by Bayesian filtering, alternating top-down action prediction with bottom-up sensory evidence. We present a preliminary experimental evaluation based on neural models trained on musculoskeletal simulations of reaching movements, showing accurate motion prediction, robust action inference under noise, and computational performance compatible with on-line human--robot collaboration.

Alessandra Bulanti, Alessandro Carfì, Fulvio Mastrogiovanni

University of Genoa, Italy

Human Behavior PredictionHuman Motion PredictionHuman-Robot Collaboration
AskPDF
Apr 2, 2026

AURA: Multimodal Shared Autonomy for Real-World Urban Navigation

Long-horizon navigation in complex urban environments relies heavily on continuous human operation, which leads to fatigue, reduced efficiency, and safety concerns. Shared autonomy, where a Vision-Language AI agent and a human operator collaborate on maneuvering the mobile machine, presents a promising solution to address these issues. However, existing shared autonomy methods often require humans and AI to operate within the same action space, leading to high cognitive overhead. We present Assistive Urban Robot Autonomy (AURA), a new multi-modal framework that decomposes urban navigation into high-level human instruction and low-level AI control. AURA incorporates a Spatial-Aware Instruction Encoder to align various human instructions with visual and spatial context. To facilitate training, we construct MM-CoS, a large-scale dataset comprising teleoperation and vision-language descriptions. Experiments in simulation and the real world demonstrate that AURA effectively follows human instructions, reduces manual operation effort, and improves navigation stability, while enabling online adaptation. Moreover, under similar takeover conditions, our shared autonomy framework reduces the frequency of takeovers by more than 44%. Demo video and more detail are provided in the project page.

Yukai Ma, Honglin He, Selina Song +2

University of California, Los Angeles · Zhejiang University

Shared AutonomyRobot NavigationHuman-Robot CollaborationVision-Language NavigationInstruction Following
AskPDF
Mar 31, 2026

Reconfiguration of supernumerary robotic limbs for human augmentation

Wearable robots aim to seamlessly adapt to humans and their environment with personalized interactions. Existing supernumerary robotic limbs (SRLs), which enhance the physical capabilities of humans with additional extremities, have thus far been developed primarily for task-specific applications in structured industrial settings, limiting their adaptability to dynamic and unstructured environments. Here, we introduce a novel reconfigurable SRL framework grounded in a quantitative analysis of human augmentation to guide the development of more adaptable SRLs for diverse scenarios. This framework captures how SRL configuration shapes workspace extension and human-robot collaboration. We define human augmentation ratios to evaluate collaborative, visible extended, and non-visible extended workspaces, enabling systematic selection of SRL placement, morphology, and autonomy for a given task. Using these metrics, we demonstrate how quantitative augmentation analysis can guide the reconfiguration and control of SRLs to better match task requirements. We validate the proposed approach through experiments with a reconfigurable SRL composed of origami-inspired modular elements. Our results suggest that reconfigurable SRLs, informed by quantitative human augmentation analysis, offer a new perspective for providing adaptable human augmentation and assistance in everyday environments.

Mustafa Mete, Anastasia Bolotnikova, Alexander Schuessler +1

Reconfigurable Robotics Lab, École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland. · Laboratory for Analysis and Architecture of Systems (LAAS), CNRS, Toulouse, France.

RoboticsAssistive RoboticsHuman-Robot CollaborationModular Robotics
AskPDF
Mar 31, 2026

SafeDMPs: Integrating Formal Safety with DMPs for Adaptive HRI

Robots operating in human-centric environments must be both robust to disturbances and provably safe from collisions. Achieving these properties simultaneously and efficiently remains a central challenge. While Dynamic Movement Primitives (DMPs) offer inherent stability and generalization from single demonstrations, they lack formal safety guarantees. Conversely, formal methods like Control Barrier Functions (CBFs) provide provable safety but often rely on computationally expensive, real-time optimization, hindering their use in high-frequency control. This paper introduces SafeDMPs, a novel framework that resolves this trade-off. We integrate the closed-form efficiency and dynamic robustness of DMPs with a provably safe, non-optimization-based control law derived from Spatio-Temporal Tubes (STTs). This synergy allows us to generate motions that are not only robust to perturbations and adaptable to new goals, but also guaranteed to avoid static and dynamic obstacles. Our approach achieves a closed-form solution for a problem that traditionally requires online optimization. Experimental results on a 7-DOF robot manipulator demonstrate that SafeDMPs is orders of magnitude faster and more accurate than optimization-based baselines, making it an ideal solution for real-time, safe, and collaborative robotics.

Soumyodipta Nath, Pranav Tiwari, Ravi Prakash

All authors are with the Cyber Physical Systems, Indian Institute of Science, Bengaluru, 560012, India.

Robot Motion GenerationSafe Motion PlanningRobot Collision AvoidanceHuman-Robot Collaboration
AskPDF
Mar 30, 2026

EBuddy: a workflow orchestrator for industrial human-machine collaboration

This paper presents EBuddy, a voice-guided workflow orchestrator for natural human-machine collaboration in industrial environments. EBuddy targets a recurrent bottleneck in tool-intensive workflows: expert know-how is effective but difficult to scale, and execution quality degrades when procedures are reconstructed ad hoc across operators and sessions. EBuddy operationalizes expert practice as a finite state machine (FSM) driven application that provides an interpretable decision frame at runtime (current state and admissible actions), so that spoken requests are interpreted within state-grounded constraints, while the system executes and monitors the corresponding tool interactions. Through modular workflow artifacts, EBuddy coordinates heterogeneous resources, including GUI-driven software and a collaborative robot, leveraging fully voice-based interaction through automatic speech recognition and intent understanding. An industrial pilot on impeller blade inspection and repair preparation for directed energy deposition (DED), realized by human-robot collaboration, shows substantial reductions in end-to-end process duration across onboarding, 3D scanning and processing, and repair program generation, while preserving repeatability and low operator burden.

Michele Banfi, Rocco Felici, Stefano Baraldo +2

Laboratory of Automation, Robotics and Machines (ARM), University of Applied Sciences of Southern Switzerland, Lugano-Viganello, Switzerland

Workflow AutomationHuman-AI InteractionHuman-Robot CollaborationHuman-Robot InteractionIndustrial Robotics
AskPDF
Mar 30, 2026

Proposing a Game Theory Approach to Explore Group Dynamics with Social Robot

Integrating social robots in our group-based society, beyond the technical challenges, requires considering the social group dynamics. Following the results from preliminary exploratory studies on the influence of social robots on group decisions, the proposed research investigates whether social robots can foster cooperation among group members. To achieve this, I propose a game theory approach, employing the Public Good Game to recreate a simplified and controlled social situation where the robot's influence can be evaluated. Clarifying the role of robots in promoting collaboration among humans might have a significant impact in educational environments, enhancing student learning, as well as in workplace settings, where they could facilitate problem-solving and lead to shared solutions.

Giulia Pusceddu

Italian Institute of Technology, Genoa, Italy

Human-Robot InteractionSocial RoboticsHuman-Robot Collaboration
AskPDF
Mar 30, 2026

On-Demand Human Assistance for Task Continuation under Physical Action Failures in LLM-based Planning

While robot action planning based on large language models (LLMs) has advanced remarkably, continuing a task after a physical action failure during execution remains a key challenge. For example, after failing to grasp an object, the robot may not notice that it has fallen off the table and keep trying to detect and pick it up, stalling the task. Existing approaches either rely on closed-loop autonomous re-planning, which fails when errors exceed the robot's capabilities, or on human intervention without updating the LLM's plan, hindering subsequent planning. We present a system for LLM-based robot action planning that integrates remote human intervention with feedback-based replanning. When the LLM detects an action failure, the robot requests remote assistance; an operator resolves the failure through teleoperation and reports the outcome in natural language, which is fed back to the LLM to update its plan so that the task can continue. We evaluated the proposed system on a real-world trash-collection task, comparing it with a fully autonomous method and a teleoperation-only baseline. The proposed system improves task progress over the autonomous method in this case study. For easy-to-retrieve trash, the gap from the teleoperation reference was small, while a larger gap remained for harder-to-retrieve trash. These results suggest that, in this case study, combining remote human intervention with feedback-based replanning was associated with continued task execution even in the face of otherwise unrecoverable failures. The project website is https://emergentsystemlabstudent.github.io/REPAIR/.

Shoichi Hasegawa, Akira Taniguchi, Lotfi El Hafi +2

Shoichi Hasegawa and Akira Taniguchi are with Ritsumeikan University; 2-150 Iwakura, Ibaraki, Osaka 567-8570, Japan. · Akira Taniguchi is with Kyushu Institute of Technology; 680-4 Kawadu, Iizuka, Fukuoka 820-0067, Japan. · Lotfi El Hafi is with Coarobo GK; 2-2-2 Hikaridai, Seika, Soura, Kyoto 619-0237, Japan. +4

Large Language Model-Based Robot PlanningRobot Failure RecoveryRobot Task PlanningHuman-Robot Collaboration
AskPDF
Mar 25, 2026

Event-Driven Proactive Assistive Manipulation with Grounded Vision-Language Planning

Assistance in collaborative manipulation is often initiated by user instructions, making high-level reasoning request-driven. In fluent human teamwork, however, partners often infer the next helpful step from the observed outcome of an action rather than waiting for instructions. Motivated by this, we introduce a shift from request-driven assistance to event-driven proactive assistance, where robot actions are initiated by workspace state transitions induced by human--object interactions rather than user-provided task instructions. To this end, we propose an event-driven framework that tracks interaction progress with an event monitor and, upon event completion, extracts stabilized pre/post snapshots that characterize the resulting state transition. Given the stabilized snapshots, the planner analyzes the implied state transition to infer a task-level goal and decide whether to intervene; if so, it generates a sequence of assistive actions. To make outputs executable and verifiable, we restrict actions to a set of action primitives and reference objects via integer IDs. We evaluate the framework on a real tabletop number-block collaboration task, demonstrating that explicit pre/post state-change evidence improves proactive completion on solvable scenes and appropriate waiting on unsolvable ones.

Fengkai Liu, Hao Su, Haozhuang Chi +6

The University of Osaka · Nanyang Technological University · The University of Tokyo

Assistive RoboticsHuman-Robot CollaborationRobot Task Planning
AskPDF
Mar 24, 2026

DiSCo: Diffusion Sequence Copilots for Shared Autonomy

Shared autonomy combines human user and AI copilot actions to control complex systems such as robotic arms. When a task is challenging, requires high dimensional control, or is subject to corruption, shared autonomy can significantly increase task performance by using a trained copilot to effectively correct user actions in a manner consistent with the user's goals. To significantly improve the performance of shared autonomy, we introduce Diffusion Sequence Copilots (DiSCo): a method of shared autonomy with diffusion policy that plans action sequences consistent with past user actions. DiSCo seeds and inpaints the diffusion process with user-provided actions with hyperparameters to balance conformity to expert actions, alignment with user intent, and perceived responsiveness. We demonstrate that DiSCo substantially improves task performance in simulated driving and robotic arm tasks. Project website: https://sites.google.com/view/disco-shared-autonomy/

Andy Wang, Xu Yan, Brandon McMahan +9

University of California, Los Angeles, Los Angeles, USA

Shared AutonomyHuman-Robot CollaborationDiffusion-Based PlanningDiffusion Policy
AskPDF
Mar 20, 2026

Legged Autonomous Surface Science In Analogue Environments (LASSIE): Making Every Robotic Step Count in Planetary Exploration

The ability to efficiently and effectively explore planetary surfaces is currently limited by the capability of wheeled rovers to traverse challenging terrains, and by pre-programmed data acquisition plans with limited in-situ flexibility. In this paper, we present two novel approaches to address these limitations: (i) high-mobility legged robots that use direct surface interactions to collect rich information about the terrain's mechanics to guide exploration; (ii) human-inspired data acquisition algorithms that enable robots to reason about scientific hypotheses and adapt exploration priorities based on incoming ground-sensing measurements. We successfully verify our approach through lab work and field deployments in two planetary analog environments. The new capability for legged robots to measure soil mechanical properties is shown to enable effective traversal of challenging terrains. When coupled with other geologic properties (e.g., composition, thermal properties, and grain size data etc), soil mechanical measurements reveal key factors governing the formation and development of geologic environments. We then demonstrate how human-inspired algorithms turn terrain-sensing robots into teammates, by supporting more flexible and adaptive data collection decisions with human scientists. Our approach therefore enables exploration of a wider range of planetary environments and new substrate investigation opportunities through integrated human-robot systems that support maximum scientific return.

Cristina G. Wilson, Marion Nachon, Shipeng Liu +20

Collaborative Robotics and Intelligent Systems Institute, Oregon State University, Corvallis, 97331, USA · Department of Geology and Geophysics, Texas A&M University, College Station, 77843, USA · Department of Electrical and Computer Engineering, University of Southern California, Los Angeles, 90089, USA +7

Robotic Data CollectionLegged RoboticsTerrain-Aware Robot LocomotionHuman-Robot Collaboration
AskPDF
Mar 18, 2026

HRI-SA: A Multimodal Dataset for Online Assessment of Human Situational Awareness during Remote Human-Robot Teaming

Maintaining situational awareness (SA) is critical in human-robot teams. Yet, under high workload and dynamic conditions, operators often experience SA gaps. Automated detection of SA gaps could provide timely assistance for operators. However, conventional SA measures either disrupt task flow or cannot capture real-time fluctuations, limiting their operational utility. To the best of our knowledge, no publicly available dataset currently supports the systematic evaluation of online human SA assessment in human-robot teaming. To advance the development of online SA assessment tools, we introduce HRI-SA, a multimodal dataset from 30 participants in a realistic search-and-rescue human-robot teaming context, incorporating eye movements, pupil diameter, biosignals, user interactions, and robot data. The experimental protocol included predefined events requiring timely operator assistance, with ground truth SA latency of two types (perceptual and comprehension) systematically obtained by measuring the time between assistance need onset and resolution. We illustrate the utility of this dataset by evaluating standard machine learning models for detecting perceptual SA latencies using generic eye-tracking features and contextual features. Results show that eye-tracking features alone effectively classified perceptual SA latency (recall=88.91%, F1=67.63%) using leave-one-group-out cross-validation, with performance improved through contextual data fusion (recall=91.51%, F1=80.38%). This paper contributes the first public dataset supporting the systematic evaluation of SA throughout a human-robot teaming mission, while also demonstrating the potential of generic eye-tracking features for continuous perceptual SA latency detection in remote human-robot teaming.

Hashini Senaratne, Richard Attfield, Samith Widhanapathirana +4

CSIRO Robotics: www.csiro.au/en/research/technology-space/robotics · Monash Robotics: www.monash.edu/engineering/robotics

Human-Robot Collaboration
AskPDF
Mar 17, 2026

Encoding Predictability and Legibility for Style-Conditioned Diffusion Policy

Striking a balance between efficiency and transparent motion is a core challenge in human-robot collaboration, as highly expressive movements often incur unnecessary time and energy costs. In collaborative environments, legibility allows a human observer a better understanding of the robot's actions, increasing safety and trust. However, these behaviors result in sub-optimal and exaggerated trajectories that are redundant in low-ambiguity scenarios where the robot's goal is already obvious. To address this trade-off, we propose Style-Conditioned Diffusion Policy (SCDP), a modular framework that constrains the trajectory generation of a pre-trained diffusion model toward either legibility or efficiency based on the environment's configuration. Our method utilizes a post-training pipeline that freezes the base policy and trains a lightweight scene encoder and conditioning predictor to modulate the diffusion process. At inference time, an ambiguity detection module activates the appropriate conditioning, prioritizing expressive motion only for ambiguous goals and reverting to efficient paths otherwise. We evaluate SCDP on manipulation and navigation tasks, and results show that it enhances legibility in ambiguous settings while preserving optimal efficiency when legibility is unnecessary, all without retraining the base policy.

Adrien Jacquet Crétides, Mouad Abrini, Hamed Rahimi +1

Institut des Systèmes Intelligents et de Robotique (ISIR), CNRS UMR7222, Inserm ERL U1150, Sorbonne Université, Paris, France

Human-Robot CollaborationDiffusion PolicyRobot Policy AdaptationRobotic Manipulation
AskPDF
Mar 16, 2026

BodyGuards: Escorting by Multiple Robots in Unknown Environment under Limited Communication

Multi-robot systems are increasingly deployed in high-risk missions such as reconnaissance, disaster response, and subterranean operations. Protecting a human operator while navigating unknown and adversarial environments remains a critical challenge, especially when the communication among the operator and robots is restricted. Unlike existing collaborative exploration methods that aim for complete coverage, this work focuses on task-oriented exploration to minimize the navigation time of the operator to reach its goal while ensuring safety under adversarial threats. A novel escorting framework BodyGuards, is proposed to explicitly integrate seamlessly collaborative exploration, inter-robot-operator communication and escorting. The framework consists of three core components: (I) a dynamic movement strategy for the operator that maintains a local map with risk zones for proactive path planning; (II) a dual-mode robotic strategy combining frontier based exploration with optimized return events to balance exploration, threat detection, and intermittent communication; and (III) multi-robot coordination protocols that jointly plan exploration and information sharing for efficient escorting. Extensive human-in-the-loop simulations and hardware experiments demonstrate that the method significantly reduces operator risk and mission time, outperforming baselines in adversarial and constrained environments.

Zhuoli Tian, Yanze Bao, Meng Guo

Human-Robot CollaborationRobot SafetyMulti-Robot CoordinationMulti-Robot Exploration
AskPDF
Mar 15, 2026

Navigation beyond Wayfinding: Robots Collaborating with Visually Impaired Users for Environmental Interactions

Robotic guidance systems have shown promise in supporting blind and visually impaired (BVI) individuals with wayfinding and obstacle avoidance. However, most existing systems assume a clear path and do not support a critical aspect of navigation - environmental interactions that require manipulating objects to enable movement. These interactions are challenging for a human-robot pair because they demand (i) precise localization and manipulation of interaction targets (e.g., pressing elevator buttons) and (ii) dynamic coordination between the user's and robot's movements (e.g., pulling out a chair to sit). We present a collaborative human-robot approach that combines our robotic guide dog's precise sensing and localization capabilities with the user's ability to perform physical manipulation. The system alternates between two modes: lead mode, where the robot detects and guides the user to the target, and adaptation mode, where the robot adjusts its motion as the user interacts with the environment (e.g., opening a door). Evaluation results show that our system enables navigation that is safer, smoother, and more efficient than both a traditional white cane and a non-adaptive guiding system, with the performance gap widening as tasks demand higher precision in locating interaction targets. These findings highlight the promise of human-robot collaboration in advancing assistive technologies toward more generalizable and realistic navigation support.

Shaojun Cai, Nuwan Janaka, Ashwin Ram +4

National University of Singapore Singapore Singapore · City University of Hong Kong Hong Kong China · Saarland University Saarbrücken Germany +3

Assistive RoboticsRobot NavigationHuman-Robot Collaboration
AskPDF
Mar 13, 2026

Perceive What Matters: Relevance-Driven Scheduling for Multimodal Streaming Perception

In modern human-robot collaboration (HRC) applications, multiple perception modules jointly extract visual, auditory, and contextual cues to achieve comprehensive scene understanding, enabling the robot to provide appropriate assistance to human agents intelligently. While executing multiple perception modules on a frame-by-frame basis enhances perception quality in offline settings, it inevitably accumulates latency, leading to a substantial decline in system performance in streaming perception scenarios. Recent work in scene understanding, termed Relevance, has established a solid foundation for developing efficient methodologies in HRC. However, modern perception pipelines still face challenges related to information redundancy and suboptimal allocation of computational resources. Drawing inspiration from the Relevance concept and the information sparsity in HRC events, we propose a novel lightweight perception scheduling framework that efficiently leverages output from previous frames to estimate and schedule necessary perception modules in real-time based on scene context. The experimental results demonstrate that the proposed perception scheduling framework effectively reduces computational latency by up to 27.52% compared to conventional parallel perception pipelines, while also achieving a 72.73% improvement in MMPose activation recall. Additionally, the framework demonstrates high keyframe accuracy, achieving rates of up to 98%. The results validate the framework's capability to enhance real-time perception efficiency without significantly compromising accuracy. The framework shows potential as a scalable and systematic solution for multimodal streaming perception systems in HRC.

Dingcheng Huang, Xiaotong Zhang, Kamal Youcef-Toumi

Mechatronics Research Laboratory, Massachusetts Institute of Technology, Cambridge, MA, 02139, USA

Robotic PerceptionHuman-Robot Collaboration
AskPDF
Mar 12, 2026

CoViLLM: An Adaptive Human-Robot Collaborative Assembly Framework Using Large Language Models

With increasing demand for mass customization, traditional manufacturing robots that rely on rule-based operations lack the flexibility to accommodate customized or new product variants. Human-Robot Collaboration has demonstrated potential to improve system adaptability by leveraging human versatility and decision-making capabilities. However, existing Human-Robot Collaborative frameworks typically depend on predefined perception-manipulation pipelines, limiting their ability to autonomously generate task plans for new product assembly. In this work, we propose CoViLLM, an adaptive human-robot collaborative assembly framework that supports the assembly of customized and previously unseen products. CoViLLM combines depth-camera-based localization for object position estimation, human operator classification for identifying new components, and a Large Language Model for assembly task planning based on natural language instructions. The framework is validated on the NIST Assembly Task Board for known, customized, and new product cases. Experimental results show that the proposed framework enables flexible collaborative assembly by extending Human-Robot Collaboration beyond predefined product and task settings.

Jiabao Zhao, Jonghan Lim, Hongliang Li +1

The Pennsylvania State University, University Park, PA, USA

Human-Robot CollaborationRobotic AssemblyGeneralization in Robotic ManipulationLarge Language Model-Based Robot Planning
AskPDF
Mar 11, 2026

SUBTA: A Framework for Supported User-Guided Bimanual Teleoperation in Structured Assembly

In human-robot collaboration, shared autonomy enhances human performance through precise, intuitive support. Effective robotic assistance requires accurately inferring human intentions and understanding task structures to determine optimal support timing and methods. In this paper, we present SUBTA, a supported teleoperation system for bimanual assembly that couples learned intention estimation, scene-graph task planning, and context-dependent motion assists. We validate our approach through a user study (N=12) comparing standard teleoperation, motion-support only, and SUBTA. Linear mixed-effects analysis revealed that SUBTA significantly outperformed standard teleoperation in position accuracy (p<0.001, d=1.18) and orientation accuracy (p<0.001, d=1.75), while reducing mental demand (p=0.002, d=1.34). Post-experiment ratings indicate clearer, more trustworthy visual feedback and predictable interventions in SUBTA. The results demonstrate that SUBTA greatly improves both effectiveness and user experience in teleoperation.

Xiao Liu, Prakash Baskaran, Songpo Li +4

Authors are with Honda Research Institute USA, San Jose, CA 95134 USA. · Authors are with Honda Research Institute Europe GmbH, 63073 Offenbach am Main, Germany.

Robot TeleoperationHuman-Robot CollaborationShared AutonomyBimanual Robotic ManipulationRobotic Assembly
AskPDF
Mar 10, 2026

TATIC: Task-Aware Temporal Learning for Human Intent Inference from Physical Corrections in Human-Robot Collaboration

In human-robot collaboration (HRC), robots must adapt online to dynamic task constraints and evolving human intent. While physical corrections provide a natural, low-latency channel for operators to convey motion-level adjustments, extracting task-level semantic intent from such brief interactions remains challenging. Existing foundation-model-based approaches primarily rely on vision and language inputs and lack mechanisms to interpret physical feedback. Meanwhile, traditional physical human-robot interaction (pHRI) methods leverage physical corrections for trajectory guidance but struggle to infer task-level semantics. To bridge this gap, we propose TATIC, a unified framework that utilizes torque-based contact force estimation and a task-aware Temporal Convolutional Network (TCN) to jointly infer discrete task-level intent and estimate continuous motion-level parameters from brief physical corrections. Task-aligned feature canonicalization ensures robust generalization across diverse layouts, while an intent-driven adaptation scheme translates inferred human intent into robot motion adaptations. Experiments achieve a 0.904 Macro-F1 score in intent recognition and demonstrate successful hardware validation in collaborative disassembly (see experimental video at https://youtu.be/xF8A52qwEc8).

Jiurun Song, Xiao Liang, Minghui Zheng

Department of Mechanical Engineering, Texas A&M University, College Station, TX 77843, USA · Department of Civil and Environmental Engineering, Texas A&M University, College Station, TX 77843, USA

Human-Robot CollaborationHuman-Robot Interaction
AskPDF
Mar 10, 2026

Dance2Hesitate: A Multi-Modal Dataset of Dancer-Taught Hesitancy for Understandable Robot Motion

In human-robot collaboration, a robot's expression of hesitancy is a critical factor that shapes human coordination strategies, attention allocation, and safety-related judgments. However, designing hesitant robot motion that generalizes is challenging because the observer's inference is highly dependent on embodiment and context. To address these challenges, we introduce and open-source a multi-modal, dancer-generated dataset of hesitant motion where we focus on specific context-embodiment pairs (i.e., manipulator/human upper-limb approaching a Jenga Tower, and anthropomorphic whole body motion in free space). The dataset includes (i) kinesthetic teaching demonstrations on a Franka Emika Panda reaching from a fixed start configuration to a fixed target (a Jenga tower) with three graded hesitancy levels (slight, significant, extreme) and (ii) synchronized RGB-D motion capture of dancers performing the same reaching behavior using their upper limb across three hesitancy levels, plus full human body sequences for extreme hesitancy. We further provide documentation to enable reproducible benchmarking across robot and human modalities. Across all dancers, we obtained 70 unique whole-body trajectories, 84 upper limb trajectories spanning over the three hesitancy levels, and 66 kinesthetic teaching trajectories spanning over the three hesitancy levels. The dataset can be accessed here: https://brsrikrishna.github.io/Dance2Hesitate/.

Srikrishna Bangalore Raghu, Anna Soukhovei, Divya Sai Sindhuja Vankineni +2

University of Colorado Boulder Boulder, Colorado, United States · Pacific Lutheran University Tacoma, Washington, United States

Human-Robot CollaborationRobot Motion Generation
AskPDF
Mar 9, 2026

Human-Aware Robot Behaviour in Self-Driving Labs

Self-driving laboratories (SDLs) are rapidly transforming research in chemistry and materials science to accelerate new discoveries. Mobile robot chemists (MRCs) play a pivotal role by autonomously navigating the lab to transport samples, effectively connecting synthesis, analysis, and characterisation equipment. The instruments within an SDL are typically designed or retrofitted to be accessed by both human and robotic chemists, ensuring operational flexibility and integration between manual and automated workflows. In many scenarios, human and robotic chemists may need to use the same equipment simultaneously. Currently, MRCs rely on simple LiDAR-based obstruction detection, which forces the robot to passively wait if a human is present. This lack of situational awareness leads to unnecessary delays and inefficient coordination in time-critical automated workflows in human-robot shared labs. To address this, we present an initial study of an embodied, AI-driven perception method that facilitates proactive human-robot interaction in shared-access scenarios. Our method features a hierarchical human intention prediction model that allows the robot to distinguish between preparatory actions (waiting) and transient interactions (accessing the instrument). Our results demonstrate that the proposed approach enhances efficiency by enabling proactive human-robot interaction, streamlining coordination, and potentially increasing the efficiency of autonomous scientific labs.

Satheeshkumar Veeramani, Anna Kisil, Abigail Bentley +3

Department of Chemistry University of Liverpool United Kingdom · School of Computer Science and Informatics University of Liverpool United Kingdom · School of Computer Science and Informatics Department of Chemistry University of Liverpool United Kingdom

Human-Robot CollaborationHuman-Robot InteractionLaboratory Automation
AskPDF
Mar 9, 2026

Adaptive Vision-Based Control of Redundant Robots with Null-Space Interaction for Human-Robot Collaboration

Human-robot collaboration aims to extend human ability through cooperation with robots. This technology is currently helping people with physical disabilities, has transformed the manufacturing process of companies, improved surgical performance, and will likely revolutionize the daily lives of everyone in the future. Being able to enhance the performance of both sides, such that human-robot collaboration outperforms a single robot/human, remains an open issue. For safer and more effective collaboration, a new control scheme has been proposed for redundant robots in this paper, consisting of an adaptive vision-based control term in task space and an interactive control term in null space. Such a formulation allows the robot to autonomously carry out tasks in an unknown environment without prior calibration while also interacting with humans to deal with unforeseen changes (e.g., potential collision, temporary needs) under the redundant configuration. The decoupling between task space and null space helps to explore the collaboration safely and effectively without affecting the main task of the robot end-effector. The stability of the closed-loop system has been rigorously proved with Lyapunov methods, and both the convergence of the position error in task space and that of the damping model in null space are guaranteed. The experimental results of a robot manipulator guided with the technology of augmented reality (AR) are presented to illustrate the performance of the control scheme.

Xiangjie Yan, Chen Chen, Xiang Li

Department of Automation, Tsinghua University

Robotic ControlHuman-Robot CollaborationVision-Based Robot Control
AskPDF
Mar 8, 2026

Reasoning Knowledge-Gap in Drone Planning via LLM-based Active Elicitation

Human-AI joint planning in Unmanned Aerial Vehicles (UAVs) typically relies on control handover when facing environmental uncertainties, which is often inefficient and cognitively demanding for non-expert operators. To address this, we propose a novel framework that shifts the collaboration paradigm from control takeover to active information elicitation. We introduce the Minimal Information Neuro-Symbolic Tree (MINT), a reasoning mechanism that explicitly structures knowledge gaps regarding obstacles and goals into a queryable format. By leveraging large language models, our system formulates optimal binary queries to resolve specific ambiguities with minimal human interaction. We demonstrate the efficacy of this approach through a comprehensive workflow integrating a vision-language model for perception, voice interfaces, and a low-level UAV control module in both high-fidelity NVIDIA Isaac simulations and real-world deployments. Experimental results show that our method achieves a significant improvement in the success rate for complex search-and-rescue tasks while significantly reducing the frequency of human interaction compared to exhaustive querying baselines.

Zeyu Fang, Beomyeol Yu, Cheng Liu +5

George Washington University, Washington, D.C., USA · Northeastern University, Boston, MA, USA

Human-Robot CollaborationLarge Language Model-Based Robot PlanningUAV Navigation
AskPDF
Mar 8, 2026

Knowing When to Ask: Resolving Uncertainty in Human-Robot Joint Planning via Explicit Dialogue and Implicit Intent Cues

Effective human-robot collaboration in open-world environments requires joint planning under uncertainty about the task, the environment, and the human teammate. Communication is the most direct means of resolving such uncertainty, yet most existing systems support only one-way communication: robots listen and act, treating humans as passive supervisors rather than conversational teammates capable of two-way dialogue. We propose a unified human-robot joint planning system in which the robot actively resolves uncertainty through two complementary communication channels. When uncertainty is decision-critical, an uncertainty-mitigation joint planning module engages the human in clarification dialogue: it grounds ambiguous instructions via an LLM-assisted active elicitation mechanism, enumerates traversability hypotheses through a hypothesis-augmented A* search, and computes a cost-optimal querying policy via dynamic programming, so that the robot asks only the questions whose answers actually matter for the plan. When explicit dialogue is unnecessary or impractical, a real-time intent-aware collaboration module instead reads implicit, nonverbal cues, maintaining a probabilistic belief over the human's latent task intent from spatial and directional signals to enable coordination-aware task selection without any communication overhead. We validate the proposed system in both Gazebo simulations and real-world UAV deployments, integrated with a voice dialogue interface and a Vision-Language Model (VLM)-based 3D semantic perception pipeline. Experimental results show that cost-optimal clarification dialogue cuts the interaction cost by 51.9% while maintaining a 100% task success rate, and implicit intent reading reduces the cooperative task execution time by 25.4% compared to the baselines.

Zeyu Fang, Yuxin Lin, Cheng Liu +6

Department of Electrical and Computer Engineering, George Washington University · the Department of Electrical and Computer Engineering, Northeastern University · the Department of Mechanical and Aerospace Engineering, George Washington University

Human-Robot CollaborationRobot Task PlanningHuman-Robot Interaction
AskPDF
Mar 8, 2026

Directing the Robot: Scaffolding Creative Human-AI-Robot Interaction

Robots are moving beyond industrial settings into creative, educational, and public environments where interaction is open-ended and improvisational. Yet much of human-AI-robot interaction remains framed around performance and efficiency, positioning humans as supervisors rather than collaborators. We propose a re-framing of AI interaction with robots as scaffolding: infrastructure that enables humans to shape robotic behaviour over time while remaining meaningfully in control. Through scenarios from creative practice, learning-by-teaching, and embodied interaction, we illustrate how humans can act as executive directors, defining intent and steering revisions, while AI mediates between human expression and robotic execution. We outline design and evaluation implications that foreground creativity, agency, and flow. Finally, we discuss open challenges in social, scalable, and mission-critical contexts. We invite the community to rethink interacting with Robots and AI not as autonomy, but as sustained support for human creativity.

Jordan Aiko Deja, Isidro Butaslac, Nicko Reginio Caluya +1

De La Salle University Manila, Philippines · Nara Institute of Science and Technology Ikoma, Japan · Ritsumeikan University Osaka, Japan +1

Human-Robot InteractionHuman-Robot CollaborationHuman-AI Co-CreationHuman-Centered AI
AskPDF
Mar 6, 2026

Glance-Say: Multimodal Human-Robot Collaboration and Intent Recognition via Sticky Glance

Gaze and speech are promising interaction modalities for individuals with motor impairments, yet robust intent recognition in multi-object environments remains challenging due to micro-saccades, semantic ambiguity, and viewpoint changes. This paper presents a multimodal interaction framework for assistive robotic manipulation. We propose a sticky-glance algorithm that stabilizes gaze-based intent by jointly accumulating geometric distance and directional evidence, enabling robust real-time target selection and switching. We further introduce Glance-Say, a gaze-speech interaction paradigm in which gaze specifies objects and speech specifies actions, together with a continuous shared-control scheme that provides high-readiness robot motion and human-in-the-loop feedback. Experiments demonstrate a tracking rate of 0.92 for moving targets, selection accuracy of 0.97 for static targets, and reduced task duration. These results indicate improved robustness, efficiency, and usability over representative interaction paradigms.

Yuzhi Lai, Shenghai Yuan, Peizheng Li +2

University of Tuebingen, Geschwister-Scholl-Platz, 72074 Germany · Nanyang Technological University, 50 Nanyang Avenue, Singapore 639798

Robotic ControlHuman-AI InteractionHuman-Robot CollaborationAssistive Robotics
AskPDF
Mar 5, 2026

Rethinking the Role of Collaborative Robots in Rehabilitation

Current research on collaborative robots (cobots) in physical rehabilitation largely focuses on repeated motion training for people undergoing physical therapy (PuPT), even though these sessions include phases that could benefit from robotic collaboration and assistance. Meanwhile, access to physical therapy remains limited for people with disabilities and chronic illnesses. Cobots could support both PuPT and therapists, and improve access to therapy, yet their broader potential remains underexplored. We propose extending the scope of cobots by imagining their role in assisting therapists and PuPT before, during, and after a therapy session. We discuss how cobot assistance may lift access barriers by promoting ability-based therapy design and helping therapists manage their time and effort. Finally, we highlight challenges to realizing these roles, including advancing user-state understanding, ensuring safety, and integrating cobots into therapists' workflow. This view opens new research questions and opportunities to draw from the HRI community's advances in assistive robotics.

Vivek Gupte, Shalutha Rajapakshe, Emmanuel Senft

Idiap Research Institute & EPFL, Switzerland · Idiap Research Institute, Switzerland

Human-Robot CollaborationAssistive RoboticsHuman-Robot Interaction
AskPDF
Mar 4, 2026

Cognition to Control - Multi-Agent Learning for Human-Humanoid Collaborative Transport

Full-stack human-robot collaboration (HRC) can become brittle when replacing a planner, partner model, coordination policy, or controller changes the physical meaning of cross-layer signals. We introduce C2C, an object-centric cognition-to-control architecture that preserves these meanings through physical contracts. Rather than standardizing individual modules, C2C standardizes the physical semantics exchanged between them: task intent is represented by a geometrically verified payload path, partner information by a source-independent physical state, learned coordination by bounded 11-D task-space commands, and robot-specific feasibility remains inside whole-body control (WBC). The reference system combines a vision-language model with deterministic geometric verification and multi-agent reinforcement learning (MARL) for partner-aware coordination. Across nine transport scenarios, adaptive MARL variants achieve 77.1-82.1% mean success under unchanged interfaces, compared with 56.5% for a scripted-partner reference. The same contracts further support neural and interpretable hard-tree actors, two- and three-carrier teams, and simulated-to-human partner substitution. Physical Unitree G1-human tests achieve 100% success in spatially confined transport and 80% in super-long-object handling, while a three-carrier system with two G1 humanoids and one human validates the same interface structure on hardware. Together, these results show that C2C converts a tightly coupled HRC stack into a plug-compatible system in which key components can change without redefining the physical collaboration task.

Hao Zhang, Yisen Li, Ruize Geng +4

University of Texas at Arlington. · Carnegie Mellon University

Human-Robot CollaborationMulti-Agent CollaborationHumanoid RoboticsCooperative MARL
AskPDF
Mar 4, 2026

HALO: Learning Human-Robot Collaboration via Heterogeneous-Agent Lyapunov Policy Optimization

To improve generalization and resilience in human-robot collaboration (HRC), robots must contend with diverse combinations of human behaviors and contexts, motivating multi-agent reinforcement learning (MARL). However, inherent heterogeneity between robots and humans creates a rationality gap (RG), where decentralized policy updates deviate from cooperative joint optimization. The resulting learning problem is a general-sum differentiable game, so independent policy-gradient updates can oscillate or diverge without added structure. We propose heterogeneous-agent Lyapunov policy optimization (HALO), a framework that stabilizes decentralized MARL by enforcing Lyapunov-based contraction in policy-parameter space. Unlike Lyapunov-based safe RL, which targets state/trajectory constraints in constrained Markov decision processes, HALO uses Lyapunov certification to stabilize decentralized policy learning. HALO rectifies decentralized gradients via optimal quadratic projections, ensuring monotonic contraction of RG and enabling effective exploration of open-ended interaction spaces. Extensive simulations and real-world humanoid-robot experiments show that this certified stability improves generalization and robustness in collaborative corner cases. Our project website is available at https://HaoZhang-THU.github.io/HALO/.

Hao Zhang, Yaru Niu, Yikai Wang +2

ETAIC Lab, Department of Electrical Engineering, University of Texas at Arlington, USA · Safe AI Lab, Department of Mechanical Engineering, Carnegie Mellon University, USA.

Decentralized MARLHeterogeneous MARLPolicy OptimizationHuman-Robot CollaborationRobot Policy Generalization
AskPDF
Mar 4, 2026

UrbanHuRo: A Two-Layer Human-Robot Collaboration Framework for the Joint Optimization of Heterogeneous Urban Services

In the vision of smart cities, technologies are being developed to enhance the efficiency of urban services and improve residents' quality of life. However, most existing research focuses on optimizing individual services in isolation, without adequately considering reciprocal interactions among heterogeneous urban services that could yield higher efficiency and improved resource utilization. For example, human couriers could collect traffic and air quality data along their delivery routes, while sensing robots could assist with on-demand delivery during peak hours, enhancing both sensing coverage and delivery efficiency. However, the joint optimization of different urban services is challenging due to potentially conflicting objectives and the need for real-time coordination in dynamic environments. In this paper, we propose UrbanHuRo, a two-layer human-robot collaboration framework for joint optimization of heterogeneous urban services, demonstrated through crowdsourced delivery and urban sensing. UrbanHuRo includes two key designs: (i) a scalable distributed MapReduce-based K-submodular maximization module for efficient order dispatch, and (ii) a deep submodular reward reinforcement learning algorithm for sensing route planning. Experimental evaluations on real-world datasets from a food delivery platform demonstrate that UrbanHuRo improves sensing coverage by 29.7% and courier income by 39.2% on average in most settings, while also significantly reducing the number of overdue orders.

Tonmoy Dey, Lin Jiang, Zheng Dong +1

Department of Computer Science, Florida State University. · Department of Computer Science, Wayne State University.

Human-Robot CollaborationRL for Robotics
AskPDF