cs.ROSep 4, 2026

Pack It My Way: Triadic Human-Robot Collaboration for Personalized Autonomous Packing

Authors: Sandeep Chowdary Kotapati, Yanxin Gao, Tsung-Chi Lin

Organizations: Department of Computer Science, New Jersey Institute of Technology, Newark 07102, NJ, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Personalized autonomous packing requires robots to account for resident preferences that cannot be inferred from scene geometry alone. Expert teleoperators can interpret these preferences and translate them into feasible robot actions, but continuous expert involvement limits scalable deployment. In this paper, we investigate triadic human-robot collaboration among a resident, a correction mediator, and a robot by comparing human-expert and voice-agent mediation. We evaluate the two conditions in a user study across Protection, Compactness, andGrouping tasks, using a Show-Correct-Generalize process to assess preference correction and subsequent generalization after the surrounding objects are rearranged. Results show that voice-agent mediation achieves outcomes comparable to human-expert mediation in two of the three preference categories, despite receiving shorter and less detailed instructions. Both mediators are similarly easy to use, although the human expert is perceived as more reliable. These findings demonstrate the potential of voice agents to reduce expert involvement while identifying perceived reliability and preference generalization as remaining challenges.

Figures & tables

Fig. 1: Overview of triadic human-robot collaboration for personalized autonomous packing. A co-present resident serves as the preference owner, a human expert or voice agent serves as the correction mediator, and an autonomous robotic arm serves as the embodied executor. The insets illustrate human-expert mediation, in which the resident provides gesture-based intervention and the remote expert teleoperates the robot.
Fig. 2: Camera views and graphical interfaces supporting human-expert teleoperation and autonomous robot execution.
Fig. 3: Show–Correct–Generalize framework and the two correction-mediation pathways. Top: The robot first demonstrates its basic autonomous packing behavior (Execution 1), receives a resident-specific placement correction under the same object configuration (Execution 2), and generalizes the learned relational preference after the reference objects are rearranged (Execution 3). Bottom: Both pathways share autonomous object pickup and transport. During preference-based positioning, the resident provides correction through either (1) a remote human expert who teleoperates the robot following an open-palm intervention or (2) a voice agent that converts a spoken instruction into a placement target. Representative correction inputs are shown at the bottom.
Preference CategorySupported InstructionExample Voice Command(s)
Safety and constraintBasic: Relative safety constraint. Parameterized: Reference object or property with a direction and/or distance.“Between the two objects”; “3 cm from the fragile object”; “5 cm to the right of Object 2.”
Spatial organizationBasic: Qualitative packing density. Parameterized: Reference object with a direction and distance.“Pack them tightly”; “Pack them loosely”; “5 cm to the left of Object 1.”
SemanticBasic: Grouping relation with a reference object. Parameterized: Reference object with a direction and distance.“Group it with Object 2”; “3 cm to the right of Object 1.”
TABLE I: Supported voice-command structures and examples across the three resident preference categories. Basic commands express a qualitative relation, whereas parameterized commands add a reference object, direction, and/or metric distance.
Fig. 4: Object configurations across the three executions for the protection, compactness, and grouping tasks. Executions 1 and 2 use the same reference-object configuration (top). Before Execution 3 (bottom), reference Objects 1 and 2 are rearranged to create a previously unseen configuration for evaluating preference generalization. Object 3 is the target object in all executions. The red arrow indicates the front direction.
Fig. 5: Correction communication, packing quality, and overall mediator preferences. The first three panels show Correct (E2) results across packing tasks: word count, instruction complexity, and packing quality. The final panel summarizes overall preferences for ease of expression and reliability. n.s. denotes a nonsignificant difference; comparable and n.c. indicate that equivalence was demonstrated or not demonstrated, respectively.
Fig. 6: Packing-quality scores compare Correct (E2) and Generalize (E3) within each mediator and task, while the right panel compares satisfaction between mediators after Generalize . Higher scores indicate better outcomes; n.s. denotes a nonsignificant difference.
More in this topic

Human-Robot Collaboration

Human-robot collaboration studies how people and robots coordinate tasks safely, effectively, and with appropriate human control.

See all 181 papers

Explore similar work

CardsList

  1. Exploring Human-Robot Collaboration: Analysis of Interaction Modalities in Challenging Tasks

    May 13, 2026Simone Arreghini, Cristina Iani, Alessandro Giusti +3Proactive AssistanceHuman-Robot Collaboration

    Dalle Molle Institute for Artificial Intelligence (IDSIA), USI-SUPSI, Lugano, Switzerland · Department of Surgery, Medicine, Dentistry and Morphological Sciences with interest in Transplant, Oncology and Regenerative Medicine, University of Modena and Reggio Emilia, Modena, Italy. · Department of Sciences and Methods for Engineering (DISMI), University of Modena and Reggio Emilia, Reggio Emilia, Italy.

  2. When the Inference Meets the Explicitness or Why Multimodality Can Make Us Forget About the Perfect Predictor

    Feb 21, 2026J. E. Domínguez-Vidal, Alberto SanfeliuHuman-Robot InteractionHuman-Robot Collaboration

    Institut de Rob`otica i Inform`atica Industrial (CSIC-UPC), Llorens i Artigas 4-6, Barcelona, 08028, Spain. · Universitat Polit`ecnica de Catalunya - BarcelonaTech (UPC), Jordi Girona, 31, Barcelona, 08034, Spain.

  3. Knowing When to Ask: Resolving Uncertainty in Human-Robot Joint Planning via Explicit Dialogue and Implicit Intent Cues

    Mar 8, 2026Zeyu Fang, Yuxin Lin, Cheng Liu +6Human-Robot CollaborationRobot Task Planning

    Department of Electrical and Computer Engineering, George Washington University · the Department of Electrical and Computer Engineering, Northeastern University · the Department of Mechanical and Aerospace Engineering, George Washington University