In shared settings, social robots may need to exchange information with an individual while limiting co-present others' access to or interpretation of that information. Publicly observable speech and gestures may not adequately support such exchanges. We investigate a wearable-mediated channel for discreet interaction that combines in-ear audio messages with deliberate rightward head tilts as responses. To examine its feasibility and participants' experiences, we studied 20 pairs (40 participants) playing a card game with a screen-based conversational agent representing a social robot, controlled by a researcher in a Wizard-of-Oz setup. Each pair played with and without the channel. One participant wore the device (the wearer), while the other (the non-wearer) was informed about the channel after gameplay. The channel conveyed information that could help the wearer or robot character select an opponent they could defeat. The information exchange and subsequent opponent selection occurred as intended in 39 of 40 channel-mediated turns. Wearers reported low perceived pressure/tension in both conditions, with no statistically significant difference between conditions, and favorable usability immediately after use. Retrospective interviews, however, revealed varied accounts of noticing the interaction among non-wearers and contrasting perceptions of teamwork and unfairness. These findings support the feasibility of wearable-mediated exchanges within this game and identify awareness and fairness as considerations for future designs.
Figures & tables
Figure 1. Two hypothetical scenarios illustrating discreet interaction. On the left, a social robot suggests ordering pizza to a mother via in-ear audio instead of raising the idea openly in front of the child. On the right, the robot uses the wearable-mediated channel to request permission before making a potentially embarrassing suggestion to an older adult. Both scenes illustrate a bidirectional exchange embedded within an ongoing shared activity. Generated with ChatGPT 6. Two illustrated hypothetical domestic applications of discreet interaction with social robots. On the left, a robot sends a suggestion to a mother while she cooks with a child. On the right, a robot offers assistance during a conversation between an older adult and a younger visitor. In both scenes, in-ear audio and a deliberate rightward head-tilt response illustrate a bidirectional exchange embedded within an ongoing shared activity.
Figure 2. Study setup for the three-player card game: (a) schematic showing the wearer and non-wearer seated across a table, with the screen-based robot character acting as both host and fellow player. In the discreet condition, the wearer used an in-ear wearable to receive in-ear audio and perform a deliberate rightward head-tilt response. The non-wearer remained an active participant and was not informed about the wearable-mediated channel until after gameplay; (b) the robot character displayed on the screen; (c) the cardholder camera feed showing the robot character’s card; and (d) the overhead camera feed showing the table and the participants’ revealed cards. Both camera feeds were available to the researcher controlling the robot character. Four panels illustrate the study setup. The top schematic shows two participants seated across a table with a screen-based robot character between them. The wearer uses an in-ear device, while the non-wearer is not informed of the channel. The bottom panels show the robot character on the screen, a rock card in the cardholder camera feed, and an overhead camera view of the table and participants holding cards.
Figure 3. Information flow in the Wizard-of-Oz setup. The researcher monitored the cardholder and overhead camera feeds and controlled the screen-based robot character’s speech, expressions, and movements through the robot GUI. In the discreet condition, the researcher used the earable GUI to deliver a card-information cue when the wearer was the guesser or a binary-response prompt when the robot character was the guesser. Following the binary-response prompt, detection of a qualifying rightward head tilt set the displayed response state to “yes,” while non-detection within three seconds set it to “no.” The researcher monitored this state to guide the robot character’s opponent selection. In the baseline condition, the robot GUI selected the robot character’s opponent using its random-selection function. Interaction-flow diagram arranged from observation and researcher control to the robot character and participants. Solid arrows show the flow used in the baseline and discreet conditions: the researcher observes the cardholder camera feed and the overhead camera feed, operates the robot GUI, and triggers the robot character's publicly observable behavior. Dashed arrows show the additional discreet condition flow: the researcher selects a card-information cue or binary-response prompt in the earable GUI; the wearer receives in-ear audio and can perform the deliberate rightward head-tilt response; the eSense IMU supports detection or non-detection of a qualifying rightward head tilt, and the displayed response state informs the researcher.
Figure 4. Wearer-reported perceived pressure/tension and usability outcomes. Two panels showing wearer-reported outcomes. Left: boxplots of perceived pressure/tension scores in the baseline and discreet conditions. Right: boxplots of USE dimension scores for usefulness, ease of use, ease of learning, and satisfaction after the discreet condition, with the scale midpoint marked.
In this paper, we investigated how the choice of a Wizard-of-Oz (WoZ) interface affects communication with a robot from both the user's and the wizard's perspective. In a conversational setting, we used three WoZ interfaces with varying levels of dialogue input and output restrictions: a) a restricted perception GUI that showed fixed-view video and ASR transcripts and let the wizard trigger pre-scripted utterances and gestures; b) an unrestricted perception GUI that added real-time audio from the participant and the robot c) a VR telepresence interface that streamed immersive stereo video and audio to the wizard and forwarded the wizard's spontaneous speech, gaze and facial expressions to the robot. We found that the interaction mediated by the VR interface was preferred by users in terms of robot features and perceived social presence. For the wizards, the VR condition turned out to be the most demanding but elicited a higher social connection with the users. VR interface also induced the most connected interaction in terms of inter-speaker gaps and overlaps, while Restricted GUI induced the least connected flow and the largest silences. Given these results, we argue for more WoZ studies using telepresence interfaces. These studies better reflect the robots of tomorrow and offer a promising path to automation based on naturalistic contextualized verbal and non-verbal behavioral data.
Ekaterina Torubarova, Jura Miniota, Andre Pereira
Division of Speech, Music and Hearing, KTH Royal Institute of Technology, Stockholm, Sweden
People naturally chat while collaborating and share personal information (i.e., self-disclose) to build rapport and maintain social connections. As robots are increasingly developed to work with people, the effective use of these social behaviors to enhance engagement and support teamwork becomes ever more important. While prior work has shown that robot-initiated small talk can benefit human-robot collaboration, less is known about how best to design such small talk. In this work, we explore how self-disclosure may be designed to support small talk within a human-robot team---especially when the robot is an industrial manipulator that lacks anthropomorphic cues and performs physical work. We first developed an LLM-driven manipulator capable of partaking in small talk, adopting either a low-disclosure or high-disclosure strategy. We then conducted a user study (N = 50) to investigate how self-disclosure in small talk influences human-robot dynamics. Unexpectedly, participants disclosed more in the low-disclosure condition and reported stronger teaming and coordination than those in the high-disclosure condition. This effect was more pronounced among users with prior experience teaming with robots. These results suggest that increasing robot self-disclosure does not necessarily foster rapport, social connection, or reciprocal disclosure; other factors, such as prior HRI experience, should be considered.
Kaitlynn Taylor Pineda, Anvii Mishra, Brian Chien +4
Johns Hopkins University, Department of Computer Science, Baltimore, MD, USA
Wearable robot interfaces must connect user input to timely physical action. We present the proposed platform, an open smart-ring platform for studying this connection across task stages. The ring integrates a microphone, an inertial measurement unit, photoplethysmography, and skin-temperature sensing. Its software architecture separates acquisition, event interpretation, interaction policy, and robot execution. We characterize seven operating modes and evaluate speech and gesture control with 10 participants. Across five commands and 250 trials per modality, speech and gestures achieved task success rates of 90.4% and 88.8%, respectively. Recognition accuracy exceeded task success for both modalities. Gaussian hidden Markov models achieved higher accuracy than a support vector machine across three vocabulary sizes, with longer scoring times. Following a shared spoken task start, gesture-based adjustments had shorter median response times than speech. Whole-task success was 70.0% with gestures and 64.0% with speech. We further examine physiological-event-triggered robot interaction using simulated event notifications. Mean notification-receipt-to-arrival times were 11.71 s for the robot and 9.20 s for an on-site person. These descriptive findings support evaluating wearable robot interfaces through both recognition and physical outcomes, with timing measures appropriate to each task stage.
Qianggang Ding, Xingyao Wang, Rui Feng +7
Université de Montréal · Mila – Quebec Artificial Intelligence Institute · Zilo Lab +5