COFFAIL: A Dataset of Successful and Anomalous Robot Skill Executions in the Context of Coffee Preparation

Alex Mitrevski; Ayush Salunke

cs.ROApr 20, 2026

COFFAIL: A Dataset of Successful and Anomalous Robot Skill Executions in the Context of Coffee Preparation

Authors: Alex Mitrevski, Ayush Salunke

Organizations: Division of Systems and Control, Chalmers University of Technology, Gothenburg, Sweden · Autonomous Systems Group, Bonn-Rhein-Sieg University of Applied Sciences, Sankt Augustin, Germany

Abstract

In the context of robot learning for manipulation, curated datasets are an important resource for advancing the state of the art; however, available datasets typically only include successful executions or are focused on one particular type of skill. In this short paper, we briefly describe a dataset of various skills performed in the context of coffee preparation. The dataset, which we call COFFAIL, includes both successful and anomalous skill execution episodes collected with a physical robot in a kitchen environment, a couple of which are performed with bimanual manipulation. In addition to describing the data collection setup and the collected data, the paper illustrates the use of the data in COFFAIL to learn a robot policy using imitation learning.

Explore similar work

HABIT: Human-Aware Behavior and Interaction Training Dataset for Robot Manipulation
Jun 30, 2026Jaehwi Song, Suchae Jeong, Byeongguk Jeon +4Robotic Datasets Robot Policies
COBALT: Crowdsourcing Robot Learning via Cloud-Based Teleoperation with Smartphones
May 18, 2026Ayush Agarwal, Ansh Gandhi, Jeremy A. Collins +6Teleoperation Scalable Robot Learning
Ambient Diffusion Policy: Imitation Learning from Suboptimal Data in Robotics
Jun 10, 2026Adam Wei, Nicholas Pfaff, Thomas Cohn +4Diffusion Policies Imitation

Jun 30, 2026 · cs.ROJ/K move · Enter open · S save

HABIT: Human-Aware Behavior and Interaction Training Dataset for Robot Manipulation

Jaehwi Song, Suchae Jeong, Byeongguk Jeon, Sungdong Kim+3

Config · Config, KAIST

Large-scale demonstration datasets have been central to recent progress in general-purpose robot policies. However, existing datasets are collected in human-absent settings, and policies trained on such data may perform tasks competently in isolation but fail to exhibit human-aware behaviors. To address this gap, we introduce HABIT, a large-scale robot demonstration dataset for human-present environments. We organize tasks into three roles capturing distinct modes of human-robot interaction: Collaborator, where human and robot jointly accomplish a task; Coworker, where they pursue separate tasks in a shared space; and Supervisor, where the human directs the robot. The dataset comprises over 10K episodes and over 160 hours across 60 tasks. Our experiments show that training on human-present data elicits human-aware behaviors that robot-only data fails to produce: spatiotemporal synchronization in Collaborator tasks, yielding in Coworker tasks, and gesture grounding in Supervisor tasks. Moreover, training on HABIT enables rapid adaptation to new human-robot interaction tasks. By introducing human presence as a new axis of dataset diversity, HABIT extends robot policies to environments shared with humans.

Robotic Datasets Robot Policies

Open paper Ask about it PDF

Abstract

Explore similar work

HABIT: Human-Aware Behavior and Interaction Training Dataset for Robot Manipulation

COBALT: Crowdsourcing Robot Learning via Cloud-Based Teleoperation with Smartphones

Ambient Diffusion Policy: Imitation Learning from Suboptimal Data in Robotics