cs.ROOct 11, 2026

ReTrace: Cross-Stage Failure Attribution for Self-Improving Robotic Manipulation

Authors: Yue Yu, Bin Zhu, Junhui Li, Jiayu Wang, Shijie Zhou, Jingjing Chen

Organizations: College of Computer Science and Artificial Intelligence, Fudan University · Institute of Trustworthy Embodied AI, Fudan University · Singapore Management University

Abstract

General-purpose multimodal foundation models are increasingly capable of directly controlling robots, yet improving their manipulation capabilities through autonomous interaction remains challenging. A key challenge to such self-improvement is learning from failures in complex manipulation tasks involving cross-stage dependencies, where an action may succeed locally but cause a later stage to fail. To address this challenge, we introduce ReTrace, a self-improvement framework for robotic manipulation through autonomous physical interaction and retrospective reasoning, without provided demonstrations or model-parameter updates. ReTrace retrospectively traces downstream failures to earlier decisions, proposes targeted corrections, and evaluates them through subsequent interactions. Across attempts, ReTrace progressively consolidates validated corrections into a reusable manipulation skill that guide subsequent execution. We evaluate ReTrace on four real-world manipulation tasks using a Franka Research 3 robot. On the two tasks with strong cross-stage dependencies, GPT-6 Astra-low improves from 0% to 80% success on plug insertion and from 0% to 100% on ball pouring, while GPT-6 Sol-xhigh improves from 0% to 10% and 0% to 90%, respectively. The learned skills also transfer across model and reasoning configurations and reduce inference-time decision cost in most evaluated settings.

Figures & tables

Explore similar work

CardsList
  1. Reconstruct, Practice, Go Real: Guided Self-Improvement for Embodied Agents

    Oct 1, 2026Yen-Jen Wang, Haozhe Jiang, Shuying Deng +7Robot Skill LearningLarge Language Model-Based Robot Planning

  2. Grasp-Then-Plan with Failure Attribution: A Closed Two-Stage Framework for Precise and Generalizable Robotic Manipulation

    Jun 2, 2026Jiahao Xu, Peiyuan Wang, Hanzhuo Zhang +8Robotic GraspingLong-Horizon Robotic Manipulation

  3. GazeVLA: Learning Human Intention for Robotic Manipulation

    Apr 24, 2026Chengyang Li, Kaiyi Xiong, Yuan Xu +3Long-Horizon Robotic ManipulationRobot Skill Learning