Graph-Reweighted Behavior Cloning with Human Interventions for Precise Robotic Manipulation

Xiaoman Li; Zhiqiang Yang; Jiayu Hu; Jipeng Kong; Yang Zhang; Jingwei Zhang; Fangzheng Yan; Chunhua Shen

cs.ROOct 14, 2026

Graph-Reweighted Behavior Cloning with Human Interventions for Precise Robotic Manipulation

Authors: Xiaoman Li, Zhiqiang Yang, Jiayu Hu, Jipeng Kong, Yang Zhang, Jingwei Zhang, Fangzheng Yan, Chunhua Shen,

Organizations: Zhejiang University, State Key Lab of CAD & CG · The Institute of Artificial Intelligence, China Telecom (TeleAI) · South China University of Technology · University of Science and Technology of China ·

Abstract

Long-horizon precise manipulation produces mixed-quality data for post-training vision-language-action (VLA) models: even successful demonstrations and intervention-assisted rollouts may contain detours, hesitations, failed attempts, and imperfect recoveries. The key challenge is to identify which action chunks actually advance the task, rather than uniformly cloning or discarding imperfect trajectories. We propose Graph-Reweighted Behavior Cloning (GR-BC), an offline iterative post-training framework that builds a task-conditioned goal-reachability graph over demonstrations and iteratively collected rollouts, including both successful and failed episodes with human interventions. Reverse graph search computes minimal remaining steps to task success within the dataset support, and each action chunk is weighted by how much its execution reduces this potential. This consequence-based weighting reuses recoverable segments from failed trajectories, leverages interventions as recovery paths, and down-weights locally regressive chunks even inside successful trajectories. New rollouts are incorporated by appending them to the graph and rerunning search, without training a reward model, action-conditioned critic, or progress predictor. Across four precise manipulation tasks and three VLA backbones, GR-BC improves single-round supervised fine-tuning and reaches 91% success after a few offline iterations, without trajectory comparisons or progress labels. More details and videos are available at: https://gr-bc.github.io

Figures & tables

Figure 1: Overview of the GR-BC framework. (A) We construct a task-conditioned goal-reachability graph from mixed-quality offline data. (B) Reverse graph search estimates minimal remaining steps to task success, providing dense potential signals. (C) Action chunks are weighted by these consequence-based potentials for VLA SFT. (D) New real-world rollouts, including failed episodes and human interventions, are iteratively incorporated to refine the graph and policy.

Figure 2: Real-world evaluation tasks. We evaluate GR-BC on four long-horizon precise manipulation tasks requiring precise visual alignment, contact-rich interactions, and recovery from execution errors.

Figure 3: Multi-view node encoding for graph construction.

Figure 4: Cross-trajectory similarity computation. We compute the pair-wise feature distance between states $v_{i}$ and $v_{j}$ , normalized by their local adaptive scales $\sigma_{c}(v)$ , to establish connectivity that reflects consistent physical state compatibility.

Figure 5: Comparison of progress estimation and transition weights. GRP (ours) provides more consistent progress signals than reward modeling baselines (SARM, Robometer, Robo-Dopamine, ReWiND). Our consequence-based weights effectively isolate task-advancing segments from local regressions or failed behaviors.

Method	$\Delta_{\mathrm{SF}}\uparrow$	$\rho_{\mathrm{dev}}\uparrow$	Class. SE $\uparrow$	Class. FE $\uparrow$	Multi-cam.	Failure data	Progress labels
ReWiND [ 34 ]	0.003	0	30/30	1/30	–	–	–
SARM [ 17 ]	0.24	0.48	26/30	14/30	–	–	✓
Robo-Dopamine [ 21 ]	0.29	0.59	29/30	8/30	✓	–	✓
RoboMeter [ 19 ]	0.159	0.69	30/30	6/30	–	✓	✓
GRP (ours)	0.97	0.92	30/30	30/30	✓	✓	–

Table 1: Progress-model interfaces and evaluation results.

Figure 6: Policy improvement across offline post-training rounds. GR-BC achieves more consistent gains than DAgger, AWR, and RA-BC on precise assembly tasks.

Variant	Failure reach. $\uparrow$	Zero-wt. ratio $\downarrow$	Pre/ normal $\downarrow$	Mismatch edges $\downarrow$
Temporal only	0.0%	62.3%	1.02	–
w/o gripper	97.8%	26.3%	0.46	15.8%
Ours	98.3%	25.0%	0.42	0.0%

Table 2: Graph ablation on the Assembly Model task.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Real-world robot workspace used for GR-BC data collection and evaluation.

Table 10

Table 11

Statistic	Assembly Model	Graphics Card	Screw Tightening	Memory Stick
Demo trajectories	215	125	100	150
Successful rollouts	85	270	160	240
Failed rollouts	100	120	80	80
Mean rollouts per round	90	120	80	100

Appendix

Table 7: Aggregate dataset statistics.

Parameter	Meaning	Default
$k$	temporal subsampling interval	15 frames
$K$	mutual KNN size per camera view	20
$\alpha_{\mathrm{edge}}$	adaptive similarity-edge threshold	1.0
$\epsilon_{c}$	stability constant for adaptive length	$10^{-4}$
$N$	action chunk horizon	30 frames
$\delta_{\min}$	minimum net progress threshold	0.002

Appendix

Table 8: Hyperparameters mentioned in the paper.

Figure 8: Two-trajectory graph construction example on the Assembly Model task. Blue and orange nodes denote two different trajectories, solid edges denote temporal transitions, and dashed red edges denote zero-cost edges from successful terminal nodes to $g_{\star}$ . Node colors indicate retrieved remaining steps $R_{\mathrm{rec}}(v)$ , where lower values correspond to states closer to task completion. The right image pairs show representative matched states; $R$ is the graph-retrieved remaining-step estimate and $d$ is the MCR feature-space distance between the matched states.

Figure 9: Graph overview constructed from the demonstration data and one rollout round for all four tasks. Darker node colors indicate larger remaining-step estimates.

Figure 10: Demo-only zero-shot progress evaluation on held-out Screw Tightening rollouts.

Split	# Traj.	Terminal Progress	Trajectory Avg. Progress	Low-Progress State Ratio	Reachable State Ratio	False Classification
Held-out success	18	$0.879$	$0.440$	$0.328$	$0.998$	$2/18$ FN
Held-out failure	17	$\mathbf{0.000}$	$\mathbf{0.019}$	$\mathbf{0.952}$	$0.127$	$\mathbf{0/17}$ FP
Success–failure gap	–	$\mathbf{0.879}$	$\mathbf{0.421}$	$\mathbf{-0.624}$	$\mathbf{0.871}$	–

Appendix

Table 9: Demo-only zero-shot progress evaluation. The graph is built from successful demonstrations only. Held-out rollouts are inserted one at a time, scored by graph-retrieved progress, and removed before the next rollout is evaluated. The diagnostic split yields a large terminal-progress gap between held-out successful and failed rollouts.

Parameter	X-VLA	SmolVLA	$\pi_{0.5}$
Input image resolution	Raw videos: $480\times 640$ ; model inputs: $256\times 256$ , $256\times 256$ , $224\times 224$	$480\times 640$ raw RGB videos; padded to $512\times 512$	$480\times 640$ raw RGB videos; resized to $224\times 224$
Number of camera views	3	3	3
Action representation	8-D joint/gripper command	8-D joint/gripper command	8-D joint/gripper command
Action chunk horizon $N$	30 frames	50 frames	30 frames
Hardware	4 NVIDIA H100 GPUs	4 NVIDIA H100 GPUs	4 NVIDIA H100 GPUs
Batch size	32 per GPU; 128 effective	32 per GPU; 128 effective	16 per GPU; 64 effective

Appendix

Table 10: VLA post-training hyperparameters.

Figure 11: Single-task backbone compatibility on Graphics Card. GR-BC improves over the DAgger-style baseline across X-VLA, SmolVLA, and $\pi_{0.5}$ in this controlled study.

Backbone	Method	BC	$r=1$	$r=2$	$r=3$
X-VLA	DAgger-style	0	8	17	63
X-VLA	GR-BC	0	14	67	92
SmolVLA	DAgger-style	0	8	8	14
SmolVLA	GR-BC	0	10	12	35
$\pi_{0.5}$	DAgger-style	3	12	18	55
$\pi_{0.5}$	GR-BC	3	14	69	93

Appendix

Table 11: Single-task backbone compatibility study on the Graphics Card task. We report success rate (%) across behavior cloning initialization and three iterative post-training rounds.

Parameter	Meaning	Recommended range	Effect
$k$	Temporal subsampling interval	10–20 frames	Controls graph density. Smaller values preserve fine contact dynamics but increase graph size; larger values reduce cost but may miss short regressions.
$K$	Mutual KNN size per camera view	10–30	Controls similarity-edge recall. Larger values improve connectivity but may introduce false visual neighbors.
$\alpha_{\mathrm{edge}}$	Adaptive similarity-edge threshold	0.8–1.2	Controls cross-trajectory stitching. Larger values improve failure reachability; smaller values give more conservative edges.
$\epsilon_{c}$	Stability constant for adaptive length	$10^{-5}$ – $10^{-3}$	Prevents unstable normalization when local visual scale is very small.
$R_{\max}$	Normalization scale for remaining steps	95th percentile; 95–99th percentile recommended range	Maps remaining steps to normalized progress. A small value saturates progress early; a large value compresses progress differences.
$N$	Action chunk horizon	20–40 frames	Defines the lookahead for chunk consequence. Smaller values give local credit; larger values capture longer effects but may mix behaviors.

Appendix

Table 12: Hyperparameters and recommended ranges for GR-BC.

$k$	Sim. edges	Fail reach.	$R_{\max}$	Zero wt.	High wt.
5	62,349	95.8%	125.0	26.8%	1.9%
10	30,221	97.5%	194.0	26.4%	21.7%
15	20,229	98.3%	255.0	24.6%	38.3%

Appendix

Table 13: Hyperparameter sensitivity diagnostics.

Figure 12: Encoder comparison overview on the Assembly Model rollout dataset. The visualization compares graph structure and progress behavior across R3M, MCR, and V-JEPA2.1 encoders under the same graph-construction rule.

Encoder	Sim. edges $\downarrow$	$R_{\max}$	Reachable $\uparrow$	Mean regression $\downarrow$	Total variation $\downarrow$	End progress $\uparrow$	Encode time $\downarrow$
R3M	12913	145.1	96.3%	0.388	1.420	0.745	60.2s
MCR	9973	223.5	94.4%	0.437	1.289	0.713	64.3s
V-JEPA2.1-B/384	19310	129.4	96.9%	0.656	1.688	0.567	123.7s
V-JEPA2.1-L/384	18980	131.8	95.1%	0.642	1.655	0.562	176.9s

Appendix

Table 14: Encoder comparison on the Assembly Model rollout dataset. All methods use stride

15

, history length

0

, and the same multi-view consensus and gripper-consistency check. Arrows indicate the preferred direction.

Figure 13: Multiple Success Modes on Assembly Model.

Figure 14: Shortest-path visualization from a failed state to a successful terminal state. The path alternates between temporal transitions and similarity bridges, showing how failed data can be routed through reusable recovery segments.

Figure 15: Examples of Progress prediction on demonstration data.

Figure 16: Examples of Progress prediction on policy rollouts.

Robot Imitation Learning

Robot imitation learning trains robots to perform tasks by reproducing and adapting behaviors shown in demonstrations.

See all 659 papers

GRAFT: Grounded and Efficient Online Reinforcement Adaptation for Fine-Grained Robot Manipulation
Aug 27, 2026Yibo Qiu, Haoliang Ye, Shu'ang Sun +3Visual Grounding for Robotics RL for Robotics
Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, China, and also with the School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, China
RePO-VLA: Recovery-Driven Policy Optimization for Vision-Language-Action Models
May 10, 2026Weijia Liufu, Xiaoyu Guo, Ruiyi Chen +16VLM Robustness Robot Failure Recovery
1Sun Yat-sen University · 2South China University of Technology · 3Peng Cheng Laboratory +3
ROVE: Unlocking Human Interventions for Humanoid Manipulation via Reinforcement Learning
Jun 15, 2026Wei Xiao, Weiliang Tang, Yuying Ge +4RL for Robotics Robotic Manipulation
1XPENG Robotics · 2Fudan University · 3The Chinese University of Hong Kong +1

Aug 27, 2026J/K move · Enter open · S save

GRAFT: Grounded and Efficient Online Reinforcement Adaptation for Fine-Grained Robot Manipulation

Yibo Qiu, Haoliang Ye, Shu'ang Sun, Zan Huang+2

Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, China, and also with the School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, China

Visual Grounding for Robotics RL for Robotics Robotic Manipulation Test-Time Adaptation of VLA Models

Open paper AskPDF arXiv

Pretrained vision-language-action (VLA) policies provide strong priors for robot manipulation, yet adapting them online to fine-grained biomedical tasks remains challenging. Task success often hinges on subtle, view-dependent visual cues, while task-level rewards provide little guidance about which regions matter, making it difficult to learn task-relevant visual grounding from limited real-robot interaction. Online adaptation is further constrained by the computational cost of VLA inference and replay-based updates. We introduce GRAFT (Grounded Reinforcement Adaptation for Fast Task Learning), a framework for efficient online VLA adaptation through grounded perception. GRAFT uses region-level supervision to learn view-specific visual anchors that focus perception on task-relevant local cues without requiring region proposals at deployment. It further combines single-step action generation with cached visual-language prefix reuse to accelerate online learning. Across four biomedical manipulation tasks, GRAFT improves success rates by 32.5 percentage points under matched adaptation budgets, while reducing the computational overhead of online policy updates.

Abstract

Robot Imitation Learning

Explore similar work

GRAFT: Grounded and Efficient Online Reinforcement Adaptation for Fine-Grained Robot Manipulation

RePO-VLA: Recovery-Driven Policy Optimization for Vision-Language-Action Models

ROVE: Unlocking Human Interventions for Humanoid Manipulation via Reinforcement Learning