Evidence-Guided Counterfactual Learning for Long-Tailed Robotic Policies

Weiwei Chen; Chenghao Xu; Meiling Yu; Xu Yang; Cheng Deng

cs.ROOct 9, 2026

Evidence-Guided Counterfactual Learning for Long-Tailed Robotic Policies

Authors: Weiwei Chen, Chenghao Xu, Meiling Yu, Xu Yang, Cheng Deng

Organizations: Hohai University

Abstract

In the practical deployment of vision--language--action policies, handling long-tailed task distributions is a common challenge due to the highly imbalanced composition of robot demonstration datasets. To investigate this issue, we study the head-task action bias in shared robot policies, particularly focusing on its relationship with task-specific visual evidence. Our experimental analysis shows that, when the visual evidence required by a tail task is insufficient, the policy increasingly falls back on action patterns reinforced by data-rich head tasks, resulting in degraded performance on under-represented tasks. Motivated by this observation, we propose Evidence-Guided Counterfactual Learning, a training framework for long-tailed robot policies. Our method identifies decision-relevant visual evidence using instruction-related and action-related signals, and constructs counterfactual observations by removing the selected evidence. The resulting counterfactual predictions are used to strengthen the association among task instructions, visual evidence, and actions, with the supervision adaptively adjusted according to the number of demonstrations available for each task. The proposed framework requires neither additional training data nor region-level annotations and incurs no additional computation during deployment. Extensive experiments across simulated benchmarks, different VLA architectures, and real-world robot platforms demonstrate that our method robustly and consistently improves long-tailed robot policy learning, particularly for tail tasks, while maintaining reliable performance on head tasks.

Figures & tables

Panel of Figure 1 — Figure 1: Head vs.tail task performance and intermediate-level visual grounding. (a)Success rates for policies trained on FULL and LT. (b) Visual attention for “Pick up the black bowl next to the plate and place it on the plate.” (c) Visual attention for “Put the wine bottle on the rack.”

Panel of Figure 2 — Figure 2: Diagnosing Head Task Action Bias. (a) Two-task environment. (b) Action distributions before and after relational masking at the same state under a tail-task instruction. (c) Head-Prior Score and Head-Action Rate versus task imbalance.

Figure 3: Overview of ECL. (I) Joint language–action relevance selects top- $K$ visual tokens for masking. (II) A shared policy produces factual, masked, and contrastive training outputs, supervised by $\mathcal{L}_{\mathrm{BC}}$ , $\mathcal{L}_{\mathrm{mask}}$ , and $\mathcal{L}_{\mathrm{CF}}$ , respectively. (III) Task frequency determines the per-sample weight of the counterfactual loss. At deployment, only the factual policy is used.

	MiniVLA Success Rate (%)				$\pi_{0.5}$ Success Rate (%, synthetic)
Task	FULL	LT(Baseline)	APA	ECL(Ours)	FULL	LT(Baseline)	APA	ECL(Ours)
Task 0	56.6 $\,\pm\,4.0$	68.4 $\,\pm\,2.3$	72.6 $\,\pm\,2.3$	80.0 $\,\pm\,2.2$	56.7 $\,\pm\,2.4$	72.4 $\,\pm\,3.4$	69.5 $\,\pm\,1.6$	75.6 $\,\pm\,2.5$
Task 1	78.2 $\,\pm\,4.0$	49.6 $\,\pm\,3.6$	67.3 $\,\pm\,1.4$	68.9 $\,\pm\,1.3$	61.5 $\,\pm\,3.5$	64.4 $\,\pm\,1.6$	65.8 $\,\pm\,1.9$	71.7 $\,\pm\,1.8$
Task 2	72.7 $\,\pm\,3.8$	62.2 $\,\pm\,3.2$	66.1 $\,\pm\,1.8$	74.7 $\,\pm\,1.8$	72.6 $\,\pm\,2.4$	58.7 $\,\pm\,2.4$	57.8 $\,\pm\,1.5$	66.4 $\,\pm\,0.7$
Task 3	50.3 $\,\pm\,3.8$	55.2 $\,\pm\,3.5$	70.0 $\,\pm\,2.1$	83.5 $\,\pm\,1.8$	50.2 $\,\pm\,1.6$	52.6 $\,\pm\,2.6$	55.4 $\,\pm\,1.9$	59.6 $\,\pm\,0.4$
Task 4	66.3 $\,\pm\,4.2$	46.7 $\,\pm\,3.6$	56.5 $\,\pm\,1.6$	64.7 $\,\pm\,1.4$	68.3 $\,\pm\,2.0$	45.8 $\,\pm\,1.5$	43.3 $\,\pm\,3.4$	52.1 $\,\pm\,3.4$

Table 1: Per-task success rates (%, mean

\pm

binomial standard deviation of the rate) on the ten-task LIBERO-Core benchmark. The table reports four training configurations for MiniVLA and

\pi_{0.5}

.The Average standard deviation is computed across the three seed-wise mean success rates.

	AGILEX PIPER				xArm6
	MiniVLA		$\pi_{0.5}$		MiniVLA		$\pi_{0.5}$
Task	APA	ECL(Ours)	APA	ECL(Ours)	APA	ECL(Ours)	APA	ECL(Ours)
Task 0	52 $\,\pm\,7.1$	68 $\,\pm\,6.6$	42 $\,\pm\,7.0$	42 $\,\pm\,7.0$	60 $\,\pm\,6.9$	62 $\,\pm\,6.9$	64 $\,\pm\,6.8$	68 $\,\pm\,7.0$
Task 1	18 $\,\pm\,5.4$	26 $\,\pm\,6.2$	36 $\,\pm\,6.8$	38 $\,\pm\,6.9$	20 $\,\pm\,5.7$	38 $\,\pm\,6.9$	20 $\,\pm\,5.7$	40 $\,\pm\,6.9$
Task 2	10 $\,\pm\,4.2$	16 $\,\pm\,5.2$	18 $\,\pm\,5.4$	20 $\,\pm\,5.7$	12 $\,\pm\,4.6$	20 $\,\pm\,5.7$	22 $\,\pm\,5.9$	24 $\,\pm\,6.0$
Task 3	4 $\,\pm\,2.8$	6 $\,\pm\,3.4$	16 $\,\pm\,5.2$	18 $\,\pm\,5.4$	4 $\,\pm\,2.8$	6 $\,\pm\,2.8$	14 $\,\pm\,4.9$	20 $\,\pm\,5.7$

Table 2: Real-world per-task success rates (%) on AGILEX PIPER and xArm6. For each platform, APA and ECL are evaluated with MiniVLA and

\pi_{0.5}

on six tasks ordered from head tasks (Task 0-1) to tail tasks (Task 2-5).

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Language instruction	FULL	LT
0	Pick up the black bowl next to the plate and place it on the plate	46	46
1	Pick up the black bowl next to the cookie box and place it on the plate	47	28
2	Pick up the black bowl on the cookie box and place it on the plate	45	19
3	Pick up the ketchup and place it in the basket	42	15
4	Pick up the alphabet soup and place it in the basket	47	11
5	Push the plate to the front of the stove	39	9

Appendix

Table 3: LIBERO-Core task definitions and demonstration counts. “FULL” and “LT” denote the full and long-tailed training configurations, respectively. Task IDs follow the ordering in the project benchmark map.

Task	Language instruction	FULL	LT
0	Pick up the block and place it in the box	50	50
1	Stack the red block on top of the blue block	48	46
2	Stack the two blocks on the left on top of the block on the right	47	29
3	Insert the three-prong plug into the socket	49	13
4	Push the box to the designated location	46	10
5	Pick up the water bottle and place it in the box	45	9

Appendix

Table 4: Real-world task definitions and demonstration allocation. FULL denotes the approximately balanced reference configuration, and LT denotes the long-tailed configuration. Task IDs follow the order used in the real-world success-rate table.

Figure 6: Examples from the real-world task-level long-tailed dataset. AGILEX PIPER provides global-camera and wrist-camera observations, while xArm6 provides global-camera observations. The columns show the six manipulation tasks listed in Table 4 .

Figure 7: Examples from the Long-Tailed Context (LTC) dataset. Columns correspond to Scenes 0–5 in Table 5 ; the top and bottom rows show the global-camera and wrist-camera observations, respectively.

Language instruction	Scene	FULL	LT
Put the green block into the box	0	50	50
	1	50	25
	2	50	20
	3	50	15
	4	50	10
	5	50	5

Appendix

Table 5: Context-level task definition and demonstration counts used in Section 4.4. All scenes share the same language instruction. FULL contains 50 demonstrations per scene, while LT progressively reduces the number of demonstrations for rarer scene configurations.

Backbone	Rollout limit	Episodes/task	Seeds	Episodes/seed	Action horizon
MiniVLA	350 steps	225	7, 14, 21	75	—
$\pi_{0.5}$	350 steps	225	7, 14, 21	75	10 actions

Appendix

Table 6: Configuration for the simulation rollout evaluation.

Panel of Table 7 — Table 7: MiniVLA rollout for Task 3, “Pick up the ketchup and place it in the basket.”

Panel of Table 8 — Table 8: MiniVLA rollout for Task 6, “Push the plate to the front of the stove.”

Panel of Table 9 — Table 9: $\pi_{0.5}$ rollout for Task 8, “Put the wine bottle on top of the cabinet.”

Panel of Table 10 — Table 10: $\pi_{0.5}$ rollout for Task 9, “Put the wine bottle on the rack.”

Robot	Visual observations	Backbones	Comparison / trials
AGILEX PIPER	Global camera + wrist	MiniVLA, $\pi_{0.5}$	APA vs. ECL / 50 per task
xArm6	Global camera	MiniVLA, $\pi_{0.5}$	APA vs. ECL / 50 per task

Appendix

Table 11: Configuration for the real-world rollout evaluation.

Panel of Table 12 — Table 12: AGILEX PIPER rollout with MiniVLA (global-camera and wrist-camera views).

Panel of Table 13 — Table 13: AGILEX PIPER rollout with $\pi_{0.5}$ (global-camera and wrist-camera views).

Figure 8: xArm6 per-task success rates for MiniVLA and $\pi_{0.5}$ , comparing APA with ECL.

Panel of Table 14 — Table 14: xArm6 rollout comparison with MiniVLA.

Panel of Table 15 — Table 15: xArm6 rollout comparison with $\pi_{0.5}$ .

Figure 9: Illustrative comparison of mask selection strategies. From left to right: object-only, action-only, random, and object–action relation selection. Peach overlays with white borders mark selected patches; all panels select 26/256 patches. The supplied visualization uses semantic-mask and saved-attention proxies, as described below, rather than a direct export of the complete ECL selector.

Panel of Figure 10 — Figure 10: Visual evidence on representative LT tail tasks. Each square shows a paired comparison with the LT baseline on the left and ECL on the right. The top row shows “put the bowl on top of the cabinet,” and the bottom row shows “put the wine bottle on the rack.” Relative to the LT baseline, ECL more consistently grounds its decisions in instruction-relevant objects, interaction regions, and goal locations, supporting our finding that ECL effectively learns the correct task-specific visual evidence for tail-task decision making.

Method	Data size	GPUs	Batch	Logged steps	Mean (s)	Median (s)	Logged (h)
FULL	55,966	1	20/20	100,738	0.692	0.660	18.41
LT (Baseline)	19,234	1	20/20	34,622	0.608	0.570	5.89
APA	21,049	1	20/20	37,888	0.673	0.651	7.21
ECL (Ours)	19,234	1	20/20	34,622	1.068	1.090	10.27

Appendix

Table 16: Recorded MiniVLA training cost. GPU counts and batch sizes are taken from saved configurations. Batch denotes per-device/global batch size. Logged hours cover only the supplied timing records, which span different training intervals; these rows are not a controlled overhead comparison.

Explore similar work

CounterAlign: Counterfactual Supervision for Vision-Language-Action Models
Aug 22, 2026Haru Kondoh, Kei Ota, Asako Kanezaki +1RL for VLMs Robustness of VLA Models
Institute of Science Tokyo · AI Robot Association (AIRoA) · RIKEN AIP +1
SeeTraceAct: Visibility-Aware Latent Planning from Cross-Embodiment Demonstration Videos
Jun 1, 2026Jaehyeon Son, Junhyun Kim, Kyle Kam +7Robot Skill Learning Generalization in Robotic Manipulation
Georgia Institute of Technology · Allen Institute for AI · Johns Hopkins University +1
EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policies
Jun 18, 2026Ganlin Yang, Zhangzheng Tu, Yuqiang Yang +10Long-Horizon Robotic Manipulation Vision-Language-Action Models
University of Science and Technology of China · Shanghai AI Laboratory · Dalian University of Technology +5

Aug 22, 2026J/K move · Enter open · S save

CounterAlign: Counterfactual Supervision for Vision-Language-Action Models

Haru Kondoh, Kei Ota, Asako Kanezaki, Yueh-Hua Wu

Institute of Science Tokyo · AI Robot Association (AIRoA) · RIKEN AIP+1

RL for VLMs Robustness of VLA Models

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models are typically trained with behavior cloning (BC) on expert demonstrations. However, BC provides only positive supervision for expert actions, without explicit negative supervision indicating which actions are instruction-inconsistent or otherwise inappropriate. Reinforcement learning (RL) can provide such corrective signals, but often relies on externally specified rewards or curated non-expert data, both of which are costly to obtain in robotics. We show that offline RL for VLA models need not rely on curated non-expert trajectories: successful expert demonstrations alone can be transformed into dense corrective supervision through instruction relabeling. Specifically, by pairing expert actions with mismatched alternative instructions, we synthesize counterfactual instruction-observation-action tuples from the dataset and combine them with adversarial discriminator training to learn an instruction-grounded reward model for offline RL, without collecting additional rollouts or annotations. On the robustness-focused LIBERO-PRO benchmark, our method improves robustness to object position and task perturbations over a strong state-of-the-art baseline. It also outperforms competitive baselines in real-robot experiments on the TX-G2 (compatible with AGIBot G2). More broadly, our results suggest that, for data-constrained VLA learning, extracting denser supervision from each demonstration can complement collecting additional data.

Abstract

Explore similar work

CounterAlign: Counterfactual Supervision for Vision-Language-Action Models

SeeTraceAct: Visibility-Aware Latent Planning from Cross-Embodiment Demonstration Videos

EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policies