CounterAlign: Counterfactual Supervision for Vision-Language-Action Models

Haru Kondoh; Kei Ota; Asako Kanezaki; Yueh-Hua Wu

cs.ROAug 22, 2026

CounterAlign: Counterfactual Supervision for Vision-Language-Action Models

Authors: Haru Kondoh, Kei Ota, Asako Kanezaki, Yueh-Hua Wu

Organizations: Institute of Science Tokyo · AI Robot Association (AIRoA) · RIKEN AIP · Tohoku University

Abstract

Vision-Language-Action (VLA) models are typically trained with behavior cloning (BC) on expert demonstrations. However, BC provides only positive supervision for expert actions, without explicit negative supervision indicating which actions are instruction-inconsistent or otherwise inappropriate. Reinforcement learning (RL) can provide such corrective signals, but often relies on externally specified rewards or curated non-expert data, both of which are costly to obtain in robotics. We show that offline RL for VLA models need not rely on curated non-expert trajectories: successful expert demonstrations alone can be transformed into dense corrective supervision through instruction relabeling. Specifically, by pairing expert actions with mismatched alternative instructions, we synthesize counterfactual instruction-observation-action tuples from the dataset and combine them with adversarial discriminator training to learn an instruction-grounded reward model for offline RL, without collecting additional rollouts or annotations. On the robustness-focused LIBERO-PRO benchmark, our method improves robustness to object position and task perturbations over a strong state-of-the-art baseline. It also outperforms competitive baselines in real-robot experiments on the TX-G2 (compatible with AGIBot G2). More broadly, our results suggest that, for data-constrained VLA learning, extracting denser supervision from each demonstration can complement collecting additional data.

Figures & tables

Figure 1: Overview of CounterAlign . Through adversarial and relabeling-based discriminator training, CounterAlign converts expert demonstrations into an instruction-grounded reward. Relabeling creates counterfactual negative or unlabeled tuples that teach the reward model which instruction–action pairings are semantically inconsistent. The learned reward is then used to train a VLA policy with offline RL, improving robustness to object position and task perturbations without additional data collection or annotation.

Figure 2: The two discriminators share the same architecture but are trained on different types of samples. (a) The adversarial discriminator is trained on original expert samples and samples whose actions are replaced with policy-generated actions. (b) The relabeling discriminator is trained on samples whose instructions or actions are replaced with other instructions or actions from the dataset.

Method	LIBERO-Spatial				LIBERO-Object				LIBERO-Goal				LIBERO-10
Method	Obj	Pos	Sem	Task	Obj	Pos	Sem	Task	Obj	Pos	Sem	Task	Obj	Pos	Sem	Task
OpenVLA-OFT [ 20 ]	0.30	0.14	0.65	0.0	0.67	0.10	0.89	0.0	0.85	0.0	0.44	0.03	0.06	0.0	0.39	0.0
VLA-Adapter [ 49 ]	0.98	0.0	0.98	0.49	0.89	0.0	0.99	0.08	0.61	0.0	0.75	0.12	0.47	0.0	0.91	0.10
xVLA [ 62 ]	0.90	0.0	0.69	0.39	0.92	0.05	0.98	0.0	0.72	0.10	0.94	0.08	0.61	0.10	0.71	0.20
$\pi_{0.5}$ [ 2 ]	0.99	0.53	0.97	0.55	0.89	0.19	0.95	0.10	0.90	0.29	0.95	0.17	0.66	0.06	0.91	0.17
CounterAlign	0.98	0.60	0.97	0.63	0.80	0.51	0.99	0.26	0.82	0.41	0.94	0.46	0.64	0.11	0.97	0.29

Table 1: Success rate comparisons on the LIBERO-PRO [ 63 ] . CounterAlign yields its largest improvements in the Pos and Task settings, where the policy cannot succeed by simply replaying actions observed in the training data.

Ent.	Unlabel	D	C	A	LIBERO-Spatial				LIBERO-Object				LIBERO-Goal				LIBERO-10
Ent.	Unlabel	D	C	A	Obj	Pos	Sem	Task	Obj	Pos	Sem	Task	Obj	Pos	Sem	Task	Obj	Pos	Sem	Task
-	-	$\times$	$\times$	$\times$	0.96	0.56	0.97	0.60	0.86	0.49	1.0	0.22	0.85	0.36	0.93	0.29	0.62	0.09	0.96	0.14
$\times$	$\times$	$\checkmark$	$\times$	$\times$	1.0	0.59	0.96	0.57	0.92	0.54	0.99	0.22	0.84	0.34	0.98	0.32	0.66	0.08	0.92	0.16
$\checkmark$	$\times$	$\checkmark$	$\times$	$\times$	0.98	0.55	0.94	0.65	0.82	0.63	0.99	0.18	0.87	0.38	0.97	0.41	0.69	0.08	0.92	0.19
$\checkmark$	$\checkmark$	$\checkmark$	$\times$	$\times$	0.94	0.54	0.97	0.66	0.87	0.47	0.98	0.29	0.83	0.37	0.96	0.47	0.63	0.12	0.89	0.20
$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	$\times$	0.98	0.60	0.97	0.63	0.80	0.51	0.99	0.26	0.82	0.41	0.94	0.46	0.64	0.11	0.97	0.29

Table 2: Ablation study on LIBERO-PRO [ 63 ] . Base algorithms are IQL [ 23 ] and advantage-weighted flow matching. Ent. denotes whether the entropy regularization is used in discriminator training, and Unlabel denotes whether unlabeled samples are used for discriminator training. D, C, and A indicate whether relabeling is applied to the discriminator, critic, and actor, respectively.

Method	Task 1	Task 2	Task 3	Task 4
VLA-Adapter [ 49 ]	0.0	0.0	0.0	0.0
xVLA [ 62 ]	0.0	0.08	0.05	0.05
$\pi_{0.5}$ [ 2 ]	0.05	0.48	0.80	0.60
CounterAlign	0.20	0.53	0.93	0.85

Table 3: Comparisons on the TX-G2.

Figure 4: Trajectories from the LIBERO dataset and the TX-G2 dataset, along with the values of the learned advantage.

Short-horizon task	Num. demos.
Pick up the cutleries from the bowl and place them in the other bowl.	206
Pick up the bowls from the desk and stack them.	129
Pick up the clothes from the desk and place them into the basket.	514
Pick up the dishes from the desk and place them in the wooden dish rack.	349
Total	1,198

Table 1: Short-horizon task statistics for the TX-G2 dataset.

Short-horizon task	Primitive actions
Pick up the cutleries from the bowl and place them in the other bowl.	1. Pick up the light blue spoon from the grey bowl. 2. Place the light blue spoon in the yellow bowl. 3. Pick up the pink fork from the grey bowl. 4. Place the pink fork in the yellow bowl.
Pick up the bowls from the desk and stack them.	1. Pick up the yellow bowl from the desk. 2. Stack the yellow bowl on the grey bowl. 3. Pick up the light blue bowl from the desk. 4. Stack the light blue bowl on the yellow bowl.
Pick up the clothes from the desk and place them into the basket.	1. Pick up the green socks from the desk. 2. Place the green socks into the basket. 3. Pick up the handkerchief from the desk. 4. Place the handkerchief into the basket. 5. Pick up the yellow socks from the desk. 6. Place the yellow socks into the basket.
Pick up the dishes from the desk and place them in the wooden dish rack.	1. Pick up the yellow dish from the desk. 2. Place the yellow dish in the wooden dish rack. 3. Pick up the green dish from the desk. 4. Place the green dish in the wooden dish rack.

Table 2: Decomposition of short-horizon tasks into primitive actions for the TX-G2 dataset.

Algo.	Tasks completed in a row					Avg. len.
Algo.	1	2	3	4	5	Avg. len.
BC	0.906	0.835	0.776	0.733	0.682	3.93
Ours	0.913	0.848	0.804	0.757	0.703	4.03

Table 3: Comparison on the CALVIN ABC

\rightarrow

D benchmark [ 35 ] . The base VLA model is

\pi_{0.5}

[ 2 ] .

Explore similar work

When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAs
Feb 19, 2026Yu Fang, Yuchun Feng, Dong Jing +5VLM Robustness Counterfactual Evaluation of VLMs
University of North Carolina at Chapel Hill
Imagining Recovery: Inference-Time Counterfactual Realignment for Vision-Language-Action Models
Aug 14, 2026Yanyan Zhang, Disheng Liu, Kai Ye +6Vision-Language-Action Models Robustness of VLA Models
Case Western Reserve University Cleveland, OH, USA
Restoring Linguistic Grounding in VLA Models via Train-Free Attention Recalibration
Mar 6, 2026Ninghao Zhang, Bin Zhu, Shijie Zhou +1Vision-Language-Action Models Robustness of VLA Models
Tsinghua University · Singapore Management University · Institute of Trustworthy Embodied AI, Fudan University +1

Feb 19, 2026J/K move · Enter open · S save

When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAs

Yu Fang, Yuchun Feng, Dong Jing, Jiaqi Liu+4

University of North Carolina at Chapel Hill

VLM Robustness Counterfactual Evaluation of VLMs Robotic Manipulation Vision-Language-Action Models Language-Conditioned Robot Control Robustness of VLA Models

Open paper AskPDF arXiv

Vision-Language-Action models (VLAs) promise to ground language instructions in robot control, yet in practice often fail to faithfully follow language. When presented with instructions that lack strong scene-specific supervision, VLAs suffer from counterfactual failures: they act based on vision shortcuts induced by dataset biases, repeatedly executing well-learned behaviors and selecting objects frequently seen during training regardless of language intent. To systematically study it, we introduce LIBERO-CF, the first counterfactual benchmark for VLAs that evaluates language following capability by assigning alternative instructions under visually plausible LIBERO layouts. Our evaluation reveals that counterfactual failures are prevalent yet underexplored across state-of-the-art VLAs. We propose Counterfactual Action Guidance (CAG), a simple yet effective dual-branch inference scheme that explicitly regularizes language conditioning in VLAs. CAG combines a standard VLA policy with a language-unconditioned Vision-Action (VA) module, enabling counterfactual comparison during action selection. This design reduces reliance on visual shortcuts, improves robustness on under-observed tasks, and requires neither additional demonstrations nor modifications to existing architectures or pretrained models. Extensive experiments demonstrate its plug-and-play integration across diverse VLAs and consistent improvements. For example, on LIBERO-CF, CAG improves

π_{0.5}

by 9.7% in language following accuracy and 3.6% in task success on under-observed tasks using a training-free strategy, with further gains of 15.5% and 8.5%, respectively, when paired with a VA model. In real-world evaluations, CAG reduces counterfactual failures of 9.4% and improves task success by 17.2% on average.

Abstract

Explore similar work

When Vision Overrides Language: Evaluating and Mitigating Counterfactual Failures in VLAs

Imagining Recovery: Inference-Time Counterfactual Realignment for Vision-Language-Action Models

Restoring Linguistic Grounding in VLA Models via Train-Free Attention Recalibration