Vision-Language-Action (VLA) models are typically trained with behavior cloning (BC) on expert demonstrations. However, BC provides only positive supervision for expert actions, without explicit negative supervision indicating which actions are instruction-inconsistent or otherwise inappropriate. Reinforcement learning (RL) can provide such corrective signals, but often relies on externally specified rewards or curated non-expert data, both of which are costly to obtain in robotics. We show that offline RL for VLA models need not rely on curated non-expert trajectories: successful expert demonstrations alone can be transformed into dense corrective supervision through instruction relabeling. Specifically, by pairing expert actions with mismatched alternative instructions, we synthesize counterfactual instruction-observation-action tuples from the dataset and combine them with adversarial discriminator training to learn an instruction-grounded reward model for offline RL, without collecting additional rollouts or annotations. On the robustness-focused LIBERO-PRO benchmark, our method improves robustness to object position and task perturbations over a strong state-of-the-art baseline. It also outperforms competitive baselines in real-robot experiments on the TX-G2 (compatible with AGIBot G2). More broadly, our results suggest that, for data-constrained VLA learning, extracting denser supervision from each demonstration can complement collecting additional data.
Figures & tables
Figure 1: Overview of CounterAlign . Through adversarial and relabeling-based discriminator training, CounterAlign converts expert demonstrations into an instruction-grounded reward. Relabeling creates counterfactual negative or unlabeled tuples that teach the reward model which instruction–action pairings are semantically inconsistent. The learned reward is then used to train a VLA policy with offline RL, improving robustness to object position and task perturbations without additional data collection or annotation.
Figure 2: The two discriminators share the same architecture but are trained on different types of samples. (a) The adversarial discriminator is trained on original expert samples and samples whose actions are replaced with policy-generated actions. (b) The relabeling discriminator is trained on samples whose instructions or actions are replaced with other instructions or actions from the dataset.
Method
LIBERO-Spatial
LIBERO-Object
LIBERO-Goal
LIBERO-10
Obj
Pos
Sem
Task
Obj
Pos
Sem
Task
Obj
Pos
Sem
Task
Obj
Pos
Sem
Task
OpenVLA-OFT [ 20 ]
0.30
0.14
0.65
0.0
0.67
0.10
0.89
0.0
0.85
0.0
0.44
0.03
0.06
0.0
0.39
0.0
VLA-Adapter [ 49 ]
0.98
0.0
0.98
0.49
0.89
0.0
0.99
0.08
0.61
0.0
0.75
0.12
0.47
0.0
0.91
0.10
xVLA [ 62 ]
0.90
0.0
0.69
0.39
0.92
0.05
0.98
0.0
0.72
0.10
0.94
0.08
0.61
0.10
0.71
0.20
π0.5 [ 2 ]
0.99
0.53
0.97
0.55
0.89
0.19
0.95
0.10
0.90
0.29
0.95
0.17
0.66
0.06
0.91
0.17
CounterAlign
0.98
0.60
0.97
0.63
0.80
0.51
0.99
0.26
0.82
0.41
0.94
0.46
0.64
0.11
0.97
0.29
Table 1: Success rate comparisons on the LIBERO-PRO [ 63 ] . CounterAlign yields its largest improvements in the Pos and Task settings, where the policy cannot succeed by simply replaying actions observed in the training data.
Ent.
Unlabel
D
C
A
LIBERO-Spatial
LIBERO-Object
LIBERO-Goal
LIBERO-10
Obj
Pos
Sem
Task
Obj
Pos
Sem
Task
Obj
Pos
Sem
Task
Obj
Pos
Sem
Task
-
-
×
×
×
0.96
0.56
0.97
0.60
0.86
0.49
1.0
0.22
0.85
0.36
0.93
0.29
0.62
0.09
0.96
0.14
×
×
✓
×
×
1.0
0.59
0.96
0.57
0.92
0.54
0.99
0.22
0.84
0.34
0.98
0.32
0.66
0.08
0.92
0.16
✓
×
✓
×
×
0.98
0.55
0.94
0.65
0.82
0.63
0.99
0.18
0.87
0.38
0.97
0.41
0.69
0.08
0.92
0.19
✓
✓
✓
×
×
0.94
0.54
0.97
0.66
0.87
0.47
0.98
0.29
0.83
0.37
0.96
0.47
0.63
0.12
0.89
0.20
✓
✓
✓
✓
×
0.98
0.60
0.97
0.63
0.80
0.51
0.99
0.26
0.82
0.41
0.94
0.46
0.64
0.11
0.97
0.29
Table 2: Ablation study on LIBERO-PRO [ 63 ] . Base algorithms are IQL [ 23 ] and advantage-weighted flow matching. Ent. denotes whether the entropy regularization is used in discriminator training, and Unlabel denotes whether unlabeled samples are used for discriminator training. D, C, and A indicate whether relabeling is applied to the discriminator, critic, and actor, respectively.
Figure 3: TX-G2 experimental setup.
Method
Task 1
Task 2
Task 3
Task 4
VLA-Adapter [ 49 ]
0.0
0.0
0.0
0.0
xVLA [ 62 ]
0.0
0.08
0.05
0.05
π0.5 [ 2 ]
0.05
0.48
0.80
0.60
CounterAlign
0.20
0.53
0.93
0.85
Table 3: Comparisons on the TX-G2.
Figure 4: Trajectories from the LIBERO dataset and the TX-G2 dataset, along with the values of the learned advantage.
Short-horizon task
Num. demos.
Pick up the cutleries from the bowl and place them in the other bowl.
206
Pick up the bowls from the desk and stack them.
129
Pick up the clothes from the desk and place them into the basket.
514
Pick up the dishes from the desk and place them in the wooden dish rack.
349
Total
1,198
Table 1: Short-horizon task statistics for the TX-G2 dataset.
Short-horizon task
Primitive actions
Pick up the cutleries from the bowl and place them in the other bowl.
1. Pick up the light blue spoon from the grey bowl. 2. Place the light blue spoon in the yellow bowl. 3. Pick up the pink fork from the grey bowl. 4. Place the pink fork in the yellow bowl.
Pick up the bowls from the desk and stack them.
1. Pick up the yellow bowl from the desk. 2. Stack the yellow bowl on the grey bowl. 3. Pick up the light blue bowl from the desk. 4. Stack the light blue bowl on the yellow bowl.
Pick up the clothes from the desk and place them into the basket.
1. Pick up the green socks from the desk. 2. Place the green socks into the basket. 3. Pick up the handkerchief from the desk. 4. Place the handkerchief into the basket. 5. Pick up the yellow socks from the desk. 6. Place the yellow socks into the basket.
Pick up the dishes from the desk and place them in the wooden dish rack.
1. Pick up the yellow dish from the desk. 2. Place the yellow dish in the wooden dish rack. 3. Pick up the green dish from the desk. 4. Place the green dish in the wooden dish rack.
Table 2: Decomposition of short-horizon tasks into primitive actions for the TX-G2 dataset.
Algo.
Tasks completed in a row
Avg. len.
1
2
3
4
5
BC
0.906
0.835
0.776
0.733
0.682
3.93
Ours
0.913
0.848
0.804
0.757
0.703
4.03
Table 3: Comparison on the CALVIN ABC → D benchmark [ 35 ] . The base VLA model is π0.5 [ 2 ] .
Vision-Language-Action models (VLAs) promise to ground language instructions in robot control, yet in practice often fail to faithfully follow language. When presented with instructions that lack strong scene-specific supervision, VLAs suffer from counterfactual failures: they act based on vision shortcuts induced by dataset biases, repeatedly executing well-learned behaviors and selecting objects frequently seen during training regardless of language intent. To systematically study it, we introduce LIBERO-CF, the first counterfactual benchmark for VLAs that evaluates language following capability by assigning alternative instructions under visually plausible LIBERO layouts. Our evaluation reveals that counterfactual failures are prevalent yet underexplored across state-of-the-art VLAs. We propose Counterfactual Action Guidance (CAG), a simple yet effective dual-branch inference scheme that explicitly regularizes language conditioning in VLAs. CAG combines a standard VLA policy with a language-unconditioned Vision-Action (VA) module, enabling counterfactual comparison during action selection. This design reduces reliance on visual shortcuts, improves robustness on under-observed tasks, and requires neither additional demonstrations nor modifications to existing architectures or pretrained models. Extensive experiments demonstrate its plug-and-play integration across diverse VLAs and consistent improvements. For example, on LIBERO-CF, CAG improves π0.5 by 9.7% in language following accuracy and 3.6% in task success on under-observed tasks using a training-free strategy, with further gains of 15.5% and 8.5%, respectively, when paired with a VA model. In real-world evaluations, CAG reduces counterfactual failures of 9.4% and improves task success by 17.2% on average.
Vision-language-action (VLA) models have improved the flexibility and generality of robotic manipulation, yet they remain fragile to online disruptions, such as changes in task goal, scene configuration, or robot state. Existing recovery methods often require failure data, policy retraining, or external corrective agents, introducing additional data requirements and execution risks. We propose Counterfactual Realignment (CoRe), a training-free framework that recovers a frozen VLA at inference time without failure data. Upon detecting a deviation, CoRe imagines how the policy would continue toward the current goal from a recent viable state, using synthesized observations in place of physical execution, and then minimally realigns the robot and scene to rejoin this imagined continuation before returning control to the policy. Recovery is therefore planned without physical trial-and-error, preserves completed task progress, and handles both mid-episode instruction changes and physical perturbations in a unified manner. Extensive experiments across multiple simulators, VLA backbones, and real-world settings show that CoRe improves success rates by up to 85.0 percentage points to near-nominal levels while reducing physical restorations by 42.2%, without policy fine-tuning or failure-specific recovery training.
Yanyan Zhang, Disheng Liu, Kai Ye +6
Case Western Reserve University Cleveland, OH, USA
Vision-Language-Action (VLA) models enable robots to perform manipulation tasks directly from natural language instructions and are increasingly viewed as a foundation for generalist robotic policies. However, their reliability under Out-of-Distribution (OOD) instructions remains underexplored. In this paper, we reveal a critical failure mode in which VLA policies continue executing visually plausible actions even when the language instruction contradicts the scene. We refer to this phenomenon as linguistic blindness, where VLA policies prioritize visual priors over instruction semantics during action generation. To systematically analyze this issue, we introduce ICBench, a diagnostic benchmark constructed from the LIBERO dataset that probes language-action coupling by injecting controlled OOD instruction contradictions while keeping the visual environment unchanged. Evaluations on three representative VLA architectures, including Pi0, Pi0.5 and OpenVLA OFT, show that these models frequently succeed at tasks despite logically impossible instructions, revealing a strong visual bias in action generation. To mitigate this issue, we propose Instruction-Guided Attention Recalibration (IGAR), a train-free inference-time mechanism that rebalances attention distributions to restore the influence of language instructions. IGAR operates without retraining or architectural modification and can be directly applied to existing VLA models. Experiments across 30 LIBERO tasks demonstrate that IGAR substantially reduces erroneous execution under OOD contradictory instructions while preserving baseline task performance. We additionally validate the approach on a real Franka robotic arm, where IGAR effectively prevents manipulation triggered by inconsistent instructions.
Ninghao Zhang, Bin Zhu, Shijie Zhou +1
Tsinghua University · Singapore Management University · Institute of Trustworthy Embodied AI, Fudan University +1