Instruction Generalization in VLA Models

VLA: Vision-Language-Action

Vision-language-action models must connect a command to what they see and then choose actions, even when wording, objects, locations, or task combinations differ from training. Researchers test this with paraphrase and language-variation benchmarks such as LIBERO-Para, as well as held-out object–location pairings and real-robot tasks. Approaches include synthetic instruction augmentation, counterfactual training, explicit grounding and subgoal planning, and test-time guidance. These methods can improve robustness, but models still show brittle language grounding and visual shortcuts; greater responsiveness to instructions does not always mean better task success or transfer to harder commands.

Latest papers

40 papers18 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

DISC: Decoupling Instruction from State-Conditioned Control via Policy Generation
May 20, 2026Hanxiang Ren, Pei Zhou, Xunzhe Zhou +1Language-Conditioned Robot Manipulation Visuomotor Policy Learning
‡Zhejiang University §The University of Hong Kong ¶TranscEngram
Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Training
Apr 25, 2026Suning Huang, Jiaqi Shao, Ke Wang +5VLA Model Post-Training Vision-Language Grounding
Stanford University
ProGAL-VLA: Grounded Alignment through Prospective Reasoning in Vision-Language-Action Models
Apr 10, 2026Nastaran Darabi, Amit Ranjan TrivediSpatial Grounding for VLA Models Vision-Language-Action Models
University of Illinois Chicago, IL, USA
LIBERO-Para: A Diagnostic Benchmark and Metrics for Paraphrase Robustness in VLA Models
Mar 30, 2026Chanyoung Kim, Minwoo Kim, Minseok Kang +2Vision-Language Grounding Vision-Language-Action Models
School of Computer Science and Engineering, Soongsil University, Seoul, Korea · Department of Artificial Intelligence, Chung-Ang University, Seoul, Korea
Sparse Autoencoders Reveal Interpretable and Steerable Features in VLA Models
Mar 19, 2026Aiden Swann, Lachlain McGranahan, Hugo Buurmeijer +2Vision-Language-Action Models Instruction Generalization in VLA Models
Department of Mechanical Engineering, Stanford University · Department of Aeronautics & Astronautics, Stanford University · Department of Computer Science, Stanford University
Enhancing Linguistic Generalization of VLA: Fine-Tuning OpenVLA via Synthetic Instruction Augmentation
Mar 17, 2026Dongik ShinSynthetic Instruction Generation Vision-Language-Action Models
Red-Teaming Vision-Language-Action Models via Quality Diversity Prompt Generation for Robust Robot Policies
Mar 12, 2026Siddharth Srikanth, Freddie Liang, Ya-Chuan Hsu +9Robustness of VLA Models Adversarial Attacks on VLA Models
Thomas Lord Department of Computer Science, University of Southern California · Sony AI · Sibley School of Mechanical and Aerospace Engineering, Cornell University.
See, Plan, Rewind: Progress-Aware Vision-Language-Action Models for Robust Robotic Manipulation
Mar 10, 2026Tingjun Dai, Mingfei Han, Tingwen Du +6Vision-Language-Action Models Language-Conditioned Robot Manipulation
School of Information Science and Technology, University of Science and Technology of China, Anhui, China. · University of Technology Sydney, Ultimo, NSW, Australia. · Department of Computer Vision, Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates. +3
LangGap: Diagnosing and Closing the Language Gap in Vision-Language-Action Models
Feb 28, 2026Yuchen Hou, Lin ZhaoVision-Language-Action Models Instruction Generalization in VLA Models
Department of Electrical and Computer Engineering, National University of Singapore, Singapore
Robust Skills, Brittle Grounding: Diagnosing Restricted Generalization in Vision-Language Action Policies via Multi-Object Picking
Feb 27, 2026David Emukpere, Romain Deffayet, Jean-Michel RendersInstruction Generalization in VLA Models Generalization in Robotic Manipulation
Naver Labs Europe, 6 Chem. de Maupertuis, Meylan, France.

May 20, 2026J/K move · Enter open · S save

DISC: Decoupling Instruction from State-Conditioned Control via Policy Generation

Hanxiang Ren, Pei Zhou, Xunzhe Zhou, Yanchao Yang

‡Zhejiang University §The University of Hong Kong ¶TranscEngram

Language-Conditioned Robot Manipulation Visuomotor Policy Learning AI Control Hypernetworks Policy Learning Instruction Generalization in VLA Models

Open paper AskPDF arXiv

Language-conditioned manipulation policies typically process instructions and observations through shared network parameters. This task-state entanglement provides a pathway for observation leakage -- networks learn scene-to-action shortcuts that bypass language grounding entirely. DISC eliminates this failure structurally. Rather than conditioning a universal policy on language, DISC uses a hypernetwork to generate the entire parameter set of a task-specific visuomotor policy from the instruction alone. The generated policy never directly accesses language; therefore, its task-awareness must come from the language. Consequently, observation leakage has no pathway to emerge. On the other hand, generating coherent high-dimensional policy weights is itself a challenging problem. We address it with a two-stage hypernetwork whose refinement stage embeds the structure of gradient-based optimization as a feed-forward inductive bias, producing globally consistent parameters without actual gradient computation. Trained entirely from scratch on standard data budgets, DISC outperforms all entangled baselines on LIBERO-90 and Meta-World, with advantages that widen on complex, long-horizon tasks -- and surpasses the large-scale pretrained

π_0

despite using no external pretraining data. On a real-world benchmark where all tasks share identical visual context, DISC substantially outperforms entangled alternatives, directly confirming that language-generated policy parameters, not visual shortcuts, drive behavior. The hypernetwork further learns a semantically structured parameter manifold that enables few-shot adaptation from minimal demonstrations and robust generalization across paraphrased instructions. Our code is available at: {https://github.com/ReNginx/DISC}.