cs.ROAug 29, 2026

Brain-Language-Action (BLA) Models: Language-Conditioned EEG for Robotics Control

Authors: Alexandr Plashchinsky

Organizations: VECTOR Labs San Francisco Bay Area

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Electroencephalography (EEG)-based robotic control is commonly formulated as a direct classification problem, in which electrical neural signals are mapped to a fixed set of discrete actions. However, the limited separability and high noise of EEG signals make it difficult to scale this approach to fine-grained robotic control spaces. We introduce Brain-Language-Action (BLA) models, a framework in which language conditions the interpretation of neural representations for robotic action generation. In a BLA, a small set of reliably distinguishable brain states can be dynamically associated with different actions through a language-defined control mapping, allowing a small number of neural classes to apply to a larger global action space. We develop a proof-of-concept BLA for drone control using motor-imagery EEG from the BCI Competition IV 2a dataset. The system is trained in two stages. First, we evaluate multiple candidate EEG encoder architectures using subject-specific four-class motor-imagery classification, converting 250Hz, 3.5-second, 22-channel EEG samples into five 128-dimensional brain-token embeddings. Second, these embeddings are projected into the embedding space of a pretrained large language model (LLM) and jointly fine-tuned with language instructions to autoregressively generate structured three-token drone actions. Across 840 possible language-defined mappings between four neural states and seven flight action combinations, the resulting BLA achieves 90% per-token accuracy during evaluation. These results provide an initial demonstration that language conditioning can expand the effective control range of EEG-based robotic interfaces without requiring a corresponding increase in the number of directly distinguishable neural states.

Figures & tables

ActionHorizontalYawVertical
ForwardACT_2ACT_1ACT_1
BackwardACT_0ACT_1ACT_1
Turn leftACT_1ACT_2ACT_1
Turn rightACT_1ACT_0ACT_1
AscendACT_1ACT_1ACT_2
DescendACT_1ACT_1ACT_0
Table 1: Structured action-token representation used for drone control.
Figure 1: Average motor-imagery classification accuracy for each candidate brain encoder, averaged across subject-specific model variants. The EEGInceptionEncoder achieved the strongest overall performance.
Figure 2: Best subject-specific EEGInceptionEncoder results. Subjects 1, 3, 7, and 8 were selected for downstream BLA fine-tuning.
Figure 3: Per-token evaluation accuracy of the four subject-specific BLA models. The average per-token BLA evaluation accuracy across subjects was 0.904.
Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

SettingEncoder PretrainingBLA Fine-Tuning
DatasetBCI Competition IV 2aBCI Competition IV 2a
Subjects1–91, 3, 7, 8
EEG Input875×22875\times 22875×22875\times 22
Brain EncoderCandidate architecturesEEG-Inception
Brain Embeddings5×1285\times 1285×1285\times 128
Projection–128→1024128\rightarrow 1024 MLP
Appendix
Table 2: Experimental configuration for the two training stages.
More in this topic

Language-Conditioned Robot Control

Language-conditioned robot control connects natural-language instructions to grounded, safe actions across tasks and robot embodiments.

See all 74 papers

Explore similar work

CardsList
Feb 23, 2026

EEG-Driven Intention Decoding: Offline Deep Learning Benchmarking on a Robotic Rover

Brain-computer interfaces (BCIs) provide a hands-free control modality for mobile robotics, yet decoding user intent during real-world navigation remains challenging. This work presents a brain-robot control framework for offline decoding of driving commands during robotic rover operation. A 4WD Rover Pro platform was remotely operated by 12 participants who navigated a predefined route using a joystick, executing the commands forward, reverse, left, right, and stop. Electroencephalogram (EEG) signals were recorded with a 16-channel OpenBCI cap and aligned with motor actions at Delta = 0 ms and future prediction horizons (Delta > 0 ms). After preprocessing, several deep learning models were benchmarked, including convolutional neural networks, recurrent neural networks, and Transformer architectures. ShallowConvNet achieved the highest performance for both action prediction and intent prediction. By combining real-world robotic control with multi-horizon EEG intention decoding, this study introduces a reproducible benchmark and reveals key design insights for predictive deep learning-based BCI systems.

Ghadah Alosaimi, Maha Alsayyari, Yixin Sun +3

Department of Computer Science, Imam Mohammad Ibn Saud Islamic University, Saudi Arabia · Department of Computer Science, Durham University, UK · Department of Computer Science, King Saud University, Saudi Arabia +1

Brain-Computer InterfacesEEG DecodingRobotic ControlMobile Robotics
AskPDF
Sep 28, 2026

Brain-Conditioned Action Policies for Neural Motor Decoding

Motor brain-computer interfaces (BCIs) aim to decode motor intention, enabling people with paralysis to control external devices. Neural motor decoding typically learns task-specific mappings from neural activity to kinematics, yet remains constrained by scarce paired neural-action data. We propose BrainVLA, a framework that enables neural motor decoding by drawing on a pretrained vision-language-action (VLA) model through language-mediated alignment. BrainVLA mitigates reliance on scarce paired neural-action data by leveraging VLA policies. We first construct VLA-compatible datasets including paired neural activity, action signals, language instructions, and rendered visual observations. Then, we adapt the OpenVLA-OFT policy to the target action spaces through LoRA fine-tuning. To establish an effective interface through which neural activity can convey motor intention to adapted VLA policies and guide action generation, we train a neural encoder via neural-language alignment, using language representations as semantic targets to capture latent motor intent from neural activity. The resulting neural representations serve as an endogenous intention signal to guide VLA policies to generate executable actions, while visual observations provide complementary information about the evolving task state. BrainVLA is evaluated on two neural motor datasets with different action dimensionalities using causal rollout decoding. It outperforms the evaluated baselines in cross-session decoding R2R^2 and task success rate, while demonstrating high training data efficiency. These results establish a route for neural motor decoding to draw on large-scale robotic priors through brain-conditioned VLA policies.

Luyao Jin, Running Zhao, Huan Zhao +2

Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong · Department of Electrical and Computer Engineering, University of Hong Kong · School of Biomedical Sciences, The Chinese University of Hong Kong

Brain-Computer InterfacesNeural DecodingVision-Language-Action Models
AskPDF
Apr 30, 2026

Motubrain: An Advanced World Action Model for Robot Control

Vision-Language-Action (VLA) models generalize semantically well but often lack fine-grained modeling of world dynamics. We present Motubrain, a unified World Action Model that jointly models video and action under a UniDiffuser formulation with a three-stream Mixture-of-Transformers architecture. A single model supports policy learning, world modeling, video generation, inverse dynamics, and joint video-action prediction, while scaling to heterogeneous multimodal data such as video-only, task-agnostic, and cross-embodiment robot data. Building on Motus, Motubrain further introduces unified multiview modeling, an independent text stream for stronger language-action coupling, a shared cross-embodiment action representation, and an efficient post-training and deployment recipe for long-horizon real-world control. Our inference stack combines step reduction, compilation, FP8 quantization, DiT caching, V2A-style action-only inference, and real-time chunked closed-loop execution, achieving over 50x speedup over a naive baseline and up to 11 Hz inference. Experimentally, Motubrain achieves 95.8% and 96.1% average success on RoboTwin 2.0 under clean and randomized settings, respectively, attains the strongest reported EWMScore in our WorldArena comparison, and adapts to new humanoid embodiments with only 50--100 trajectories. These results show that unified world action models can scale in generality, predictive accuracy, and real-world deployability.

Motubrain Team, Chendong Xiang, Fan Bao +17

Motubrain Team

Robotic ControlWorld Models for RoboticsAction-Conditioned World ModelsEfficient VLA Model InferenceCross-Embodiment Robot LearningEfficient VLA Models+4
AskPDF