Vision-Language-Action (VLA) models have shown strong performance in robotic manipulation, but often struggle in long-horizon or out-of-distribution scenarios due to the lack of explicit mechanisms for multimodal reasoning and anticipating how the world will evolve under action. Recent works introduce textual chain-of-thought or visual subgoal prediction within VLA models to reason, but still fail to offer a unified human-like reasoning framework for joint textual reasoning, visual foresight, and action prediction. To this end, we propose HALO, a unified VLA model that enables embodied multimodal chain-of-thought (EM-CoT) reasoning through a sequential process of textual task reasoning, visual subgoal prediction for fine-grained guidance, and EM-CoT-augmented action prediction. We instantiate HALO with a Mixture-of-Transformers (MoT) architecture that decouples semantic reasoning, visual foresight, and action prediction into specialized experts while allowing seamless cross-expert collaboration. To enable HALO learning at scale, we introduce an automated pipeline to synthesize EM-CoT training data along with a carefully crafted training recipe. Extensive experiments demonstrate that: (1) HALO achieves superior performance in both simulated and real-world environments, surpassing baseline policy pi_0 by 34.1% on RoboTwin benchmark; (2) all proposed components of the training recipe and EM-CoT design help improve task success rate; and (3) HALO exhibits strong generalization capabilities under aggressive unseen environmental randomization with our proposed EM-CoT reasoning.
Figures & tables
Figure 1: Halo first performs textual reasoning and task planning, then predicts visual subgoals for fine-grained guidance, and finally generates actions conditioned on EM-CoT. This process is implemented using a Mixture-of-Transformers (MoT) architecture that integrates a multimodal understanding expert, a visual generation expert, and an action prediction expert together through shared self-attention.
Figure 2: Overview of EM-CoT Data Synthesis Pipeline. The pipeline converts raw robotic trajectories into EM-CoT data in three phases: (1) action primitives are extracted from robot proprioception via rule-based matching; (2) a VLM acts as an annotator to generate task plans, decompose trajectories into subtasks, and align each subtask with explicit textual reasoning; and (3) the terminal frame of each subtask is selected as a visual subgoal image, producing structured embodied multimodal chain-of-thought supervision.
Figure 3
Task
π0
RDT-1B
Diffusion Policy
Halo -w/o EM-CoT
Halo
Easy
Hard
Easy
Hard
Easy
Hard
Easy
Hard
Easy
Hard
Adjust Bottle
90
56
81
75
97
0
99
2
97
9
Beat Block Hammer
43
21
77
37
42
0
95
3
96
11
Blocks Ranking RGB
19
5
3
0
0
0
85
3
94
7
Blocks Ranking Size
7
1
0
0
1
0
37
0
58
2
Click Alarmclock
63
11
61
12
61
5
85
10
83
14
Table 1: Quantitative results in simulation on RoboTwin 2.0. Each task is evaluated 100 times under the Easy (Clean) and Hard (Domain-randomized) settings, and the average success rate across 50 tasks is reported. The complete results are provided in Appendix D .
Panel A: Training Recipe
Panel B: EM-CoT
Setting
Easy
Hard
Setting
Easy
Hard
Full
75.3
21.2
Halo
80.5
26.4
w/o V
58.2
10.5
w/o T
77.8
18.3
w/o V+T
42.9
3.9
w/o V
76.1
22.5
w/o V+T+A
32.4
0.0
w/o V & T
75.3
21.2
Table 2: Ablation studies on training recipe and EM-CoT. Panel A: training recipe ablation (V/T/A denote visual generation, textual VQA, and action prediction data. Note that in the Full setting, we pre-train the model with V&T&A datasets and then fine-tune it on the official RoboTwin 2.0 dataset). Panel B: EM-CoT ablation (V = visual subgoal images, T = textual reasoning).
Figure 6
Figure 7: Quantitative results of real-world experiments on both the basic setting and the generalization setting. Each task is evaluated over 50 trials, and the success rate is reported.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Parameter
Value
Parameter
Value
architectures
[”Qwen2ForCausalLM”]
num_attention_heads
12
attention_dropout
0.0
num_hidden_layers
28
bos_token_id
151643
num_key_value_heads
2
eos_token_id
151645
rms_norm_eps
1×10−6
hidden_act
”silu”
rope_scaling
null
hidden_size
1536
rope_theta
1000000.0
Appendix
Table 3: Qwen2.5-1.5B-Instruct Model Configuration Parameters
Configuration
Pre-training
Fine-tuning
Base Architecture
Qwen2.5-1.5B × 3 Experts
Total Parameters
≈ 4.5B
Optimizer
AdamW
Learning Rate
1×10−4
5×10−5
Learning Rate Schedule
Constant
Constant
Weight Decay
0.0
0.0
Appendix
Table 4: Training Hyperparameters. Detailed configurations for the two-stage training process of Halo .
Task
π0
RDT-1B
Diffusion Policy
Halo -w/o CoT
Halo -w/ CoT
Easy
Hard
Easy
Hard
Easy
Hard
Easy
Hard
Easy
Hard
Adjust Bottle
90
56
81
75
97
0
99
2
97
9
Beat Block Hammer
43
21
77
37
42
0
95
3
96
11
Blocks Ranking RGB
19
5
3
0
0
0
85
3
94
7
Blocks Ranking Size
7
1
0
0
1
0
37
0
58
2
Click Alarmclock
63
11
61
12
61
5
85
10
83
14
Appendix
Table 5: Complete Result on RoboTwin
Figure 8: Attention map during pre-training phase.