Models for physical AI tasks, such as robotics and autonomous driving, need not reason solely in language. They can use visual plans--points, bounding boxes, and trajectories--to represent spatial goals and anticipate motion. However, autoregressive vision-language models generate these plans one coordinate token at a time, making richer predictions increasingly costly. We introduce VFlash, a method for fast, accurate visual planning through Flash Primitive Decoding. VFlash exploits the known structure of visual primitives to predict their coordinates jointly, using a lightweight drafter that generates an entire primitive in one forward pass. The layout adapts to the primitive's type, dimensionality, and length, supporting compact boxes and long, variable-length trajectories. Crucially, spatial accuracy need not require reproducing an autoregressive model's exact token sequence. VFlash therefore trains on annotated primitives and directly accepts structurally valid predictions without target-token agreement. Across the evaluated 2D and 3D robot trajectory tasks, VFlash consistently reduces prediction errors relative to autoregressive decoding. Preliminary single-rollout measurements indicate approximately 25x faster generation for 3D point motion forecasting. A world action model conditioned on VFlash plans achieves 70.2% average success across four MolmoSpaces task suites, compared with 62.6% for the baseline.
Figures & tables
Figure 1: VFlash is a generalist decoding framework for fast, accurate visual planning across Physical AI tasks such as robotics and autonomous driving. By aligning prediction blocks with the known structure of visual primitives, it enables parallel decoding of common structured outputs such as bounding boxes, variable-length 2D/3D trajectories, and dense point motion, while retaining verified speculative decoding for text and sub-task plans. Compared with autoregressive (AR) decoding, VFlash improves accuracy on 3D robot trajectory planning while substantially reducing decoding latency across NVIDIA platforms, including Jetson Thor, DGX Spark, and H100.
Figure 2: FPD design. A primitive-start token triggers a prediction block layout determined by the primitive type and the preceding tool calling. By adapting the prediction block to each primitive’s structure, VFlash supports different spatial formats and lengths without requiring token-level agreement with the VLM.
Primitive
Layout
B-box
<box> u1v1u2v2 </>
2D+3D traj.
<2D> (uv)×P </> <3D> (xyz)×P </>
Dense motion
<Dense> t1t2[k(sd1⋯d5)×3]×P </>
Table 1: Primitive layouts. Each block begins with the start token a0 . P denotes the number of points; (u,v) and (x,y,z) denote image-plane and metric coordinates in special tokens.
Table 4
InternA1 ( 2026 )
AgiBot ( 2025 )
MolmoBot ( 2026 )
Training Variant
Decoder
ADE
FDE
SR@5
SR@10
ADE
FDE
SR@5
SR@10
ADE
FDE
SR@5
SR@10
AR
0.101
0.155
0.412
0.658
0.083
0.125
0.503
0.799
0.066
0.104
0.526
0.804
3D-only
VFlash
0.065
0.098
0.638
0.820
0.042
0.067
0.731
0.914
0.054
0.083
0.653
0.870
AR
0.153
0.233
0.273
0.473
0.105
0.151
0.398
0.698
0.076
0.116
0.467
0.767
2D+3D
VFlash
0.062
0.091
0.640
0.827
0.041
0.064
0.739
0.919
0.055
0.085
0.640
0.862
AR
0.130
0.196
0.360
0.554
0.084
0.129
0.456
0.749
0.066
0.102
0.547
0.813
Table 4: Open-loop 3D robot trajectory prediction. ADE and FDE in meters ( ↓ ); SR@5 and SR@10 are the fraction of waypoints within 5 cm and 10 cm ( ↑ ). The three variants differ in what the model generate (3D only, 2D+3D) and whether wrist-camera views were added in training. All groups are evaluated with the same single-view inputs. Bold marks the best value per column.
Input
HOT3D
WorldTrack
DAVIS
Method
Frames
Text
ADE ↓
FDE ↓
PWT ↑
ADE ↓
FDE ↓
PWT ↑
ADE ↓
FDE ↓
PWT ↑
ObjectForesight ( 2026 )
3
No
0.129
0.192
0.353
—
—
—
—
—
—
EgoScaler ( 2025 )
1
Yes
0.170
0.179
0.218
—
—
—
—
—
—
Robot4DGen ( 2026d )
3
No
0.212
0.271
0.112
0.548
0.704
0.121
2.120
3.382
0.081
Track2Act ( 2024 )
1
No
0.294
0.413
0.202
1.230
1.567
0.053
4.853
8.110
0.018
MolmoMotion-FM ( 2026a )
3
Yes
0.135
0.255
0.382
0.158
0.295
0.438
1.480
2.520
0.130
Table 5: 3D point motion forecasting on PointMotionBench. VFlash uses MolmoMotion-AR ( Zhang et al., 2026a ) as its target. VFlash achieves approximately 25× faster single-rollout generation than AR (Appendix F.3 ). ADE/FDE are in meters ( ↓ ); Frames and Text indicate observation history and language input. Bold and underline mark the best and second-best values.
Table 7
Figure 3: LIBERO family. Average success (%) of the same PIv3 head conditioned on a frozen Qwen3-VL-4B (Baseline), on our VLM’s states up to the generated subtask (w/o Visual Primitives), and on the full visual plan (VFlash).
Method
RL
NC ↑
DAC ↑
EP ↑
TTC ↑
Comf. ↑
PDMS ↑
Hydra-MDP ( 2024b )
–
98.3
96.0
78.7
94.6
100.0
86.5
DiffusionDrive ( 2025 )
–
98.2
96.2
82.2
94.7
100.0
88.1
ReCogDrive ( 2026 )
–
98.3
95.1
81.1
94.3
100.0
86.8
ReCogDrive ( 2026 )
✓
98.2
97.8
83.5
95.2
99.8
89.6
SpanVLA ( 2026c )
✓
99.1
97.1
86.3
95.2
100.0
90.3
ExploreVLA ( 2026 )
✓
98.8
98.4
83.5
96.5
99.9
90.4
Table 8: Autonomous driving results on NAVSIM (left) and the Waymo test split (right). RL indicates reinforcement learning; ↑ / ↓ denote higher / lower is better.
Figure 4: VFlash latency across different downstream tasks on Jetson Thor, DGX Spark and H100.
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Primitive
Layout
Length
Bounding box
<box> u1v1u2v2 </box>
6
Robot 3D
<robot_single> (xyz)P </robot>
3P+2
Robot 2D+3D
<robot_single_2d> (uv)P </robot>
5P+4
<robot_single> (xyz)P </robot>
Dual-arm 3D / 2D+3D
left-arm block, then right-arm block
6P+4 / 10P+8
Driving 3D
<drive> (xy)P </drive>
2P+2
Appendix
Table 9: Primitive specifications for our base VLM. P is the number of waypoints per arm or vehicle. Block length includes the start token and all delimiters. Dual-arm blocks repeat the single-arm layout for the left and then the right arm, with left / right replacing single in the start tokens.
Task
AR updates
AR peak LR
Draft updates
Draft peak LR
Draft batch
2D trajectories
10,800
2×10−6
7,142
2×10−4
64
Robot 3D-only
10,317
2×10−6
30,562
6×10−4
512
Robot 2D+3D
11,207
2×10−6
30,562
6×10−4
512
Robot 2D+3D, wrist camera
26,245
2×10−6
46,306
6×10−4
512
Driving 3D-only
13,740
2×10−6
4,028
6×10−4
512
Driving 2D+3D
13,740
2×10−6
4,028
6×10−4
512
Appendix
Table 10: Task adaptation budgets. AR columns describe full target fine-tuning; draft columns describe drafter training on the frozen target. Batch is the number of examples per draft update.
Figure 5: Waymo planning latency on H100 (ms, batch size one). AR, DFlash, and VFlash share the VLM, primitive, and action head and differ only in the decoder. Qwen-Drive-1.0 is measured on the same hardware; the Fast-dDrive value is from its paper and its AR baseline is not the AR bar shown here.
Device
Method
Prepro.
ViT
Prefill
Decoding
Action
Total
MolmoSpaces
Thor
AR
19.1
24.4
61.8
3,107.6
339.8
3,552.7
DFlash
1,383.3
1,828.4
VFlash
258.2
703.3
Spark
AR
18.2
19.0
53.4
2,950.9
315.0
3,356.5
DFlash
1,403.6
1,809.2
Appendix
Table 11: Per-step latency by stage (ms). Batch size one, FP16 VLM engines. Action is WAN encoder plus DiT for MolmoSpaces and the ReCogDrive DiT (ten DDIM steps) or the Qwen-Drive planner for driving. Within each task and device, vision, prefill, and action means are shared across AR, DFlash, and VFlash and shown once; decoding is profiled separately and for VFlash includes the target prefill over the accepted primitive. Total is the sum of stage means. Qwen-Drive-1.0 is measured end to end on its own stack (Appendix E.1 ); Fast-dDrive is reported by its paper without a stage split.
Long-horizon robot planning requires jointly reasoning over semantic task structure and geometric feasibility. To successfully execute a task, a robot must decompose goals, select task-relevant objects, and sequence actions, while ensuring that plans satisfy spatial constraints such as limited free space and object collisions. In this work, we propose APIVOT, a VLM-based planner that adaptively interleaves language and visual thoughts for long-horizon planning. APIVOT learns to leverage language for semantic reasoning, while using visual thoughts as imagined future states for internal verification of geometric feasibility. On long-horizon kitchen tasks, APIVOT outperforms general-purpose VLMs and prior planning frameworks, achieving the largest gains in spatially constrained settings. We find that APIVOT learns meaningful modality selection behavior, demonstrating that adaptive interleaving of vision-language thoughts improves both planning success and reasoning efficiency.
Vision-Language-Action (VLA) models typically map visual observations and linguistic instructions directly to control signals. This "black-box" mapping forces a single forward pass to simultaneously handle instruction interpretation, spatial grounding, and low-level control, often leading to poor spatial precision and limited robustness in out-of-distribution scenarios. To address these limitations, we propose VP-VLA, a dual-system framework that decouples high-level reasoning and low-level execution via a structured visual prompting interface. Specifically, a "System 2 Planner" decomposes complex instructions into sub-tasks and identifies relevant target objects and goal locations. These spatial anchors are rendered directly within the native RGB observation space as modality-consistent visual prompts, such as crosshairs and bounding boxes. This avoids the modality mismatch introduced by dense masks, affordance maps, or additional control-specific representations. Guided by these prompts and enhanced by a novel auxiliary visual grounding objective during training, a "System 1 Controller" reliably generates precise low-level execution motions. Extensive experiments in simulation and real world demonstrate that VP-VLA surpasses state-of-the-art end-to-end baselines including QwenOFT and GR00T-N1.6. Project page: https://visualprompt-vla.github.io/
Most Vision-Language-Action (VLA) models map observations directly to actions without explicit reasoning, limiting their capacity for reasoning-intensive long-horizon tasks. To address this, existing approaches adopt Chain-of-Thought (CoT) reasoning to enable subgoal decomposition and spatial anticipation. However, those methods lack a unified architecture for effective cross-modal reasoning and fail to explicitly include inverse reasoning ability based on the target state. We argue that manipulation planning naturally decomposes into prediction, anticipating the next visual state, and inverse dynamics, inferring the actions to reach it. Bridging both requires a unified autoregressive architecture that interleaves textual and visual reasoning in a single generation process. We propose ThinkingVLA, a generative model that realizes this decomposition within a unified Mixture-of-Transformers architecture. ThinkingVLA consists of a forward CoT that identifies the immediate subgoal and guides the visual forecasting; the predicted image then serves as the target state, grounding an inverse CoT that reasons about spatial relationships and action intent based on the predicted image; and the final action is generated conditioned on this full reasoning context. Extensive experiments on simulation and real-world benchmarks demonstrate that ThinkingVLA consistently outperforms state-of-the-art baselines, with particularly large gains on long-horizon manipulation tasks.
Tianyi Lu, Hui Zhang, Zijie Diao +8
Fudan University · Shanghai Innovation Institue · China Unicom +1