Vision-language-action (VLA) models built on pretrained vision-language models have demonstrated strong performance across diverse robotic manipulation tasks. However, VLA models that directly map current 2D observations to actions often lack sufficient spatial and temporal understanding, limiting their performance in precise and long-horizon manipulation. Recent methods enhance VLA models through geometric supervision and future-state prediction across the entire scene. However, these methods can suffer from redundant scene information, distracting the model from learning the geometry and dynamics relevant to the current interaction. To address this issue, we propose FOCAL-VLA, a framework that combines subtask-guided geometry distillation with implicit world modeling to learn representations of current spatial structure and future interaction dynamics. To focus geometric learning on the current subtask, we transfer geometric knowledge from VGGT to the VLA model by aligning geometry latents with features from subtask-relevant image regions. To capture the future 3D evolution of the current interaction, we incorporate implicit world modeling using Track4World features from current and future demonstration frames. The two complementary representations jointly guide action generation without running VGGT or Track4World at inference time. Experiments show that FOCAL-VLA outperforms baselines on both simulation benchmarks and real-world manipulation tasks. Project website: https://zhiyuan-gao.github.io/FOCAL-VLA/.
Figures & tables
Fig. 1: Overview of FOCAL-VLA. The π0.5 backbone processes current observations, the task instruction, proprioceptive state, and two groups of learnable queries. The output hidden states at the query positions form geometry and future motion latents, which jointly condition the action expert alongside the native vision–language context. Geometry distillation and implicit world modeling use cached, subtask-relevant targets constructed from frozen VGGT and Track4World features, respectively (Fig. 2 ). An auxiliary language-modeling branch predicts subtask text through the native LM head. Dashed borders indicate training-only branches; distillation prediction heads are omitted for clarity. The motion thumbnails show enlarged regions at t and t+H , where H is the action horizon; Track4World receives complete frames.
Fig. 2: Construction of subtask-relevant geometry and future motion targets. (a–b) PICK and CLOSE from the same demonstration select different entities: the black bowl and the bottom drawer, respectively. (c) VGGT features are pooled within the selected regions in each current view and averaged across valid views to form the geometry target. (d) Track4World processes current and future frames over the action horizon H . The current subtask mask selects features at time t , enriched by future context, for motion-target pooling. Mask overlays indicate pooling regions; both teachers receive complete images. Color blocks show standardized cached target vectors on a shared [−4,4] scale, not spatial maps; feature stacks are schematic.
Method
Spatial
Object
Goal
Long
Avg.
OpenVLA [ 2 ]
84.7
88.4
79.2
53.7
76.5
π0 [ 3 ]
96.8
98.8
95.8
85.2
94.2
π0.5 [ 4 ]
98.8
98.2
98.0
92.4
96.9
OpenVLA-OFT [ 39 ]
97.6
98.4
97.9
94.5
97.1
GR00T N1.7 [ 40 ]
95.0
100.0
98.0
93.0
96.5
SpatialVLA [ 12 ]
88.2
89.9
78.6
55.5
78.1
TABLE I: Comparison on LIBERO (success rate, %). Bold indicates the best result in each column.
Fig. 3: Real-world platform with a Franka Emika Panda and a fixed external camera. The workspace contains the objects used across the four manipulation tasks.
Fig. 4: Illustration of the four real-world tasks. Representative frames from each task are shown in temporal order from left to right, viewed from the fixed external policy camera. Task descriptions appear below each sequence. A uniform color correction is applied for display.
Method
Avg. SR (%)
Base VLAs
π0 [ 19 ]
38.8
π0.5 [ 4 ]
55.2
OpenVLA-OFT [ 19 ]
33.1
Being-H0.5 [ 44 ]
53.9
3D inputs required at inference
TABLE II: Comparison on RoboCasa. The best success rate is shown in bold.
Task
π0.5
FOCAL-VLA
Two-fruit rearrangement
25.0
50.0
Drawer opening
70.0
80.0
Drawer storage
20.0
40.0
Close the microwave
75.0
85.0
Overall
47.5
63.8
TABLE III: Real-world task success rates (%). Each method is evaluated in 20 trials per task.
Harbin Institute of Technology, Shenzhen, Shenzhen 518055, China · Great Bay University, Dongguan 523000, China · Information Hub, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou 511400, China +1
Nanjing University, Nanjing, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China · University of Chinese Academy of Sciences, Beijing, China