GaussVLA: Geometry-Aware Spatial Reasoning for Vision-Language-Action Model

Md Selim Sarowar; Md Tanvir Islam; Sungho Kim; Sangtae Ahn

cs.ROAug 25, 2026

GaussVLA: Geometry-Aware Spatial Reasoning for Vision-Language-Action Model

Authors: Md Selim Sarowar, Md Tanvir Islam, Sungho Kim, Sangtae Ahn

Organizations: Yeungnam University, Republic of Korea · Kyungpook National University, Republic of Korea

Abstract

Vision-Language-Action (VLA) models encode visual observations as flat 2D patch tokens that carry no intrinsic geometric structure, and augmenting them with dense monocular depth injects per-pixel scalar values that encode neither surface orientation nor geometric confidence. This leaves the policy with limited structured spatial reasoning for action prediction. We propose GaussVLA, a Mamba-based VLA that incorporates two custom modules: Gaussian Spatial Tokenizer (GST) to lift frozen semantic and depth features into compact 3D Gaussian tokens, pools geometrically salient regions with learned queries, and Depth-Aware Chain-of-Thought (DA-CoT) that performs structured, non-autoregressive geometric reasoning under language and flow-time conditioning. Across both simulation and real-world evaluations, GaussVLA demonstrates strong spatial-manipulation performance while remaining parameter-efficient. On LIBERO, it achieves 93.5% average success and 100.0% success on the Spatial suite with only 200M parameters, improving over SpatialVLA by 19.7% relative average success while remaining significantly more parameter-efficient.

Figures & tables

Figure 2: Framework of our proposed GaussVLA for VLA-based Robot Manipulation. A frozen dual-stream encoder (SigLIP and Depth Anything V2) extracts complementary semantic and dense depth features for geometric context. These streams are fused by the Gaussian Spatial Tokenizer (GST), which lifts 2D image evidence into structured 3D Gaussian tokens via Gaussian parameters $(\mu,\sigma,\alpha)$ , forming geometry-aware patch tokens, and then aggregates them through spatial attention pooling. The resulting GST tokens are combined with language, CoT, action, and temporal embeddings, and processed by a shared Mamba-based multimodal backbone with positional encoding. Finally, a DA-CoT layer refines spatial reasoning before the flow-matching action prediction head.

Method	Venue	Spatial	Object	Goal	Long	Average	Parameters
DP [ 6 ]	RSS’23	78.3	92.5	68.3	50.5	72.4	80M
MaIL [ 10 ]	CoRL’24	53.8	81.5	56.3	41.7	58.3	24M
QueST [ 23 ]	NeurIPS’24	89.0	90.0	88.4	87.0	88.6	152M
OpenVLA [ 13 ]	CoRL’24	84.7	88.4	79.2	53.7	76.5	7B
SpatialVLA [ 27 ]	RSS’25	88.2	89.9	78.6	55.5	78.1	4B
ThinkAct [ 8 ]	NeurIPS’25	88.3	91.4	87.1	70.9	84.4	7B

Table 1: Performance comparison on the LIBERO [ 16 ] benchmark. GaussVLA achieves the highest overall average success rate while using substantially fewer parameters than large-scale VLA baselines.

Figure 3: Representative LIBERO [ 16 ] rollouts showing GaussVLA executing tasks from initial state to completion.

Method	Venue	Meta-World					Method	Venue	Tasks completed in a row (CALVIN)					Avg.
Method	Venue	Easy	Mid	Hard	V.Hard	Avg.	Method	Venue	1	2	3	4	5	Length
BC-RNN [ 19 ]	CoRL’21	4.5	3.8	3.2	3.0	3.6	MCIL [ 18 ]	RSS’21	0.37	0.027	0.002	0.000	0.000	0.40
DP [ 6 ]	IJRR’25	83.6	31.1	9.0	26.6	37.6	MT-R3M [ 24 ]	CoRL’22	0.408	0.146	0.043	0.014	0.002	0.61
TinyVLA [ 31 ]	ICRA’25	77.6	21.5	11.4	15.8	31.6	RT-1 [ 1 ]	RSS’23	0.249	0.069	0.015	0.006	0.000	0.34
$\pi_{0}$ [ 4 ]	RSS’26	71.8	48.2	41.7	30.0	47.9	GR-1 [ 2 ]	ICLR’24	0.778	0.533	0.332	0.218	0.139	2.0
GaussVLA	BMVC’26	92.7	47.9	37.3	41.6	54.9	GaussVLA	BMVC’26	0.637	0.367	0.291	0.179	0.000	1.474

Table 2: Comparison with recent imitation learning and VLA baselines on Meta-World [ 21 ] and CALVIN [ 22 ] . The Meta-World panel reports SR across splits by task difficulty. The CALVIN panel reports long-horizon performance as the fraction of rollouts that complete 1-5 consecutive tasks, along with the average length of completed sequences.

Model	LIBERO Goal					LIBERO Spatial					LIBERO 10					LIBERO Object					Avg.
Model	Obj	Pos	Sem	Task	Env	Obj	Pos	Sem	Task	Env	Obj	Pos	Sem	Task	Env	Obj	Pos	Sem	Task	Env	SR $\uparrow$
OVLA [ 13 ]	0.96	0.00	0.98	0.00	0.98	0.97	0.00	0.97	0.00	0.89	0.81	0.00	0.96	0.00	0.85	0.98	0.00	0.98	0.00	0.00	0.52
$\pi_{0.5}$ [ 3 ]	0.97	0.38	0.97	0.00	0.46	0.97	0.20	0.97	0.01	0.46	0.92	0.08	0.93	0.01	0.46	0.98	0.17	0.96	0.01	0.73	0.53
$\pi_{0}$ [ 4 ]	0.94	0.00	0.93	0.00	0.39	0.95	0.00	0.97	0.00	0.60	0.79	0.00	0.82	0.00	0.27	0.94	0.00	0.90	0.00	0.29	0.44
GaussVLA	0.81	0.00	0.70	0.00	0.00	0.93	0.00	0.73	0.00	0.27	0.83	0.00	0.67	0.00	0.00	0.92	0.00	0.63	0.00	0.17	0.33

Table 3: Comparison on LIBERO-PRO [ 41 ] benchmark. Normalized success rates (SR) under five perturbation types. Here, OVLA : OpenVLA [ 13 ] .

Figure 4: Real-world SO-101 evaluation comparing GaussVLA with ACT and SpatialVLA across multi-task settings and Pick-Place ID and OOD trials, alongside rollout frames showing successful execution of a physical stacking task.

Variant	2D Tokens	GST	DA-CoT	LIBERO $\uparrow$	LIBERO-PRO $\uparrow$	Params (M) $\downarrow$	Trainable (M) $\downarrow$	GFLOPs $\downarrow$	Latency (ms) $\downarrow$
Vanilla GaussVLA	✓	✗	✗	78.1	11.2	179	158	3.50	10.85
+ GST only	✗	✓	✗	90.5	29.0	190.2	169.2	4.33	12.27
+ DA-CoT only	✓	✗	✓	82.1	16.7	188.8	167.8	4.00	11.55
Full GaussVLA	✗	✓	✓	93.5	33.3	200	179	4.83	12.97

Table 4: Core contribution and efficiency ablation. Vanilla GaussVLA denotes the same architecture with GST and DA-CoT removed. GST contributes the dominant performance gain, while DA-CoT provides a complementary improvement. Efficiency is measured on a single NVIDIA RTX Pro 6000 Blackwell. Latency is end-to-end, including frozen encoder forwards and the 10-step Euler ODE of the flow-matching action expert.

$N_{g}$	8	32	64	128	256
LIBERO (%) $\uparrow$	86.4	90.7	92.4	93.5	93.6
Latency (ms) $\downarrow$	11.6	12.0	12.4	12.97	14.1
$N_{r}$	1	2	4	8	16
LIBERO (%) $\uparrow$	91.8	92.8	93.5	93.4	92.9
Latency (ms) $\downarrow$	12.7	12.8	12.97	13.2	13.7

Table 5: Sensitivity to GST queries

N_{g}

and DA-CoT queries

N_{r}

. Default

N_{g}{=}128

N_{r}{=}4

is near-optimal on LIBERO and Pareto-optimal on the LIBERO/latency frontier.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

GST		DA-CoT
$N_{g}$ Gaussian queries	128	$N_{r}$ reasoning queries	4
Fourier bands $L$ ( $d_{\gamma}{=}6L$ )	10	Reasoning dim $d_{c}$	512
Workspace radius $\rho$	0.8 m
Backbone		Action expert
Mamba blocks $L_{b}$	5	Action horizon $H$	10
Backbone dim $d_{b}$	512	Action dim $d_{a}$	7

Appendix

Table 6: Architecture and training hyperparameters used for all reported GaussVLA runs.

Component	Trainable	Latency (ms)
Frozen SigLIP-SO400M/14 forward	✗	5.20
Frozen Depth-Anything-V2 (ViT-L)	✗	3.30
GST module (lift + pool, $N_{g}{=}128$ )	✓	1.42
Mamba backbone	✓	1.50
DA-CoT conditioner	✓	0.70
Action decoder	✓	0.25

Appendix

Table 7: End-to-end inference latency on a single NVIDIA RTX Pro 6000 Blackwell with batch size 1 and BF16 precision. All component latencies sum to 12.97 ms per step.

Probe target	Flat 2D patches	Pooled GST	DA-CoT $R_{\tau}$
Object center (3D position)	0.34	0.66	0.78
Gripper–object distance	0.41	0.71	0.82
Object orientation (principal axis)	0.28	0.49	0.61
Surface normal (linear from $\sigma$ )	—	0.67	—

Appendix

Table 8: Linear-probe regression

R^{2}

on LIBERO simulator state. DA-CoT tokens

R_{\tau}

encode richer 3D geometric structure than flat 2D patch tokens or pooled GST representations.

Backbone	Action Decoder	Query Type	Overall Avg. $\uparrow$	Params (M) $\downarrow$	GFLOPs $\downarrow$
Transformer	Standard head	–	67.49	256	9.82
Mamba	Standard head	–	73.36	110	3.28
Mamba	Action decoder	action-token queries	78.1	179	3.50

Appendix

Table 9: Overall Avg. is the LIBERO average. The

+4.74

pt jump from row 2 to row 3 jointly reflects the action-token query decoder design and the

+69

M-Parameters/

+0.22

GFLOPs capacity uplift; we hold the Mamba backbone fixed across the comparison to isolate decoder-family effects from backbone-family effects. Here, row 3 ( Mamba + Action decoder ) corresponds to the Vanilla GaussVLA configuration of Table 4 of the main paper; Adding GST + DA-CoT yields Full GaussVLA at 4.83 GFLOPs (Table 4, main paper).

Visual Representation	Depth	3D Lifting	Gaussian Param.	Spatial Pooling	LIBERO $\uparrow$	Spatial Tasks $\uparrow$	PRO Avg. $\uparrow$
Flat 2D patch tokens	✗	✗	✗	✗	78.1	71.2	11.2
2D patch tokens + depth concat	✓	✗	✗	✗	73.3	78.0	14.9
GST tokens	✓	✓	✓	✓	90.5	100	29.0

Appendix

Table 10: GST design ablation (without DA-CoT). Gains arise from the structured Gaussian representation with confidence-aware pooling, not depth alone: scalar-depth concatenation regresses LIBERO (78.1

\rightarrow

73.3).

Reasoning Variant	Depth-Aware	Structured Tokens	CoT Supervision	LIBERO $\uparrow$	Long-Horizon $\uparrow$	Avg. $\uparrow$
No reasoning module	✗	✗	✗	78.1	75.2	76.7
CoT without depth cues	✗	✓	✓	80.5	87.3	83.9
DA-CoT without supervision	✓	✓	✗	81.5	90.0	85.8
DA-CoT (full)	✓	✓	✓	82.1	91.6	87.0

Appendix

Table 11: Depth-aware grounding is the dominant reasoning ingredient (+3.1 pt on the 2-suite avg, comparing ”CoT without depth cues” 83.9

\rightarrow

”DA-CoT (full)” 87.0). Adding CoT supervision on top of depth-aware structured tokens contributes a complementary boost on both LIBERO (+0.6 pt, 81.5 → 82.1) and Long-Horizon (+1.6 pt, 90.0 → 91.6), for a +1.1 pt 2-suite-avg lift over the no-supervision variant.

Depth-uncertainty proxy	Pearson $\rho(\alpha,-u)$ $\uparrow$	$p$ -value
$u_{\nabla}$ (depth-gradient magnitude)	$+0.61$	$<10^{-6}$
$u_{\sigma_{d}}$ (MC-dropout variance)	$+0.54$	$<10^{-6}$
$u_{\mathrm{pc}}$ (photometric inconsist.)	$+0.47$	$<10^{-6}$

Appendix

Table 12: Pearson correlation between the learned opacity

\alpha

and three depth-uncertainty proxies (sign convention: higher

\alpha

\Leftrightarrow

lower uncertainty). Although

\alpha

is not directly conditioned on any of these proxies, the opacity-weighted GST loss in Eq. 21 in the main paper induces a strong correlation consistent with the ”depth-aware confidence” interpretation.

$H$	1	2	3	4	5	Avg. Len.
10 (default)	0.637	0.367	0.291	0.179	0.000	1.474
20	0.671	0.402	0.318	0.198	0.063	1.65
50	0.598	0.331	0.237	0.122	0.030	1.32

Appendix

Table 13: CALVIN ABCD

\rightarrow

D (10%) performance under varying action-chunk horizon

H

. The default setting

H{=}10

provides a strong trade-off between performance and efficiency.

Panel of Figure 5 — Figure 5: Instruction(very hard): ”Pick and place a puck onto a shelf.” Sequential progression of successful task execution in the Meta-world environment.

Explore similar work

GST-VLA: Structured Gaussian Spatial Tokens for 3D Depth-Aware Vision-Language-Action Models
Mar 10, 2026Md Selim Sarowar, Omer Tariq, Sungho KimVision-Language-Action Models VLMs for Robotics
Yeungnam University · Korea Advanced Institute of Science and Technology, KAIST
G $^3$ VLA: Geometric Inductive Bias for Vision-Language-Action Models
Jun 23, 2026Yue Peng, Yongzhe Zhao, Artur Habuda +5Multi-View Learning Robotic Manipulation
New York University Shanghai · Technical University of Denmark · MBZUAI - Mohamed bin Zayed University of Artificial Intelligence
VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulation
Jul 14, 2026Mohan Liu, Zhihao Gu, Xuanyu Chen +5Language-Conditioned Robot Manipulation 3D Spatial Reasoning
EmPACT Lab, Nanyang Technological University, Singapore · Institute for Infocomm Research (I2R), A*STAR, Singapore

Mar 10, 2026J/K move · Enter open · S save

GST-VLA: Structured Gaussian Spatial Tokens for 3D Depth-Aware Vision-Language-Action Models

Md Selim Sarowar, Omer Tariq, Sungho Kim

Yeungnam University · Korea Advanced Institute of Science and Technology, KAIST

Vision-Language-Action Models VLMs for Robotics Visual Tokenization Reasoning for VLA Models 3D Visual Grounding

Open paper AskPDF arXiv

VLA models encode visual observations as 2D patch tokens with no intrinsic geometric structure. We introduce GST-VLA with two contributions. First, the Gaussian Spatial Tokenizer (GST) converts frozen dense depth and frozen semantic patch features into

N_g{=}128

anisotropic 3D Gaussian primitives, each parameterized by a metric residual mean

μ\in \mathbb{R}^3

, log-scale covariance

\log σ\in \mathbb{R}^3

, and learned opacity

α\in (0,1)

. The covariance eigenstructure encodes local surface orientation, and opacity provides per-primitive geometric confidence, both inaccessible from scalar depth. Spatial attention pooling with learned queries concentrates the fixed token budget on geometrically salient regions rather than distributing uniformly. Second, 3D Depth-Aware Chain-of-Thought (DA-CoT) reasoning supervises four structured intermediate spatial thoughts, covering 3D object grounding, grasp affordance contact geometry, pairwise metric distances, and coarse SE(3) waypoints, as explicit generation targets in the training loss. A cross-attention sublayer at every VLM transformer block provides direct access to the raw 256-primitive Gaussian field during DA-CoT generation. A 300M-parameter flow-matching action expert with mixture-of-experts feedforward sublayers decodes 7-DoF delta action chunks via conditional ODE integration, conditioned on both VLM hidden states and DA-CoT outputs through dual cross-attention. Trained with composite

\mathcal{L}_\mathrm{flow} + \mathcal{L}_\mathrm{CoT} + \mathcal{L}_\mathrm{depth}

across three progressive stages, GST-VLA achieves 96.4% on LIBERO (+2.0%), and 80.2% on SimplerEnv (+5.4%). Ablations isolate the contribution of each GST component, each DA-CoT thought, and each training stage, confirming independent and synergistic gains concentrated on precision demanding tasks.

Abstract

Explore similar work

GST-VLA: Structured Gaussian Spatial Tokens for 3D Depth-Aware Vision-Language-Action Models

G3^33VLA: Geometric Inductive Bias for Vision-Language-Action Models

VistaVLA: Geometry- and Semantic-Aware 3D Gaussian-Grounded VLA for Robotic Manipulation

G $^3$ VLA: Geometric Inductive Bias for Vision-Language-Action Models