AnySlot: Goal-Conditioned Vision-Language-Action Policies for Zero-Shot Slot-Level Placement

Zhaofeng Hu; Sifan Zhou; Qinbo Zhang; Rongtao Xu; Qi Su; Jorge Mendez-Mendz; Ci-Jyun Liang

cs.ROApr 12, 2026

AnySlot: Goal-Conditioned Vision-Language-Action Policies for Zero-Shot Slot-Level Placement

Authors: Zhaofeng Hu, Sifan Zhou, Qinbo Zhang, Rongtao Xu, Qi Su, Jorge Mendez-Mendz, Ci-Jyun Liang

Organizations: Stony Brook University, Stony Brook, USA · Carnegie Mellon University, Pittsburgh, USA · MBZUAI, Abu Dhabi, UAE · Peking University, Beijing, China

Abstract

Vision-Language-Action (VLA) policies have emerged as a versatile paradigm for generalist robotic manipulation. However, precise object placement under compositional language remains challenging for end-to-end VLA policies. Slot-level placement requires reliable slot grounding and centimeter-level geometric precision. To this end, we propose AnySlot, a framework that reduces compositional complexity by introducing an explicit spatial visual goal between language grounding and control. AnySlot converts language into a visual goal by rendering a spatial marker at the intended slot, then executes this goal with a goal-conditioned VLA policy. This hierarchical design decouples high-level slot selection from low-level execution, improving semantic accuracy and spatial robustness. Furthermore, recognizing the lack of benchmarks for such precision-demanding tasks, we introduce SlotBench, a structured simulation benchmark with nine task categories for evaluating spatial reasoning in slot-level placement. Extensive experiments show that AnySlot significantly outperforms flat VLA baselines and modular grounding methods in zero-shot slot-level placement.

Figures & tables

Figure 1: Real-world goal-guided slot-level placement. (1) Given a language instruction, the robot captures a head-camera RGB-D observation. (2) The image-generation module renders a spatial visual goal at the instructed slot. (3) The goal is reconstructed and projected into head and wrist camera views to guide low-level manipulation, enabling successful placement on a real UR10e platform. AnySlot demonstrates real-world generalization and robustness.

Figure 2: Overview of flat (a), modular (b), and (c) our goal-conditioned policy. For slot-level placement tasks, our method achieves superior zero-shot generalization by transforming language into explicit spatial visual goals for reliable execution.

Figure 3: AnySlot overview. We formulate slot-level placement as goal-conditioned control. High-level goal construction uses the Nano-Banana image generator to render a blue-sphere goal from the language prompt, lifting it to a view-consistent multi-view overlay via depth and camera calibration. Low-level control uses a goal-conditioned VLA policy ( $\pi_{0.5}$ ) with a PaliGemma-3B backbone and action expert, mapping goal-augmented multi-view observations to continuous actions $A_{t}$ .

Figure 4: SlotBench. A slot-level benchmark with nine challenges for spatial reasoning and language grounding in manipulation.

Figure 5: Comparison between AnySlot and a VLM-based method. AnySlot grounds the target slot and succeeds, while the VLM-based method mislocalizes the target and fails.

Group	Method	Ord.		Size		Hgt.		Dist.		Comp.		Neg.		Vague		Aff.		World
Group	Method	SR	IA	SR	IA	SR	IA	SR	IA	SR	IA	SR	IA	SR	IA	SR	IA	SR	IA
Flat	Diffusion Policy [ 4 ]	16	/	0	/	0	/	0	/	0	/	0	/	0	/	0	/	0	/
	OpenVLA-OFT [ 10 ]	0	/	0	/	0	/	0	/	0	/	0	/	0	/	0	/	0	/
	$\pi_{0}$ [ 2 ]	12	/	0	/	0	/	0	/	0	/	0	/	0	/	0	/	0	/
	$\pi_{0.5}$ [ 8 ]	18	/	0	/	0	/	0	/	0	/	0	/	0	/	0	/	0	/
Modular	RoboPoint [ 23 ]	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0

Table 1: Performance comparison of different methods on the SlotBench benchmark. Each entry reports SR (Success Rate; higher is better, %) and IA (Instruction Accuracy; higher is better, %). The yellow rows denote our AnySlot , while the blue rows indicate the oracle upper bound.

Group	Method	Ord.		Size		Hgt.		Dist.		Comp.		Neg.		Vague		Aff.		World		Latency
Group	Method	CA	IA	CA	IA	CA	IA	CA	IA	CA	IA	CA	IA	CA	IA	CA	IA	CA	IA	Latency
VLM	Molmo-7B †	44	12	48	42	6	0	38	0	4	0	0	0	0	0	0	0	0	0	2
	GPT-5.3-instant	8	0	24	6	0	0	6	0	8	2	2	0	4	2	12	2	16	4	25
	GPT-5.2 Think	52	34	80	72	52	28	82	40	62	16	78	62	66	40	48	42	72	46	133
	Gemini-3.0 Pro	80	74	78	52	48	38	56	50	38	24	100	78	82	60	88	76	78	32	89
	Gemini-3.1 Pro	98	92	90	84	86	60	100	44	92	48	100	72	100	76	100	92	100	62	121

Table 2: Ablation on high-level grounding module. Report Coarse Accuracy (CA, %), Instruction Accuracy (IA, %) and Latency (s).

\dagger

means inference performed on a H200 GPU. The yellow rows denote the module used by AnySlot .

\ddagger

means fine-tuned and inference performed on a H200 GPU.

Method	Ord.	Size	Hgt.	Dist.	Comp.	Neg.	Vague	Aff.	World
Method	SR	SR	SR	SR	SR	SR	SR	SR	SR
AnySlot -Diffusion Policy	92	0	0	0	94	90	98	0	0
AnySlot -OpenVLA-OFT	0	0	0	0	0	0	0	0	0
AnySlot - $\pi_{0.5}$ (PEFT)	10	12	4	8	12	8	2	0	10
AnySlot - $\pi_{0.5}$ (FT)(Baseline)	98	96	82	100	100	100	100	100	98

Table 3: Effect of the low-level policy. Success Rate (SR, %) on SlotBench. The performance varies significantly with different policy choices, and the full fine-tuned

\pi_{0.5}

policy achieves the best results. yellow rows denote our proposed AnySlot .

Method	Ord.	Size	Hgt.	Dist.	Comp.	Neg.	Vague	Aff.	World
Method	SR	SR	SR	SR	SR	SR	SR	SR	SR
AnySlot (w/ $\mathcal{D}_{\mathrm{std}}$ )	2	0	0	0	0	0	0	0	0
AnySlot (w/ $\mathcal{D}_{\mathrm{syn}}$ )(Baseline)	98	96	82	100	100	100	100	100	98

Table 4: Effect of the synthetic dataset. Success Rate (SR, %) on SlotBench. Training with

\mathcal{D}_{\mathrm{syn}}

yields significantly higher SR than

\mathcal{D}_{\mathrm{std}}

across all categories. yellow rows denote AnySlot .

Method	Ord.	Size	Hgt.	Dist.	Comp.	Neg.	Vague	Aff.	World
Method	SR	SR	SR	SR	SR	SR	SR	SR	SR
$\pi_{0.5}$ (in-domain)	42	38	26	26	18	30	25	32	22
AnySlot (Baseline)	98	96	82	100	100	100	100	100	98

Table 5: Effect of in-domain training on SlotBench. Success Rate (SR, %). Directly fine-tuning

\pi_{0.5}

in-domain yields limited gains, while AnySlot achieves substantially higher performance across all categories. yellow rows denote our proposed AnySlot .

Explore similar work

Spa-Bench: A Real-Robot Benchmark for Spatial Grounding in Vision-Language-Action Models
Oct 8, 2026Justin Tien-Smith, Matthew Pidden, Rolandos Alexandros PotamiasVision-Language Grounding Vision-Language-Action Models
Imperial College London
Pointing-VLA: Typed Spatial Grounding Interfaces for Vision-Language-Action Manipulation
Aug 24, 2026Xiwen Chen, Zelin Li, Zhiruo Zhou +3Vision-Language-Action Models Robotic Manipulation
SIGS, Tsinghua University, China · Shanghai Jiao Tong University, China · Wuhan University of Technology, China +2
VP-VLA: Visual Prompting as an Interface for Vision-Language-Action Models
Mar 23, 2026Zixuan Wang, Yuxin Chen, Yuqi Liu +6Vision-Language-Action Models Reasoning for VLA Models
HKUST · CUHK · SmartMore

Oct 8, 2026J/K move · Enter open · S save

Spa-Bench: A Real-Robot Benchmark for Spatial Grounding in Vision-Language-Action Models

Justin Tien-Smith, Matthew Pidden, Rolandos Alexandros Potamias

Imperial College London

Vision-Language Grounding Vision-Language-Action Models Robot Manipulation Benchmarks

Open paper AskPDF arXiv

Vision-language-action models (VLAs) offer a promising route toward general-purpose robotics by transferring semantic knowledge from large-scale vision-language pretraining into language-conditioned control. However, most VLA evaluations emphasize aggregate manipulation success under changes in objects, instructions, or environments, leaving unclear how reliably these policies ground spatial descriptions in the physical scenes where they act. We therefore ask a sharper question: how reliably can current VLAs translate spatial descriptions into appropriate real-robot behavior? To investigate this, we introduce Spa-Bench, a real-robot benchmark that evaluates spatial grounding across six tabletop tasks covering Object State, Relative Size, Referential Description, Relational Placement, Ordinal Position, and Counting. Our primary evaluation compares in-distribution (ID) and out-of-distribution (OOD) spatial configurations while retaining training support for the relevant objects, concepts, and actions, reducing object and motor unfamiliarity as alternative explanations for failure. Four diagnostic conditions further probe physical execution, sensitivity to wording, object-task associations, and recognition that no action is required. We release 4,092 episodes collected on the SO-101 robot arm: 1,200 human-teleoperated demonstrations and 2,892 real-robot evaluation rollouts from five state-of-the-art VLA policies. Across the three fully evaluated policies, success falls from 67.5-73.3% on ID spatial configurations to 40.7-52.0% on OOD spatial configurations, while matched direct instructions improve success by 28.3-40.0 percentage points. These results show that the evaluated VLAs can often execute the required manipulations but lack robust spatial grounding. Project materials, including videos, code, and benchmark data, are available at https://justintiensmith.github.io/Spa-Bench/.

Abstract

Explore similar work

Spa-Bench: A Real-Robot Benchmark for Spatial Grounding in Vision-Language-Action Models

Pointing-VLA: Typed Spatial Grounding Interfaces for Vision-Language-Action Manipulation

VP-VLA: Visual Prompting as an Interface for Vision-Language-Action Models