ReSiReg: Towards Spatially Consistent Semantics in Language-Conditioned Robotic Tasks
Authors: Simon Schwaiger, David Seyser, Alessandro Scherl, Wilfried Wöber, Gerald Steinbauer-Wagner
Organizations: Graz University of Technology, Institute of Software Engineering and Artificial Intelligence · University of Applied Sciences Technikum Wien, Department of Industrial Engineering · 1Graz University of Technology, Institute of Software Engineering and Artificial Intelligence · University of Alicante, Department of Computer Technology · University of Natural Resources and Life Sciences, Institute for Integrative Nature Conservation Research
Vision-Language Models (VLMs) enable robots to follow open-language instructions. However, dense VLM embeddings have shown to be noisy and lack spatial consistency. This is problematic for robotic applications, which require simultaneous reasoning over semantics and 3D space. We examine spatial structure across recent VLMs and propose ReSiReg, a feature reconstruction method that uses spatially consistent VLM intermediates to improve dense language-grounded retrieval. ReSiReg clusters intermediates into visual prototypes, derives their language descriptors, and reconstructs each patch as a soft mixture of prototype-level language embeddings. We evaluate quantitatively on OVSS and 3D mapping across backbones, and qualitatively in real-world manipulation scenes. Quantitative results show improved dense retrieval; manipulation scenes show more spatially consistent target activations. We further provide a compact 25M dense VLM for robotic applications, substantially smaller than and competitive with ViT-B baselines. Available at https://resireg.github.io