E-SocialNav: Efficient Socially Compliant Navigation with Language Models

Ling Xiao; Daeun Song; Xuesu Xiao; Toshihiko Yamasaki

cs.ROMar 21, 2026

E-SocialNav: Efficient Socially Compliant Navigation with Language Models

Authors: Ling Xiao, Daeun Song, Xuesu Xiao, Toshihiko Yamasaki

Organizations: Hokkaido University, Sapporo, Japan · George Mason University, Virginia, United States · The University of Tokyo, Tokyo, Japan

Abstract

Language models (LMs) are increasingly applied to robotic navigation; however, existing benchmarks primarily emphasize navigation success rates while paying limited attention to social compliance. Moreover, relying on large-scale LMs can raise efficiency concerns, as their heavy computational overhead leads to slower response times and higher energy consumption, making them impractical for real-time deployment on resource-constrained robotic platforms. In this work, we evaluate the social compliance of GPT-4o and Claude in robotic navigation and propose E-SocialNav, an efficient LM designed for socially compliant navigation. Despite being trained on a relatively small dataset, E-SocialNav consistently outperforms zero-shot baselines in generating socially compliant behaviors. By employing a two-stage training pipeline consisting of supervised fine-tuning followed by direct preference optimization, E-SocialNav achieves strong performance in both text-level semantic similarity to human annotations and action accuracy. The source code is available at https://github.com/Dr-LingXiao/ESocialNav.

Figures & tables

Figure 1: The detailed structure of E-SocialNav. E-SocialNav is trained in two phases: SFT on multi-turn dialogues, followed by DPO on single-turn pairs. During SFT, only the projector is updated; during DPO, only the LoRA adapter is updated.

Panel of Figure 2 — Figure 2: Visualization of constructed DPO training pairs. The chosen response is annotated by humans, whereas the rejected response is generated by modifying certain facts in the chosen response.

	Model	VT	LM	BERTScore-P $\uparrow$	BERTScore-R $\uparrow$	BERTScore-F1 $\uparrow$	SBERT-cos $\uparrow$	SMS $\uparrow$	FPS $\uparrow$	AA $\uparrow$
Off-the-shelf	Claude	-	-	-0.233	0.387	0.059	0.664	0.641	0.087	0.417
Off-the-shelf	GPT-4o	-	-	0.076	0.443	0.254	0.672	0.651	0.212	0.450
Finetuned	Social-LLaVA [ 12 ]	CLIP ViT-L/14	Vicuna-7B	0.672	0.653	0.641	0.784	0.813	1.113	0.483
E-SocialNav (Ours)	SFT(projector+lora+vision)	SigLIP [ 17 ]	Phi-2-2.7B [ 4 ]	0.585	0.434	0.509	0.744	0.802	0.978	0.383
	SFT(projector+lora)	SigLIP [ 17 ]	Phi-2-2.7B [ 4 ]	0.640	0.545	0.592	0.756	0.813	1.553	0.400
	SFT(projector)	SigLIP [ 17 ]	Phi-2-2.7B [ 4 ]	0.551	0.658	0.604	0.780	0.828	1.828	0.433

Table 1: Experimental results comparing off-the-shelf models and variants of the proposed method. SFT(X) means the components X are trainable in Stage I (supervised fine-tuning); DPO(Y) means Y are trainable in Stage II (direct preference optimization). Components not listed are frozen. Best performance is bolded.

Figure 3: Visualizations: E-SocialNav accurately captures social-compliance cues from the image.

	VT	SLM	BERTScore-P $\uparrow$	BERTScore-R $\uparrow$	BERTScore-F1 $\uparrow$	SBERT-cos $\uparrow$	SMS $\uparrow$
E-SocialNav (Ours)	CLIP [ 13 ]	Phi-2-2.7B [ 4 ]	0.555	0.658	0.605	0.803	0.768
	Dino [ 11 ]	Phi-2-2.7B [ 4 ]	0.692	0.663	0.677	0.801	0.833
	SigLIP [ 17 ]	TinyLlama-1.1B-Chat-v1.0 [ 18 ]	0.473	0.576	0.523	0.733	0.789
	SigLIP [ 17 ]	stablelm-2-zephyr-1_6b [ 1 ]	0.700	0.622	0.661	0.788	0.837
	SigLIP [ 17 ]	Phi-2-2.7B [ 4 ]	0.706	0.671	0.688	0.814	0.846

Table 2: Experimental results across different VT and SLM configurations. Best performance is bolded.

Figure 4: Visualization of failure cases. Gt: Ground truth.

Social Robot Navigation

Social robot navigation studies how robots move safely and comfortably around people while respecting human behavior, preferences, and shared-space norms.

See all 79 papers

SOPD-SocialNav: Selective On-Policy Distillation for Vision-Language Social Navigation
Jul 22, 2026Xinyu Zhang, Zishuo Wang, Ling XiaoEfficient VLM Inference VLM Distillation
Graduate School of Information Science and Technology, Hokkaido University, Sapporo, Japan
G2-Nav: Grounded and Guarded Vision-Language Costmaps for Robot Social Navigation
Jul 18, 2026Yuwen Liao, Yihang Lan, Yizhuo Yang +4Robot Navigation Safe Robot Navigation
School of Electrical and Electronic Engineering Nanyang Technological University Singapore
Robots Ask the Way: Communication-Enabled Social Navigation
Jul 1, 2026Valentino Sacco, Luca Scofano, Indro Spinelli +1Robot Navigation Social Robot Navigation
Sapienza University of Rome, Italy

Jul 22, 2026J/K move · Enter open · S save

SOPD-SocialNav: Selective On-Policy Distillation for Vision-Language Social Navigation

Xinyu Zhang, Zishuo Wang, Ling Xiao

Graduate School of Information Science and Technology, Hokkaido University, Sapporo, Japan

Efficient VLM Inference VLM Distillation Social Robot Navigation On-Policy Distillation VLMs for Robotics

Open paper AskPDF arXiv

Vision-language models have shown strong potential for social robot navigation by leveraging rich semantic understanding of complex environments and human behaviors. However, large scale VLMs are difficult to deploy on resource-constrained robotic platforms, while lightweight VLMs often lack sufficient social reasoning capability. To address this problem, we propose SOPD-SocialNav, a selective on-policy distillation (SOPD) method that transfers social navigation knowledge from a large teacher VLM to a lightweight student VLM. SOPD introduces an entropy-based token selection mechanism that uses teacher uncertainty to identify socially informative decision tokens, while suppressing gradients from low-entropy tokens corresponding to trivial navigation states. A temperature-controlled Jensen-Shannon divergence objective is then used to align the student and teacher distributions on the selected tokens. Experiments on the SNEI and MUSON benchmarks demonstrate that SOPD consistently outperforms supervised fine-tuning, off-policy distillation, and standard on-policy distillation baselines in action prediction, perception consistency, and reasoning consistency. Real-world deployment on a Scout Mini robot further shows that the distilled model can generate more socially appropriate navigation behaviors in conversational and queuing scenarios. These results suggest that SOPD is an effective strategy for building lightweight yet socially aware VLM-based navigation systems.

Abstract

Social Robot Navigation

Explore similar work

SOPD-SocialNav: Selective On-Policy Distillation for Vision-Language Social Navigation

G2-Nav: Grounded and Guarded Vision-Language Costmaps for Robot Social Navigation

Robots Ask the Way: Communication-Enabled Social Navigation