Language models (LMs) are increasingly applied to robotic navigation; however, existing benchmarks primarily emphasize navigation success rates while paying limited attention to social compliance. Moreover, relying on large-scale LMs can raise efficiency concerns, as their heavy computational overhead leads to slower response times and higher energy consumption, making them impractical for real-time deployment on resource-constrained robotic platforms. In this work, we evaluate the social compliance of GPT-4o and Claude in robotic navigation and propose E-SocialNav, an efficient LM designed for socially compliant navigation. Despite being trained on a relatively small dataset, E-SocialNav consistently outperforms zero-shot baselines in generating socially compliant behaviors. By employing a two-stage training pipeline consisting of supervised fine-tuning followed by direct preference optimization, E-SocialNav achieves strong performance in both text-level semantic similarity to human annotations and action accuracy. The source code is available at https://github.com/Dr-LingXiao/ESocialNav.
Figures & tables
Figure 1: The detailed structure of E-SocialNav. E-SocialNav is trained in two phases: SFT on multi-turn dialogues, followed by DPO on single-turn pairs. During SFT, only the projector is updated; during DPO, only the LoRA adapter is updated.
Figure 2: Visualization of constructed DPO training pairs. The chosen response is annotated by humans, whereas the rejected response is generated by modifying certain facts in the chosen response.
Model
VT
LM
BERTScore-P ↑
BERTScore-R ↑
BERTScore-F1 ↑
SBERT-cos ↑
SMS ↑
FPS ↑
AA ↑
Off-the-shelf
Claude
-
-
-0.233
0.387
0.059
0.664
0.641
0.087
0.417
GPT-4o
-
-
0.076
0.443
0.254
0.672
0.651
0.212
0.450
Finetuned
Social-LLaVA [ 12 ]
CLIP ViT-L/14
Vicuna-7B
0.672
0.653
0.641
0.784
0.813
1.113
0.483
E-SocialNav (Ours)
SFT(projector+lora+vision)
SigLIP [ 17 ]
Phi-2-2.7B [ 4 ]
0.585
0.434
0.509
0.744
0.802
0.978
0.383
SFT(projector+lora)
SigLIP [ 17 ]
Phi-2-2.7B [ 4 ]
0.640
0.545
0.592
0.756
0.813
1.553
0.400
SFT(projector)
SigLIP [ 17 ]
Phi-2-2.7B [ 4 ]
0.551
0.658
0.604
0.780
0.828
1.828
0.433
Table 1: Experimental results comparing off-the-shelf models and variants of the proposed method. SFT(X) means the components X are trainable in Stage I (supervised fine-tuning); DPO(Y) means Y are trainable in Stage II (direct preference optimization). Components not listed are frozen. Best performance is bolded.
Figure 3: Visualizations: E-SocialNav accurately captures social-compliance cues from the image.
VT
SLM
BERTScore-P ↑
BERTScore-R ↑
BERTScore-F1 ↑
SBERT-cos ↑
SMS ↑
E-SocialNav (Ours)
CLIP [ 13 ]
Phi-2-2.7B [ 4 ]
0.555
0.658
0.605
0.803
0.768
Dino [ 11 ]
Phi-2-2.7B [ 4 ]
0.692
0.663
0.677
0.801
0.833
SigLIP [ 17 ]
TinyLlama-1.1B-Chat-v1.0 [ 18 ]
0.473
0.576
0.523
0.733
0.789
SigLIP [ 17 ]
stablelm-2-zephyr-1_6b [ 1 ]
0.700
0.622
0.661
0.788
0.837
SigLIP [ 17 ]
Phi-2-2.7B [ 4 ]
0.706
0.671
0.688
0.814
0.846
Table 2: Experimental results across different VT and SLM configurations. Best performance is bolded.
Figure 4: Visualization of failure cases. Gt: Ground truth.
Social robot navigation studies how robots move safely and comfortably around people while respecting human behavior, preferences, and shared-space norms.
Vision-language models have shown strong potential for social robot navigation by leveraging rich semantic understanding of complex environments and human behaviors. However, large scale VLMs are difficult to deploy on resource-constrained robotic platforms, while lightweight VLMs often lack sufficient social reasoning capability. To address this problem, we propose SOPD-SocialNav, a selective on-policy distillation (SOPD) method that transfers social navigation knowledge from a large teacher VLM to a lightweight student VLM. SOPD introduces an entropy-based token selection mechanism that uses teacher uncertainty to identify socially informative decision tokens, while suppressing gradients from low-entropy tokens corresponding to trivial navigation states. A temperature-controlled Jensen-Shannon divergence objective is then used to align the student and teacher distributions on the selected tokens. Experiments on the SNEI and MUSON benchmarks demonstrate that SOPD consistently outperforms supervised fine-tuning, off-policy distillation, and standard on-policy distillation baselines in action prediction, perception consistency, and reasoning consistency. Real-world deployment on a Scout Mini robot further shows that the distilled model can generate more socially appropriate navigation behaviors in conversational and queuing scenarios. These results suggest that SOPD is an effective strategy for building lightweight yet socially aware VLM-based navigation systems.
Xinyu Zhang, Zishuo Wang, Ling Xiao
Graduate School of Information Science and Technology, Hokkaido University, Sapporo, Japan
Social navigation requires the robot to reason and respond in complex real-world environments. While recent works attempt to incorporate human-level intelligence into robot planning using large Vision-Language Models (VLMs), end-to-end frameworks often create an unpredictable black-box, and existing instruction-following methods are not designed for full autonomy. To bridge this gap, we present G2-Nav, a novel framework that grounds abstract social reasoning and guards safe real-world deployment. Instead of asking the VLM for direct planning decisions, G2-Nav translates its semantic reasoning into a vision-language costmap with reliability and interpretability. The VLM evaluates traversable regions and social agents from open-set perception, mapping social context into the costmap. To improve real-world robustness, the VLM performs semantic verification on upstream tracking, and we introduce a high-frequency safety check to guard against system latency prior to trajectory generation. We demonstrate through real-world experiments that G2-Nav delivers safe, efficient, and socially compliant autonomous navigation in unstructured environments. Code is available at https://github.com/centiLinda/G2-Nav.
Yuwen Liao, Yihang Lan, Yizhuo Yang +4
School of Electrical and Electronic Engineering Nanyang Technological University Singapore
Assistive autonomous robots operating in multi-agent environments require efficient strategies to locate specific individuals among multiple residents. Current social navigation methods focus on reactive collision avoidance and trajectory adaptation, but lack mechanisms to proactively gather information through human-robot communication. We introduce Communication-enabled Social Navigation (CommNav). In this novel task, robotic agents actively seek assistance from residents to locate target individuals by requesting information about recent sightings, locations, and movements. To evaluate CommNav, we extend Habitat 3.0 to create Habitat 3.0c, a communication-enabled variant supporting multi-human environments with information exchange protocols. Adding our communication module (COMM) to a state-of-the-art social navigation model yields a 10 percentage-point improvement in Episode Success. We further investigate the transition from structured data to natural language by evaluating models trained on LLM-generated instructions and on colloquial instructions collected from a human study. Our experiments reveal that: (i) explicit human-robot communication substantially enhances multi-person navigation performance; (ii) pre-training COMM on a communication pretext task effectively addresses the challenge of occasional interaction signals; and (iii) the navigation policy is highly robust to natural, colloquial human language, achieving an episode success statistically similar to the model using perfect structured data.