Social robot navigation requires a sophisticated integration of scene semantics and human social norms. Scaling up Vision Language Models (VLMs) generally improves reasoning and decision-making capabilities for socially compliant navigation. However, increased model size incurs substantial computational overhead, limiting suitability for real-time robotic deployment. Conversely, lightweight VLMs enable efficient inference but often exhibit weaker reasoning and decision-making performance in socially complex environments. Achieving both strong reasoning ability and efficiency remains an open challenge. To bridge this gap, we propose Group Competitive Learning (GCL), a strategy designed to amplify the capabilities of lightweight VLMs. Our strategy introduces the Group Competitive Objective (GCO) to harmonize global semantics with distributional regularization, alongside Asymmetric Group Optimization (AGO) to explore the upper limits of model performance. Empirical evaluations on social navigation benchmarks demonstrate that GCL significantly elevates VLM performance. Specifically, GCL enables the Qwen2.5-VL-3B learner model and guide Qwen3-VL-4B to achieve an F1 score of 0.968 and 0.914, representing 40% and 12% improvement over vanilla supervised fine-tuning (SFT). Notably, under vanilla SFT, the 3B model initially trails the 8B model (F1: 0.692 vs. 0.755). However, through the GCL, the 3B model outperforms (28%) the 8B baseline model. These results suggest that GCL provides an effective solution for achieving both high accuracy and computational efficiency in real-world deployment.
Figures & tables
Figure 1: The Overview of GCL. The left of the figure illustrates the GCL strategy’s inputs examples: texts and images across social navigation dataset. The middle shows the GCL strategy’s components: Group Competitive Objective, to align VLMs at both semantic and distributional token levels, and Asymmetric Group Optimization, to explore the optimization dynamics and performance boundaries of GCL. The right shows the socially compliant instruction of GCL outputs.
Regime A: Reshaping Mode Capacity L < Capacity G , Small → Large, ηlearner=1×10−5ηguide=5×10−6
Model Group (Learner / Guide)
τL / τG
Baseline ( F1L / F1G )
GCL ( F1L / F1G )
Qwen3-VL-2B / Qwen3-VL-4B
3.0 / 2.0
0.797 / 0.816
0.954 (+0.157) / 0.910 (+0.094)
Qwen2.5-VL-3B / Qwen3-VL-4B
3.0 / 2.0
0.692 / 0.816
0.968 (+0.276) / 0.914 (+0.098)
Qwen2.5-VL-3B / Qwen2.5-VL-7B
3.0 / 2.0
0.692 / 0.777
0.869 (+0.177) / 0.915 (+0.138)
Qwen2.5-VL-3B / Qwen2.5-VL-8B
3.0 / 2.0
0.692 / 0.755
0.964 (+0.272) / 0.811 (+0.056)
Qwen3-VL-2B / Qwen3-VL-2B
3.0 / 2.0
0.797 / 0.797
0.841 (+0.044) / 0.884 (+0.087)
Table 1: Performance of GCL across 12 competitive group. ηguide and ηlearner are allocated to SFT baseline superiority. The performance hierarchy from lowest to highest is: Qwen2.5-VL-3B (0.692) < Qwen3-VL-8B (0.755) < Qwen2.5-VL-7B (0.777) < Qwen3-VL-2B (0.797) < Qwen3-VL-4B (0.816). Meanwhile, temperature parameters ( τL , τG ) are assigned solely based on parameter size (larger capacity = 2.0, smaller capacity = 3.0). F1 denotes Action-F1. Bold text indicates optimal performance of the model.( For MUSON results, details provided in the Supplementary Material )
Method
Model
SNEI
MUSON
F1 ↑
Per-cos ↑
Rea-cos ↑
F1 ↑
Per-cos ↑
Rea-cos ↑
SFT (baseline)
Lsup
Qwen2.5-VL-3B
0.692
0.834
0.767
0.811
0.810
0.803
Qwen3-VL-4B
0.816
0.773
0.775
0.863
0.855
0.849
GCL
LGCO
Lsup+LGSL
Qwen2.5-VL-3B
0.818
0.880
0.846
0.877
0.872
0.864
Qwen3-VL-4B
0.821
0.806
0.838
0.905
0.879
0.874
Lsup+LDRL
Qwen2.5-VL-3B
0.832
0.903
0.851
0.851
0.840
0.837
Table 2: Performance comparison of GCL components and vanilla SFT. We report Action-F1 (F1), Perception-cos (Per-cos) and Reasoning-cos (Rea-cos) on two datasets SNEI and MUSON. These results demonstrate that GCL, enables compact models to internalize intricate structural knowledge, ultimately surpassing the generalization capabilities of larger models trained in isolation.
Figure 2: Ablation Study of Loss Components in GCO. Red and blue bars represent the learner (Qwen2.5-VL-3B) and guide (Qwen3-VL-4B) models, respectively, with bold values indicating peak performance. The left illustrates the ablation of the GSL weight, where both models achieve optimal results at 0.5. The right plot presents the ablation of the DRL weight, demonstrating that a value of 0.4 yields the highest performance for both models.
Figure 3: Impact of Capability Gaps on Asymmetric Learning Rate Dynamics. Action-F1 is evaluated against the learning rate ratio ( r=ηlearner/ηguide ), with ηguide fixed at η=5×10−6 . Shaded regions indicate the Optimization Best Spot. Fig. 3(a) shows that large capability gap group tolerate aggressive learner updates ( 1.0≤r≤3.0 ), enabling effective feature space alignment. In contrast, Fig. 3(b) reveals that minimal initial gaps confine the optimal window to ( 1.0≤r≤2.0 ). These confirms that GCL necessitates performance based asymmetric learning rate scheduling.
Figure 4: Optimal Asymmetric Temperature Gradients in VLMs Group. Heatmaps of Action-F1 scores for Qwen2.5-VL-3B learner (left) and Qwen3-VL-4B guide (right) across temperature grids. Symmetric settings τlearner=τguide ) result in suboptimal consensus or collapse (e.g., 0.720 at τ=3.0 ). The global peak (learner: 0.968) occurs at (τlearner=3.0,τguide=2.0) , highlighting the necessity of asymmetric thermal gradients to enhance learner plasticity through guide discriminative signaling. ( For small gap group, details provided in the Supplementary Material )
Figure 5: Visualization Comparison. SFT models (yellow/green) miss critical constraints and produce incorrect actions, while GCL models (red/blue) generate socially compliant instructions aligned with GT.
Regime A: Reshaping Mode Capacity L < Capacity G , Small → Large, ηlearner=1×10−5ηguide=5×10−6
Model Group (Learner / Guide)
τL / τG
Baseline ( F1L / F1G )
GCL ( F1L / F1G )
Qwen3-VL-2B / Qwen2.5-VL-3B
3.0 / 2.0
0.805 / 0.811
0.989 (+0.184) / 0.974 (+0.163)
Qwen3-VL-2B / Qwen3-VL-4B
3.0 / 2.0
0.805 / 0.863
0.963 (+0.158) / 0.969 (+0.106)
Qwen2.5-VL-3B / Qwen3-VL-4B
3.0 / 2.0
0.811 / 0.863
0.975 (+0.164) / 0.961 (+0.098)
Qwen3-VL-2B / Qwen2.5-VL-7B
3.0 / 2.0
0.805 / 0.829
0.907 (+0.102) / 0.943 (+0.114)
Qwen2.5-VL-3B / Qwen2.5-VL-7B
3.0 / 2.0
0.811 / 0.829
0.966 (+0.155) / 0.930 (+0.101)
Table 3: Performance of GCL across 12 competitive group on MUSON. ηguide and ηlearner are allocated to SFT baseline superiority. The performance hierarchy from lowest to highest is: Qwen3-VL-2B (0.805) < Qwen2.5-VL-3B (0.811) < Qwen2.5-VL-7B (0.829) < Qwen3-VL-4B (0.863) < Qwen3-VL-8B (0.873). Meanwhile, temperature parameters ( τL , τG ) are assigned solely based on parameter size (larger capacity = 2.0, smaller capacity = 3.0). F1 denotes Action-F1. Bold text indicates optimal performance of the model.
Figure 7: Asymmetric Temperature in Small Gap Group. Heatmaps of Action-F1 scores for Qwen3-VL-2B learner (left) and Qwen3-VL-4B guide (right) across temperature grids. The global peak (learner: 0.954) occurs at (τlearner=3.0,τguide=2.0) . Symmetric configuration ( τlearner=τguide ) leads to suboptimal performance and may even induce severe performance degradation (guide: 0.732).
Vision-language models have shown strong potential for social robot navigation by leveraging rich semantic understanding of complex environments and human behaviors. However, large scale VLMs are difficult to deploy on resource-constrained robotic platforms, while lightweight VLMs often lack sufficient social reasoning capability. To address this problem, we propose SOPD-SocialNav, a selective on-policy distillation (SOPD) method that transfers social navigation knowledge from a large teacher VLM to a lightweight student VLM. SOPD introduces an entropy-based token selection mechanism that uses teacher uncertainty to identify socially informative decision tokens, while suppressing gradients from low-entropy tokens corresponding to trivial navigation states. A temperature-controlled Jensen-Shannon divergence objective is then used to align the student and teacher distributions on the selected tokens. Experiments on the SNEI and MUSON benchmarks demonstrate that SOPD consistently outperforms supervised fine-tuning, off-policy distillation, and standard on-policy distillation baselines in action prediction, perception consistency, and reasoning consistency. Real-world deployment on a Scout Mini robot further shows that the distilled model can generate more socially appropriate navigation behaviors in conversational and queuing scenarios. These results suggest that SOPD is an effective strategy for building lightweight yet socially aware VLM-based navigation systems.
Xinyu Zhang, Zishuo Wang, Ling Xiao
Graduate School of Information Science and Technology, Hokkaido University, Sapporo, Japan
As mobile robots become more integrated into everyday human environments, social robot navigation is becoming essential for ensuring human comfort, safety, and trust. While reinforcement learning (RL) navigation policies provide the fast inference and reactive behavior necessary for real-time deployment, they still lack flexible semantic reasoning capabilities and often fail to generalize to complex social scenarios. Recent approaches have increasingly turned to vision-language models (VLMs) in place of RL policies to improve semantic and social reasoning in robot navigation. Nevertheless, their high computational cost and slow inference remain major barriers to real-time deployment. To overcome these limitations, we introduce HUMA (Hybrid Understanding for Multi-modal social Navigation), a hybrid architecture that dynamically balances the computational efficiency of RL policies with the deep semantic understanding of VLMs. Our approach uses a reactive RL policy to handle low-density, routine navigation tasks, while conditioning it on a post-trained high-level VLM when a human enters sensitive situations, such as the robot's proximity zone. We evaluate HUMA on the Social-MP3D and Social-HM3D benchmarks, where it achieves task success improvements of 20% and 3%, respectively, while significantly reducing personal space violations and human collisions against state-of-the-art baselines. Extensive ablation studies validate each architectural component, and real-world deployment on the Mirokaï mobile robot further demonstrates the practical viability of our approach.
Ali Ahmadi, Hamed Rahimi, Adrien Jacquet Cretides +3
Institut des Syst`emes Intelligents et de Robotique (ISIR) Sorbonne Universit´e France
Robot navigation in dynamic, human-centered environments requires socially-compliant decisions grounded in robust scene understanding. Recent Vision-Language Models (VLMs) exhibit promising capabilities such as object recognition, common-sense reasoning, and contextual understanding, capabilities that align with the nuanced requirements of social robot navigation. However, it remains unclear whether VLMs can accurately understand complex social navigation scenes (e.g., inferring the spatial-temporal relations among agents and human intentions), which is essential for safe and socially compliant robot navigation. While some recent works have explored the use of VLMs in social robot navigation, no existing work systematically evaluates their ability to meet these necessary conditions. In this paper, we introduce the Social Navigation Scene Understanding Benchmark (SocialNav-SUB), a Visual Question Answering (VQA) dataset and benchmark designed to evaluate VLMs for scene understanding in real-world social robot navigation scenarios. SocialNav-SUB provides a unified framework for evaluating VLMs against human and rule-based baselines across VQA tasks requiring spatial, spatiotemporal, and social reasoning in social robot navigation. Through experiments with state-of-the-art VLMs, we find that while the best-performing VLM achieves an encouraging probability of agreeing with human answers, it still underperforms simpler rule-based approach and human consensus baselines, indicating critical gaps in social scene understanding of current VLMs. Our benchmark sets the stage for further research on foundation models for social robot navigation, offering a framework to explore how VLMs can be tailored to meet real-world social robot navigation needs. An overview of this paper along with the code and data can be found at https://larg.github.io/stars.
Nathan Tsoi, Michael J. Munje, Tejas Oberoi +5
Department of Computer Science, The University of Texas at Austin