cs.CVNov 11, 2026

Vision-Grounded Semantic Learning and Alignment for Scalable Multilingual Text-Based Person Retrieval

Authors: Xinyu Zhou, Fan Li, Shasha Chen, Min Cao, Ahtamjan Ahmat, Qiaoming Zhu, Mang Ye

Organizations: Soochow University, Suzhou, China · The Xinjiang Technical Institute of Physics & Chemistry, Chinese Academy of Sciences, Xinjiang, China · Wuhan University, Wuhan, China

Abstract

Multilingual Text-Based Person Retrieval (TBPR) aims to retrieve individuals with texts in multiple languages. Despite its practical importance, it has been overlooked by existing research, which mainly focuses on English-only retrieval. The only prior attempt at multilingual TBPR suffers from (1) the absence of a large-scale, multilingual pretraining dataset and (2) a suboptimal model architecture for multilingual text encoding and cross-modal alignment, which employs a shared encoder that overlooks linguistic heterogeneity and relies on English as a pivot for cross-modal alignment, introducing English-centric bias. For this, we contribute MultiPedes, the first large-scale, pedestrian-centric multilingual pretraining dataset. It contains 10 million image-text pairs across four languages, constructed by a multilingual quality-aware data construction pipeline. We also propose ViSeLA (Vision-grounded Semantic Learning and Alignment), a novel framework featuring: (1) a semantic-aware multilingual learning module that extends the mixture-of-experts architecture by routing language-invariant semantics to specialized experts for improved multilingual feature learning; and (2) a vision-grounded cross-modal alignment module that employs a semantic probe to capture fine-grained semantics across modalities, enabling direct cross-modal alignment without relying on a pivot language. Extensive experiments on both multilingual and English-only TBPR datasets demonstrate the effectiveness of ViSeLA. The dataset and code are available at https://github.com/Flame-Chasers/ViSeLA

Figures & tables

Explore similar work

CardsList
  1. MultiInpaintCOCO: Fine-Grained Visual Understanding in Multiple Languages

    Nov 2, 2026Andrei-Alexandru Manea, Jindřich Libovický, Philipp J. RöschMultilingual VLM EvaluationVision-Language Retrieval

  2. Heterogeneous Vision-Language Ensemble with Disagreement-Aware Reranking for Text-Based Person Anomaly Retrieval

    Aug 13, 2026Huu-An Vu, Cam Tu Tran Thi, Thanh Toan Le Ngo +5Vision-Language ModelsMultimodal Reranking