Beyond Black Boxes: Visual Grounding for Text Embeddings

Fine-grained image retrieval via hand-drawn sketches or textual descriptions remains a critical challenge due to inherent modality gaps. While hand-drawn sketches capture complex structural contours, they lack color and texture, which text effectively provides despite omitting spatial contours. Motivated by the complementary nature of these modalities, we propose the Sketch and Text Based Image Retrieval (STBIR) framework. By synergizing the rich color and texture cues from text with the structural outlines provided by sketches, STBIR achieves superior fine-grained retrieval performance. First, a curriculum learning driven robustness enhancement module is proposed to enhance the model's robustness when handling queries of varying quality. Second, we introduce a category-knowledge-based feature space optimization module, thereby significantly boosting the model's representational power. Finally, we design a multi-stage cross-modal feature alignment mechanism to effectively mitigate the challenges of cross modal feature alignment. Furthermore, we curate the fine-grained STBIR benchmark dataset to rigorously validate the efficacy of our proposed framework and to provide data support as a reference for subsequent related research. Extensive experiments demonstrate that the proposed STBIR framework significantly outperforms state of the art methods.

Reasoning-intensive retrieval requires text representations to capture not only semantic similarity, but also the reasoning needed to determine relevance under a given retrieval instruction. Existing reasoning-enhanced embedding models improve retrieval by incorporating reasoning information into dense representations, yet their supervision is typically dominated by the final retrieval objective. As a result, latent reasoning trajectories may learn shortcut reasoning patterns that preserve retrieval performance without producing meaningful incremental retrieval gains. We propose Retrieval Grounding Latent Reasoning (RGLT), a latent reasoning framework for dense retrieval that explicitly connects intermediate latent transitions with retrieval improvements. RGLT performs non-autoregressive reasoning in hidden space through an instruction-conditioned latent reasoning trajectory constructed from silent tokens. It combines process-supervised explicit-to-implicit distillation with retrieval-grounded supervision, using stage-wise CoT reconstruction to shape intermediate latent states and retrieval-effect credit to optimize incremental retrieval gains across the latent reasoning trajectories. Experiments on reasoning-intensive retrieval benchmarks show that RGLT consistently outperforms strong baselines while preserving efficient embedding inference.

Dense retrieval enables effective semantic matching, yet the semantic factors underlying its relevance scores remain opaque. Sparse autoencoders (SAEs) support feature-level explanations by decomposing dense embeddings into sparse semantic features. However, the document semantics underlying a retriever's relevance scores may not be sufficiently explicit in the embeddings to be recovered as interpretable SAE features. We introduce Xetrieval, a framework for explaining dense retrieval through feature elicitation. Feature Elicitors make retrieval-relevant aspects of document meaning more explicit through auxiliary views, facilitating their identification as interpretable SAE features. A shared sparse dictionary learns from these varied representations to express query--document relations through describable features. We assess their influence on relevance scores through interventions on the original document embeddings. Experiments show that Xetrieval yields more interpretable features than an SAE baseline and identifies feature sets with a stronger influence on the retriever's scores. Task-level feature modulation further demonstrates more effective retrieval steering. These findings highlight the value of feature elicitation for relevance attribution and retrieval steering.