StructXLIP: Enhancing Vision-language Models with Multimodal Structural Cues

Zanxi Ruan; Songqun Gao; Qiuyu Kong; Yiming Wang; Marco Cristani

cs.CVFeb 23, 2026

StructXLIP: Enhancing Vision-language Models with Multimodal Structural Cues

Authors: Zanxi Ruan, Songqun Gao, Qiuyu Kong, Yiming Wang, Marco Cristani

Organizations: University of Verona · Sapienza University of Rome · University of Trento · Fondazione Bruno Kessler ·

Abstract

Edge-based representations are fundamental cues for visual understanding, a principle rooted in early vision research and still central today. We extend this principle to vision-language alignment, showing that isolating and aligning structural cues across modalities can greatly benefit fine-tuning on long, detail-rich captions, with a specific focus on improving cross-modal retrieval. We introduce StructXLIP, a fine-tuning alignment paradigm that extracts edge maps (e.g., Canny), treating them as proxies for the visual structure of an image, and filters the corresponding captions to emphasize structural cues, making them "structure-centric". Fine-tuning augments the standard alignment loss with three structure-centric losses: (i) aligning edge maps with structural text, (ii) matching local edge regions to textual chunks, and (iii) connecting edge maps to color images to prevent representation drift. From a theoretical standpoint, while standard CLIP maximizes the mutual information between visual and textual embeddings, StructXLIP additionally maximizes the mutual information between multimodal structural representations. This auxiliary optimization is intrinsically harder, guiding the model toward more robust and semantically stable minima, enhancing vision-language alignment. Beyond outperforming current competitors on cross-modal retrieval in both general and specialized domains, our method serves as a general boosting recipe that can be integrated into future approaches in a plug-and-play manner. Code and pretrained models are publicly available at: https://github.com/intelligolabs/StructXLIP.

Figures & tables

Figure 1: StructXLIP performs fine-tuning by adding to the standard image-text alignment with multimodal structural cue: edge maps and structure-centric captions. StructXLIP consistently improves downstream vision–language retrieval inference.

Figure 2: Examples of Visual (left) and Textual (right) extraction.

Figure 3: (i) Overview of StructXLIP fine-tuning which operates in two stages. The first structure-centric multimodal extraction stage extracts structural views by generating edge maps from original images via edge detector and performing lexicon filtering on original captions to remove appearance-related terms. The second structure-centric multimodal alignment stage fine-tunes the encoders $f_{\mathtt{img}}$ and $f_{\mathtt{txt}}$ by joining original image-text alignment $\mathcal{L}_{I,T}$ with our newly introduced structure-centric alignment objectives: the structure-centric image-text alignment loss $\mathcal{L}_{I^{\prime},T^{\prime}}$ enforces global structural alignment, the Local structure-centric image-text alignment loss $\mathcal{L}_{I^{\prime},T^{\prime}}^{local}$ captures local compositional semantics, and the Consistency regularization loss $\mathcal{L}_{I,I^{\prime}}$ aligns raw images and edge maps to prevent representation drift. At inference, only color image ( $I$ ) and captions ( $T$ ) are input to the fine-tuned $f_{\mathtt{img}}$ and $f_{\mathtt{txt}}$ , neither edge extraction nor lexicon filtering is required. (ii) Zoom-in on the Local structure-centric image-text alignment loss $\mathcal{L}_{I^{\prime},T^{\prime}}^{local}$ .

Figure 4: Empirical analysis confirming the information-theoretic view of StructXLIP. a) Loss convergence; b) Gradient norms and ratio evolution; c) Inter-task cosine similarity.

Method	SKETCHY						INSECT						DOCCI						DCI
Method	R@1	R@5	R@10	R@1	R@5	R@10	R@1	R@5	R@10	R@1	R@5	R@10	R@1	R@5	R@10	R@1	R@5	R@10	R@1	R@5	R@10	R@1	R@5	R@10
Long-CLIP [ECCV’24]	$54.32$	$80.14$	$88.43$	$52.76$	$80.31$	$88.08$	$8.20$	$23.83$	$34.97$	9.41	$24.78$	37.31	$64.49$	$87.67$	$93.43$	$63.08$	$87.45$	$93.14$	$59.23$	$80.89$	$87.04$	$60.13$	$81.44$	$87.54$
FineLIP [CVPR’25]	$40.59$	$71.16$	$81.78$	$40.33$	$72.11$	$82.38$	$8.46$	$23.32$	$33.59$	$6.86$	$23.75$	$34.46$	$67.80$	$90.22$	$94.84$	$66.39$	$89.12$	$94.47$	$66.13$	$85.34$	$89.79$	$64.58$	$84.59$	$89.54$
SmartCLIP [CVPR’25]	$50.73$	$81.09$	94.56	$51.30$	$80.83$	94.04	$4.84$	$16.84$	$34.63$	$4.66$	$15.46$	$34.02$	$74.92$	$94.08$	$97.31$	$74.91$	$94.04$	97.29	$69.88$	$86.64$	94.05	$70.94$	$87.04$	$92.77$
GOAL [CVPR’25]	63.21	87.13	$93.44$	62.44	87.82	$92.31$	8.81	24.35	35.84	$8.55$	25.91	$36.18$	79.47	96.65	98.69	79.43	96.14	97.25	72.64	89.89	93.70	72.84	90.50	93.20
StructXLIP	69.86	90.85	95.42	68.22	90.67	95.68	9.93	26.60	38.34	9.50	26.60	39.64	83.04	97.06	98.96	81.59	96.94	98.78	75.90	90.00	95.15	74.39	89.90	94.30

Table 1: Crossmodal retrieval performance of CLIP-based finetuning methods on multiple datasets . We report Recall@K (%) on both Ay Text

\rightarrow

Image and Ay Image

\rightarrow

Text settings. Each row represents one method evaluated on two datasets (left–right paired). Best results in bold ; second best underlined .

\Delta

denotes the margin over our best competitor, with gain in

\uparrow

green .

Figure 5: Data amount vs. performance: Retrieval recall R@1 in case of 5%, 20%, 50% and the standard 100% of training data. The improvement line indicates the margin of StructXLIP against the second best approach, trained with the same amount of data.

Method	SKETCHY						DOCCI
Method	R@1	R@5	R@10	R@1	R@5	R@10	R@1	R@5	R@10	R@1	R@5	R@10
Long-CLIP	$54.32$	$80.14$	$88.43$	$52.76$	$80.31$	$88.08$	$64.49$	$87.67$	$93.43$	$63.08$	$87.45$	$93.14$
+our $\mathcal{L}^{*}$	59.24	85.32	91.45	59.59	84.37	91.02	67.67	90.82	95.59	67.92	90.16	95.10
$\Delta$	$\uparrow$ 4.92	$\uparrow$ 5.18	$\uparrow$ 3.02	$\uparrow$ 6.83	$\uparrow$ 4.06	$\uparrow$ 2.94	$\uparrow$ 3.18	$\uparrow$ 3.15	$\uparrow$ 2.16	$\uparrow$ 4.84	$\uparrow$ 2.71	$\uparrow$ 1.96
FineLIP	$40.59$	$71.16$	$81.78$	$40.33$	$72.11$	$82.38$	$67.80$	$90.22$	$94.84$	$66.39$	$89.12$	$94.47$
+our $\mathcal{L}^{*}$	59.15	85.23	91.28	58.55	84.54	90.07	74.06	94.24	97.35	72.94	93.27	96.55

Table 2: Plug-and-play enhancement of our

\mathcal{L}^{*}

on CLIP-based finetuning. Results on SKETCHY and DOCCI for Ay Text

\rightarrow

Image and Ay Image

\rightarrow

Text retrieval. Upper: full-parameter finetuning; lower: parameter-efficient tuning. Our method consistently boosts diverse CLIP variants. Best in bold , with gain in

\uparrow

green .

Train on DCI → Test on DCI vs DOCCI
Setting	R@1	R@5	R@10	R@1	R@5	R@10
Long-CLIP (DCI→DCI)	59.23	80.89	87.04	60.13	81.44	87.54
Long-CLIP (DCI→DOCCI)	58.73	84.66	90.75	58.08	83.53	90.47
GOAL (DCI→DCI)	72.64	89.89	93.70	72.84	90.50	93.20
GOAL (DCI→DOCCI)	71.22	92.39	96.47	72.18	92.88	96.49
StructXLIP (DCI→DCI)	75.90	90.00	95.15	74.39	89.99	94.30

Table 3: Cross-domain generalization between DCI and DOCCI. Train on one dataset and test on another. Values are Recall@K (%), using Ay Text

\rightarrow

Image and Ay Image

\rightarrow

Text retrieval. In-domain best in italic bold , cross-domain best in bold .

Figure 6: (a) Retrieval results; (b) GRAD-CAM [ 27 ] highlighting cross-modal attention between the object-centric text and image.

Panel of Table 4 — Table 4: Ablation on losses. Results on Ay Text $\rightarrow$ Image and Ay Image $\rightarrow$ Text retrieval for SKETCHY and DOCCI. Each row adds one additional loss term to the previous configuration.

(a) Visual Extraction	SKETCHY						DOCCI
(a) Visual Extraction	R@1	R@5	R@10	R@1	R@5	R@10	R@1	R@5	R@10	R@1	R@5	R@10
Methods
Canny* [ 2 ] [F]	69.86	90.85	95.42	68.22	90.67	95.68	83.04	97.06	98.96	81.59	96.94	98.78
LoG [ 20 ] [F]	68.48	89.64	94.91	67.36	90.24	94.47	83.59	97.36	99.20	81.98	97.41	99.04
HED [ 43 ] [L]	68.91	90.67	95.43	66.93	90.41	94.47	83.14	97.18	98.98	81.73	97.08	98.69
LAD [ 52 ] [L]	69.17	90.93	95.42	66.93	90.16	94.91	82.69	97.10	98.90	81.08	96.84	98.84

Table 5: Ablation on extraction methods. Ay Text

\rightarrow

Image and Ay Image

\rightarrow

Text retrieval on SKETCHY and DOCCI datasets. [F] indicates filter-based and [L] indicates learning-based edge extraction methods. Bold numbers indicate the best results within each subtable. * denotes the option in our StructXLIP.

Figure 1: Illustration of SKETCHY dataset samples across different edge-map representations. For each example, we show the original RGB image together with its caption, followed by Canny, LoG, HED, LAD, and P2S edge maps, as well as the filtered caption $T^{\prime}$ that preserves only structure-centric information.

Figure 2: Illustration of INSECT dataset samples across different edge-map representations.

Figure 3: Illustration of DOCCI dataset samples across different edge-map representations.

Figure 4: Illustration of DCI dataset samples across different edge-map representations.

Panel of Figure 5 — Figure 5: Prompt and response used in our LLM-based construction of the general appearance vocabulary for lexicon filtering. This lexicon is generated only once during preprocessing; the pipeline consistently uses this fixed lexicon thereafter, ensuring full reproducibility.

Figure 6: Statistical analysis of the lexicon filtering effect across the four datasets. For analysis, removed tokens are coarsely grouped into color words and material words . (A) Intervention scope: percentage of captions in which at least one word was removed. (B) Target specificity: composition of removed tokens (computed over the modified captions in Panel A), showing the proportion of color vs. material words. (C) Modification intensity: average number of words removed per caption, with the percentage relative to the original caption length shown in parentheses.

Figure 7: The convergence behavior of the two contrastive losses $\mathcal{L}_{I,T}$ and $\mathcal{L}_{I^{\prime},T^{\prime}}$ (a), the evolution of their gradient norms and their ratio (b), and the cosine similarity between the corresponding gradients (c), evaluated on all two datasets under different percentages of dataset size.

Figure 8: The convergence behavior of the two contrastive losses $\mathcal{L}_{I,T}$ and $\mathcal{L}_{I^{\prime},T^{\prime}}$ (a), the evolution of their gradient norms and their ratio (b), and the cosine similarity between the corresponding gradients (c), evaluated on all two datasets under different percentages of dataset size.

Setting	R@1	R@5	R@10	R@25	R@50	R@1	R@5	R@10	R@20	R@50
SKETCHY
General Lexicon Filter	69.86	90.85	95.42	98.36	99.22	68.22	90.67	95.68	97.75	99.22
Domain-Specific Lexicon Filter	69.00	90.87	95.60	98.33	99.22	68.14	90.53	95.21	97.40	99.10
INSECT
General Lexicon Filter	9.93	26.60	38.34	56.99	69.34	9.50	26.60	39.64	54.92	68.65
Domain-Specific Lexicon Filter	9.67	28.50	39.46	55.09	67.70	9.07	28.07	39.81	54.32	68.48

Table 1: Comparison of general vs. domain-specific appearance lexicons. Ay Text

\rightarrow

Image and Ay Image

\rightarrow

Text retrieval on SKETCHY and INSECT. Bold denotes the best performance.

Fine-tune on DOCCI (General) → Test on Sketchy (Specific)
Setting	R@1	R@5	R@10	R@1	R@5	R@10
Long-CLIP (In-domain)	64.49	87.67	93.43	63.08	87.45	93.14
Long-CLIP (Cross-domain)	8.12	20.03	28.24	10.71	24.27	33.68
GOAL (In-domain)	79.47	96.65	98.69	79.43	96.14	97.25
GOAL (Cross-domain)	8.72	19.60	27.46	9.76	26.86	38.17
StructXLIP (In-domain)	83.04	97.06	98.96	81.59	96.94	98.78

Table 2: Cross-domain generalization from General (DOCCI) to Specific (Sketchy). Models are trained on the general dense-captioned dataset (DOCCI) and tested on abstract sketches (Sketchy). Due to the significant domain shift, all models experience a performance drop, but StructXLIP maintains better robustness. Values are Recall@K (%). In-domain results (DOCCI

\to

DOCCI) are in italic for reference. Best cross-domain results are in bold .

SKETCHY
Method	R@1	R@5	R@10	R@25	R@50	R@1	R@5	R@10	R@25	R@50
Long-CLIP [ECCV’24]	54.32 $\pm$ 0.46	80.14 $\pm$ 2.27	88.43 $\pm$ 1.88	95.25 $\pm$ 0.78	98.27 $\pm$ 0.75	52.76 $\pm$ 1.18	80.31 $\pm$ 1.89	88.08 $\pm$ 1.50	95.16 $\pm$ 1.03	97.75 $\pm$ 1.00
FineLIP [CVPR’25]	40.59 $\pm$ 1.91	71.16 $\pm$ 1.28	81.78 $\pm$ 0.63	91.27 $\pm$ 1.72	95.94 $\pm$ 0.57	40.33 $\pm$ 1.04	72.11 $\pm$ 0.05	82.38 $\pm$ 0.11	91.45 $\pm$ 1.85	95.77 $\pm$ 0.69
SmartCLIP [CVPR’25]	50.73 $\pm$ 1.11	81.09 $\pm$ 0.93	94.56 $\pm$ 1.67	96.11 $\pm$ 0.44	99.05 $\pm$ 1.28	51.30 $\pm$ 0.52	80.83 $\pm$ 1.89	94.04 $\pm$ 0.73	95.51 $\pm$ 1.21	98.96 $\pm$ 0.36
GOAL [CVPR’25]	63.21 $\pm$ 0.47	87.13 $\pm$ 1.58	93.44 $\pm$ 0.35	97.67 $\pm$ 1.09	99.05 $\pm$ 0.19	62.44 $\pm$ 1.37	87.82 $\pm$ 0.95	92.31 $\pm$ 1.44	96.98 $\pm$ 0.29	99.00 $\pm$ 0.54
StructXLIP	69.86 $\pm$ 0.46	90.85 $\pm$ 0.09	95.42 $\pm$ 0.07	98.61 $\pm$ 0.35	99.22 $\pm$ 0.00	68.22 $\pm$ 0.45	90.67 $\pm$ 0.12	95.68 $\pm$ 0.13	98.03 $\pm$ 0.39	99.08 $\pm$ 0.20

Table 3: Crossmodal retrieval at full ranks on SKETCHY, INSECT, DOCCI and DCI. We report mean Recall@K (%)

\pm

standard deviation for K = 1, 5, 10, 25, 50 on both Ay Text

\rightarrow

Image and Ay Image

\rightarrow

Text . All results are averaged over three independent runs with random seeds 42, 1337 and 3407. Bold indicates the best result, while underline denotes the second-best.

Method	SKETCHY										INSECT
Method	R@1	R@5	R@10	R@25	R@50	R@1	R@5	R@10	R@25	R@50	R@1	R@5	R@10	R@25	R@50	R@1	R@5	R@10	R@25	R@50
Long-CLIP	$54.32$	$80.14$	$88.43$	$95.25$	$98.27$	$52.76$	$80.31$	$88.08$	$95.16$	$97.75$	$8.20$	$23.83$	$34.97$	$51.21$	$63.30$	9.41	$24.78$	$37.31$	$53.18$	$63.39$
+our $\mathcal{L}^{*}$	59.24	85.32	91.45	96.20	98.53	59.59	84.37	91.02	96.29	98.45	9.24	25.39	36.36	52.33	66.84	$9.38$	27.29	38.60	55.35	66.15
$\Delta$	$\uparrow$ 4.92	$\uparrow$ 5.18	$\uparrow$ 3.02	$\uparrow$ 0.95	$\uparrow$ 0.26	$\uparrow$ 6.83	$\uparrow$ 4.06	$\uparrow$ 2.94	$\uparrow$ 1.13	$\uparrow$ 0.70	$\uparrow$ 1.04	$\uparrow$ 1.56	$\uparrow$ 1.39	$\uparrow$ 1.12	$\uparrow$ 3.54	$\downarrow$ 0.03	$\uparrow$ 2.51	$\uparrow$ 1.29	$\uparrow$ 2.17	$\uparrow$ 2.76
FineLIP	$40.59$	$71.16$	$81.78$	$91.27$	$95.94$	$40.33$	$72.11$	$82.38$	$91.45$	$95.77$	$8.46$	$23.32$	$33.59$	$51.21$	$66.58$	6.86	$23.75$	$34.46$	$52.42$	$65.37$
+our $\mathcal{L}^{*}$	59.15	85.23	91.28	96.63	99.40	58.55	84.54	90.07	96.20	99.05	8.89	23.83	35.75	53.63	67.18	$6.56$	23.76	36.01	53.45	66.84

Table 4: Plug-and-play enhancement of our

\mathcal{L}^{*}

on CLIP-based finetuning. Results on SKETCHY, INSECT, DOCCI, and DCI for Ay Text

\rightarrow

Image and Ay Image

\rightarrow

Text retrieval. We report R@1, R@5, R@10, R@25, and R@50. Upper: SKETCHY and INSECT; Lower: DOCCI and DCI. Best in bold , with gain in

\uparrow

green .

Setting	R@1	R@5	R@10	R@25	R@50	R@1	R@5	R@10	R@25	R@50	R@1	R@5	R@10	R@25	R@50	R@1	R@5	R@10	R@25	R@50
5% Data	SKETCHY										INSECT
Long-CLIP [ECCV’24]	21.42	47.58	59.59	76.77	86.18	23.40	49.14	62.44	78.24	87.31	2.50	6.99	10.88	18.83	27.98	2.68	8.03	12.61	20.55	30.66
FineLIP [CVPR’25]	30.92	60.28	71.68	85.49	91.88	31.00	59.50	70.12	84.97	92.57	2.50	8.12	11.87	22.13	32.99	3.28	8.33	13.04	22.31	32.06
SmartCLIP [CVPR’25]	25.22	53.02	77.12	81.35	90.59	27.98	54.15	75.76	82.90	91.97	2.68	7.43	11.66	19.86	30.14	3.37	8.44	13.15	21.66	32.25
GOAL [CVPR’25]	33.42	63.99	74.18	86.23	92.49	35.05	64.08	75.82	88.69	93.53	2.68	7.51	11.83	22.11	31.35	3.34	8.20	12.33	21.24	31.78
StructXLIP	36.70	68.05	78.07	88.43	94.39	35.75	68.05	77.46	89.12	94.91	3.33	8.46	12.87	22.28	33.68	3.57	8.98	13.56	22.54	33.16

Table 5: Data Efficiency Analysis. We report mean Recall@K (%) on SKETCHY, INSECT, DOCCI, and DCI using 5% , 20% , 50% respectively of the training data. Results are reported for K = 1, 5, 10, 25, 50 on both Ay Text

\rightarrow

Image and Ay Image

\rightarrow

Text . Bold indicates the best result, while underline denotes the second-best.

Figure 9: Qualitative results of StructXLIP and the second-best method GOAL on all four datasets. We present the attention maps between the color-highlighted texts on the visual counterpart. Compared to GOAL, StructXLIP overall shows a better correspondence between the rich textual descriptions and the visual objects.

Panel of Figure 10 — Figure 10: Prompt and response used in our LLM-based construction of the domain-specific appearance vocabulary lists for lexicon filtering.

Vision-Language Alignment

Vision-language alignment connects visual and textual representations so models can match, interpret, and ground language in images and video.

See all 361 papers

Apr 29, 2026

Topology-Aware Representation Alignment for Semi-Supervised Vision-Language Learning

Vision-language models have shown strong performance, but they often generalize poorly to specialized domains. While semi-supervised vision-language learning mitigates this limitation by leveraging a small set of labeled image-text pairs together with abundant unlabeled images, existing methods remain fundamentally pairwise and fail to model the global structure of multimodal representation manifolds. Most existing topology-based alignment methods rely on persistence diagram matching, which neither guarantees geometric alignment nor uses the image-text pairing information central to vision-language learning. We propose Topology-Aware Representation Alignment for Semi-Supervised Vision-Language Learning (ToMA), a framework that uses persistent homology to decompose pairwise relations into

H_0

-death and

H_1

-birth relations and aligns them across modalities through available cross-modal correspondences. This lightweight decomposition captures both connectivity and cycle-closing structure without constructing 2-simplices. Experiments show clear improvements on remote sensing, while fashion results reveal strong gains in classification and particularly strong retrieval improvements for difficult queries. Additional analysis shows that ToMA provides more consistent gains than alternative topology-based objectives and that lightweight

H_1

-birth edges provide useful higher-order structural signals.

Junwon You, Mihyun Jang, Sangwoo Mo +1

KAIST · POSTECH

Cross-Modal Learning Cross-Modal Representation Learning Vision-Language Alignment Semi-Supervised Learning Topological Representation Learning Multimodal Understanding

Ask PDF

Mar 17, 2026

Evo-Retriever: LLM-Guided Curriculum Evolution with Viewpoint-Pathway Collaboration for Multimodal Document Retrieval

Visual-language models (VLMs) excel at data mappings, but real-world document heterogeneity and unstructuredness disrupt the consistency of cross-modal embeddings. Recent late-interaction methods enhance image-text alignment through multi-vector representations, yet traditional training with limited samples and static strategies cannot adapt to the model's dynamic evolution, causing cross-modal retrieval confusion. To overcome this, we introduce Evo-Retriever, a retrieval framework featuring an LLM-guided curriculum evolution built upon a novel Viewpoint-Pathway collaboration. First, we employ multi-view image alignment to enhance fine-grained matching via multi-scale and multi-directional perspectives. Then, a bidirectional contrastive learning strategy generates "hard queries" and establishes complementary learning paths for visual and textual disambiguation to rebalance supervision. Finally, the model-state summary from the above collaboration is fed into an LLM meta-controller, which adaptively adjusts the training curriculum using expert knowledge to promote the model's evolution. On ViDoRe V2 and MMEB (VisDoc), Evo-Retriever achieves state-of-the-art performance, with nDCG@5 scores of 65.2% and 77.1%.

Weiqing Li, Jinyue Guo, Yaqi Wang +4

Alibaba Cloud Computing

Multimodal IR Cross-Modal Retrieval Late Interaction Retrieval Multimodal Contrastive Learning

Ask PDF

Jun 5, 2026

TEVI: Text-Conditioned Editing of Visual Representations via Sparse Autoencoders for Improved Vision-Language Alignment

Vision-language models such as CLIP are highly useful for diverse tasks due to their shared image-text embedding space. Despite this, the image and text embeddings are often poorly aligned, affecting downstream performance. Recent work has hypothesized that this can be attributed to an information imbalance: images contain more information than their captions describe. In this work, we propose TEVI, a framework that uses captions as a signal for what to retain from image embeddings. Specifically, we use sparse autoencoders to disentangle image embeddings and train a masking module to selectively reconstruct the embedding based on a given caption. In a controlled setup with synthetic captions, we show that TEVI is effective at preserving caption-described attributes while discarding others. We find that this extends to CLIP models trained on natural images, where TEVI learns to mask meaningfully and allows retrieval based on conditioning. Finally, we use TEVI to achieve improved retrieval performance across coarse-grained and fine-grained benchmarks. Code available at https://github.com/neuroexplicit-saar/TEVI.

Sweta Mahajan, Sukrut Rao, Jiahao Xie +2

Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrücken, Germany · Department of Language Science and Technology, Saarland University, Saarbrücken, Germany

Vision-Language Retrieval Vision-Language Alignment

Ask PDF

Abstract

Vision-Language Alignment

Explore similar work

Topology-Aware Representation Alignment for Semi-Supervised Vision-Language Learning

Evo-Retriever: LLM-Guided Curriculum Evolution with Viewpoint-Pathway Collaboration for Multimodal Document Retrieval

TEVI: Text-Conditioned Editing of Visual Representations via Sparse Autoencoders for Improved Vision-Language Alignment