DEO: Training-Free Direct Embedding Optimization for Negation-Aware Retrieval

Taegyeong Lee; Jiwon Park; Seunghyun Hwang; JooYoung Jang

cs.CLMar 10, 2026

DEO: Training-Free Direct Embedding Optimization for Negation-Aware Retrieval

Authors: Taegyeong Lee, Jiwon Park, Seunghyun Hwang, JooYoung Jang

Organizations: Department of Industrial Engineering, Hanyang University · Department of Applied Data Science, Sungkyunkwan University

Abstract

Recent advances in Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) have enabled diverse retrieval methods. However, existing retrieval methods often fail to accurately retrieve results for negation and exclusion queries. To address this limitation, prior approaches rely on embedding adaptation or fine-tuning, which introduce additional computational cost and deployment complexity. We propose Direct Embedding Optimization (DEO), a training-free method for negation-aware text and multimodal retrieval. DEO decomposes queries into positive and negative components and optimizes the query embedding with a contrastive objective. Without additional training data or model updates, DEO outperforms baselines on NegConstraint, with gains of +0.0738 nDCG@10 and +0.1028 MAP@100, while improving Recall@5 by +6% over OpenAI CLIP in multimodal retrieval. These results demonstrate the practicality of DEO for negation- and exclusion-aware retrieval in real-world settings.

Figures & tables

Figure 1: Overview of the proposed Direct Embedding Optimization (DEO). (a) Given an input query containing negation, we use an LLM to decompose it into positive and negative sub-queries. (b) The input query embedding is then optimized with a contrastive loss by pulling it closer to positive query embeddings and pushing it farther from negative query embeddings, enabling negation- and exclusion-aware retrieval.

	NegConstraint		NevIR
	MAP	nDCG	Pairwise
BGE-small-en-v1.5	0.6702	0.7372	0.1569
w/ DEO	0.7302	0.7795	0.1894
BGE-large-en-v1.5	0.6299	0.7139	0.2552
w/ DEO	0.7327	0.7877	0.2776
BGE-M3	0.6374	0.7250	0.2668

Table 1: Performance on the NegConstraint Xu et al. (2025) benchmark for negation retrieval. “w/ DEO” represents the performance after applying our proposed method. All nDCG scores correspond to nDCG@10.

	Recall@5
OpenAI CLIP	0.4792
w/ DEO	0.5392
CLIP-laion400m	0.5248
w/ DEO	0.5737
CLIP-datacomp	0.4984
w/ DEO	0.5513

Table 2: Performance on the COCO-Neg benchmark for text-to-image negation retrieval Alhamoud et al. (2025) . “w/ DEO” represents the performance after applying our proposed method.

	NegConstraint
Method	MAP	nDCG@10
BGE-S w/ Qwen	0.7144	0.7705
BGE-S w/ GPT	0.7302	0.7795
BGE-L w/ Qwen	0.6859	0.7553
BGE-L w/ GPT	0.7327	0.7877
BGE-M3 w/ Qwen	0.7280	0.7871

Table 3: Performance comparison on the NegConstraint dataset using different LLM backbones for query decomposition. Qwen refers to Qwen2.5-1.5B-Instruct, and GPT refers to GPT-4.1-nano. BGE-S, BGE-L denote BGE-small-en-v1.5, BGE-large-en-v1.5, respectively.

	COCO-Neg
Method	Recall@5
CLIP-laion400m w/ Qwen	0.5656
CLIP-laion400m w/ GPT	0.5737
NegCLIP w/ Qwen	0.6872
NegCLIP w/ GPT	0.6980

Table 4: Performance comparison on COCO-Neg using different LLM backbones for query decomposition.

	NegConstraint		COCO-Neg
Parameters	MAP	nDCG@10	Recall@5
Baseline	0.6374	0.7250	0.4792
0.2, 1.0, 1.0	0.7379	0.7946	0.5349
0.2, 1.0, 2.0	0.7366	0.7942	0.5275
0.2, 2.0, 1.0	0.7160	0.7769	0.5237
1.0, 1.0, 1.0	0.6713	0.7510	0.5392

Table 5: Effect of varying the weight balancing parameters

\lambda_{o}

\lambda_{p}

, and

\lambda_{n}

on the NegConstraint and COCO-Neg. NegConstraint results use BGE-M3, and COCO-Neg results use OpenAI CLIP. Optimization step is 20.

	NegConstraint
Method	MAP	nDCG@10
BGE-M3	0.6374	0.7250
Ours (only decompose, AVG)	0.6451	0.7312
Ours (only decompose, RRF)	0.6641	0.7417
Ours (full)	0.7379	0.7946

Table 6: Ablation results on NegConstraint with BGE-M3. Query decomposition alone yields limited gains, while most performance improvements arise from embedding optimization rather than decomposition itself.

Figure 2: Retrieval performance with respect to the number of optimization steps on NegConstraint .

Figure 3: Retrieval performance with respect to the number of optimization steps on COCO-Neg .

Figure 4: Examples of query decomposition. In NegConstraint (a) and COCO-Neg (b), the LLM decomposes queries into positive sub-queries capturing desired elements and negative sub-queries targeting excluded concepts.

Figure 5: Trajectory of optimized query embedding $e_{u}$ in PCA-projected space. The initial embedding (black $\bullet$ ) moves toward the final state (blue $\bullet$ ). Positive examples (green $\triangle$ ) and the ground-truth (yellow $\star$ ) act as attractors, while negative examples (red $\times$ ) exert repelling forces. Other corpus embeddings are shown in light gray.

Figure 6: Optimization trajectory of $e_{u}$ in PCA-projected CLIP space. Initial (black $\bullet$ ) embedding evolves toward final state (blue $\bullet$ ), attracted by positives (green $\triangle$ ) and ground-truth (yellow $\star$ ) while repelled by negatives (red $\times$ ). Gray points denote the corpus.

Dense Retrieval

Dense retrieval uses learned vector representations to find relevant documents, passages, products, or tools by semantic similarity.

See all 152 papers

E-SENS: Exclusion-Sensitive Penalization for Negative-Constraint Retrieval
Aug 31, 2026Yerang Kim, Jiyoon Myung, Joohyung HanLearning to Rank Dense Retrieval
Independent Researcher · Lomin
Learning to Retrieve via Reinforcement Learning in Embedding Space
Oct 6, 2026Qi Liu, Fengming Liang, Yiqun Chen +2RL for IR Text Embedding Models
Renmin University of China
DEPT: Document Embedding Preservation Tuning for Unified Query Expansion and Retrieval
Aug 18, 2026Jingyuan Wang, Richong Zhang, Zhijie Nie +2Dense Retrieval Query Expansion
Beihang University, China

Aug 31, 2026J/K move · Enter open · S save

E-SENS: Exclusion-Sensitive Penalization for Negative-Constraint Retrieval

Yerang Kim, Jiyoon Myung, Joohyung Han

Independent Researcher · Lomin

Learning to Rank Dense Retrieval

Open paper AskPDF arXiv

Retrieval-augmented language models can fail to respect negative constraints when the retriever supplies evidence about concepts the user explicitly excluded. Beyond explicit negation, queries may ask for answers that include one concept while excluding another, or for entities that belong to a category but differ from a closely related instance. Because the excluded concept still appears in the query text, dense retrievers may assign high similarity to documents about that concept even when the user asks to avoid it. We introduce E-SENS, a training-free reranking method for negation-sensitive retrieval. E-SENS extracts a compact trap query for the excluded side and subtracts trap-query similarity from the original-query retrieval score. On ExcluIR, E-SENS shows a clear recall-violation trade-off across four embedding models and reduces trap retrieval at recall-preserving settings.

Abstract

Dense Retrieval

Explore similar work

E-SENS: Exclusion-Sensitive Penalization for Negative-Constraint Retrieval

Learning to Retrieve via Reinforcement Learning in Embedding Space

DEPT: Document Embedding Preservation Tuning for Unified Query Expansion and Retrieval