Gap Detection: Mitigating Homogenization through Demand and Defensibility

Xucheng Yu; Haibo Jin; Xiaoqi Wang; Runze Kong; Haohan Wang

cs.CLOct 14, 2026

Gap Detection: Mitigating Homogenization through Demand and Defensibility

Authors: Xucheng Yu, Haibo Jin, Xiaoqi Wang, Runze Kong, Haohan Wang

Organizations: University of Illinois Urbana-Champaign · Fudan University · University of Michigan, Ann Arbor

Abstract

Language models often produce semantically similar answers to open-ended questions, even when many plausible alternatives exist. Generating useful alternatives requires identifying both what the query calls for and what existing responses already cover. We introduce Gap Detection, a framework that searches for relevant but unoccupied directions in a shared text embedding space. Its objective, GapScore, combines query relevance (Demand) with dissimilarity to the nearest existing item (Defensibility). The framework optimizes this product on the unit sphere, maps candidate directions to concept words with a language model, and uses task-specific filtering and constrained prompting to turn those concepts into text. On five curated topics and 90 INFINITY-CHAT queries, the reported mean relative reductions in output Collision are 12.8% and 10.8%, respectively. We also instantiate the framework for generative engine optimization over four product markets and for iterative synthetic-data augmentation. In the latter setting, with 100 training examples per method, Gap Detection reduces Collision from 0.324 to 0.298 relative to undirected accumulation and increases the Vendi Score from 27.16 to 29.20; the accuracy difference is one example on a 40-example test set. Component ablations expose the trade-off between semantic separation and relevance, while also showing that better direction scores do not uniformly yield better downstream metrics. These results support explicit gap search as a practical mechanism for diversifying content, with task validity checked separately from geometric novelty.

Figures & tables

Figure 1: Overview of Gap Detection. The pipeline embeds existing content and the query, searches the unit hypersphere for a high-GapScore direction, maps that direction to a concrete concept, and verifies the resulting candidate before accepting it. A rejected candidate triggers another search; an accepted direction is added to the occupied set so later directions cover a different region.

Evaluation	Before $\downarrow$	After $\downarrow$	Reduction $\uparrow$
Time metaphor	0.7362	0.6573	10.7%
Loneliness metaphor	0.7449	0.6634	10.9%
Redefine courage	0.8046	0.7160	11.0%
Memory as substance	0.8163	0.7080	13.3%
Hope via natural elements	0.8178	0.6712	17.9%
Five-topic mean (reported)	0.784	0.683	12.8%

Table 1: Reported Collision before and after Gap Detection. The last column is the reported mean relative reduction. Topic averages and the 90-query aggregate are separate evaluations.

Method	Direction GapScore $\uparrow$	Collision reduction $\uparrow$	Novelty pass $\uparrow$
Random direction	0.003	10.3%	100%
Demand only	0.298	7.8%	92%
Defensibility only	$-0.228$	11.5%	96%
GapScore	0.375	10.7%	84%

Table 2: Direction-selection ablation on five topics. Direction GapScore evaluates the optimized vectors; the other columns evaluate realized outputs. Best values in each column are bold. These runs are separate from Table 1 .

Market or method	Direction GapScore $\uparrow$	PAWC $\uparrow$	Product $\uparrow$
Water filters	0.410	0.518	0.213
Protein powder	0.479	0.540	0.258
Sports water bottles	0.398	0.513	0.204
Yoga mats	0.403	0.524	0.211
Reported market mean	0.423	0.524	0.221
Random direction	$-0.006$	0.469	$-0.003$

Table 3: GEO results. Top: best direction per market, selected using GapScore

\times

PAWC. Bottom: a separate ablation averaged across four markets. The blocks have different aggregation rules.

Method	Collision $\downarrow$	Vendi Score $\uparrow$	Accuracy $\uparrow$
No Intervention	0.362	9.06	60.0%
Data Accumulation	0.324	27.16	60.0%
GapScore	0.298	29.20	62.5%

Table 4: Synthetic-data results at generation five. All three methods have 100 training examples. Accuracy is evaluated on 40 disjoint real examples; the 2.5-point difference corresponds to one prediction.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Topic	Lower bound $a$	Search score	Difference	$b=\left\lVert P\mathbf{v}\right\rVert$
Time metaphor	0.6438	0.6492	0.0054	0.5841
Loneliness metaphor	0.7474	0.7493	0.0019	0.6644
Redefine courage	0.7156	0.7389	0.0233	0.6985
Memory as substance	0.5455	0.6011	0.0556	0.8381
Hope via natural elements	0.7731	0.7893	0.0162	0.6343

Appendix

Table 5: Recorded numerical comparison for the geometric lower bound. The search column is the best observed score, not a certified global optimum. All five occupied matrices are reported to have rank 20, giving

d-r=1516

Component	Setting
Encoder	text-embedding-3-small ; 1,536 dimensions; L2 normalization
Concept decoder	Qwen2.5-3B-Instruct-Q6_K; local GGUF on Apple M2
Default filter / generator	GPT-4o-mini
Search starts	Demand direction, demand plus noise, and random direction
Search budget	15 restarts per strategy; 100 steps; step size 0.05
Answering / GEO directions	$K=5$

Appendix

Table 6: Settings used in the experimental pipeline.

Group	Queries	Baseline Collision	Mean reduction
Hypothetical	22	0.79	11.3%
Ideal	7	0.78	11.8%
Metaphor	1	0.92	17.0%
Explain	12	0.77	10.3%
Describe	20	0.78	10.1%
Imagine	10	0.77	9.9%

Appendix

Table 7: Query-group breakdown for the 90-query open-ended generation study.

Topic	Reported GapScore	Baseline Vendi	Vendi/20
Time metaphor	0.165	2.91	14.5%
Loneliness metaphor	0.196	3.16	15.8%
Redefine courage	0.221	2.58	12.9%
Memory as substance	0.220	2.36	11.8%
Hope via natural elements	0.229	2.12	10.6%
Reported mean	0.206	2.63	13.1%

Appendix

Table 8: Additional five-topic diagnostics. Topic-level GapScore and the geometric search scores in Table 5 come from different tables. Vendi values describe the baseline response set; baseline Collision is given in Table 1 .

Model pair	Mean similarity
GPT-4o-mini / Claude	0.660
GPT-4o-mini / Gemini	0.613
GPT-4o-mini / Qwen2.5-3B	0.576
Claude / Gemini	0.629
Claude / Qwen2.5-3B	0.578
Gemini / Qwen2.5-3B	0.554

Appendix

Table 9: Between-model mean similarity in the mixed-source study. Claude denotes Claude-Sonnet-4.5 ( Anthropic,, 2025 ) ; Gemini denotes Gemini-2.0-Flash ( Google DeepMind,, 2025 ) .

Evaluation	Before	After	Reported reduction
Mixed occupied set, $N=20$	0.602	0.561	7.4%

Appendix

Table 10: Mixed-source Collision before and after Gap Detection, for the pooled four-model occupied set.

Occupied-set source	Before	After	Reduction	Novelty pass	GapScore
GPT-4o-mini	0.6860	0.6098	11.2%	86.7%	0.1609
Claude-Sonnet-4.5	0.7014	0.6126	12.7%	100.0%	0.1860
Gemini-2.0-Flash	0.5709	0.5189	8.8%	100.0%	0.1791
Qwen2.5-3B	0.6028	0.5475	8.8%	100.0%	0.1627
Reported mean	0.6403	0.5722	10.4%	96.7%	0.1722

Appendix

Table 11: Separate-source study, averaged over three topics. The realization model is fixed to GPT-4o-mini; novelty pass is the recorded Check-1 statistic.

Realization model	Collision reduction	GapScore	Novelty	Novelty pass
GPT-4o-mini	9.0%	0.1500	0.3684	80.0%
GPT-4o	11.6%	0.1579	0.4314	100.0%
Claude-Sonnet-4.5	14.8%	0.1632	0.4938	100.0%

Appendix

Table 12: Realization-model comparison, averaged over three topics with the occupied set fixed. GapScore differences across realization models reflect the sentences produced by each generator, not changes to the search objective.

Topic	Concept words	Generated excerpt
Time metaphor	poetic, nighttime, tempo	“The night hums a quiet tempo, its stars ticking softly…”
Loneliness metaphor	loner, soliloquy, joylessness	“In the quiet hours of twilight, a solitary tree stands…”
Redefine courage	fearlessness, brave, fortitude	“Grit is the lighthouse that stands tall amid the storm…”
Memory as substance	myelin, tissue, formalin	“Envision a soft, pliable tissue, reminiscent of myelin’s…”
Hope via natural elements	sunlight, warmth, naissance	“A warm breeze dances through the treetops, whispering…”

Appendix

Table 13: Concept words and generated excerpts for the five topics. Excerpts are truncated with ellipses for space.

Market	Concept words	Demand	Defensibility
Water filters	campground, wilderness, hike	0.446	0.922
Protein powder	nutritional, ketogenic, peptide	0.383	1.249
Sports water bottles	insulated, lightweight, waterproof	0.509	0.782
Yoga mats	rubber, foam, handmade	0.763	0.528
Reported mean		0.525	0.870

Appendix

Table 14: Selected market positions and direction factors. These are candidate concepts, not verified product specifications.

Direction	Concept words	Demand	Def.	GS	PAWC	Product
1	campground, wilderness, hike	0.446	0.922	0.410	0.518	0.213
2	portable, filtration, faucet	0.772	0.396	0.306	0.410	0.125
3	filter, purification, filtering	0.579	0.427	0.247	0.471	0.116
4	screened, cleansing, reservoir	0.730	0.185	0.135	0.544	0.073
5	waterproof, tank, aquatic	0.755	0.355	0.268	0.484	0.130

Appendix

Table 15: All five water-filter directions. GS denotes the reported direction GapScore. The highest PAWC and the highest GS

\times

PAWC select different directions.

Generator	$\Delta$ Collision $\downarrow$	$\Delta$ Vendi $\uparrow$
GPT-4o-mini	$-0.019$	$+1.43$
Claude-Sonnet-4.5	$-0.023$	$+3.77$
Gemini-2.0-Flash	$-0.028$	$+8.46$
Qwen2.5-3B	$-0.024$	$-0.74$

Appendix

Table 16: Separate four-generator synthetic-data runs. Differences are GapScore minus Data Accumulation. Negative Collision and positive Vendi differences are favorable. No accuracy-by-model values were supplied.

Method	SetGapScore	Mean direction GapScore (diagnostic)
Random direction	0.023	—
Demand only	0.199	0.199
Defensibility only	$-0.029$	$-0.262$
GapScore	0.276	0.192

Appendix

Table 17: Synthetic-data direction-set diagnostics (

K=3

). A dash indicates a diagnostic not computed for that method. The two columns evaluate different aggregations, not repeated measurements of the same metric.

Diverse Text Generation

Diverse text generation studies how to produce varied, useful language outputs without sacrificing coherence, accuracy, or task performance.

See all 50 papers

Jun 30, 2025

Logit-Gap Steering: A Forward-Pass Diagnostic for Alignment Robustness

RLHF-style alignment trains language models to refuse unsafe requests, but how much operational margin does this refusal rest on? We introduce the refusal-affirmation logit gap: the difference between the top refusal-token logit and the top affirmative-token logit at the first decoding step. This single scalar quantifies the per-prompt safety margin that alignment provides. Empirically, alignment widens the gap on 97.5-99.8% of toxic prompts across three model families, and median gap closure co-varies with True-ASR ranking across suffix strategies (an internal consistency check, since our method optimises gap closure). To validate the metric's practical significance, we present logit-gap steering, a gradient-free, forward-pass-only method that discovers short in-distribution suffixes (

<

10 tokens per component) whose cumulative effect closes the gap. The method requires

{\approx}26{,}000

forward-pass equivalents per family (

{\approx}2

~min on one A100),

{\approx}125\times

less than a single GCG search. Suffixes discovered on 0.5B--2B models transfer without modification to 72B within family. An 8-suffix ensemble reaches 38-96% True ASR across 13 models on AdvBench and HarmBench, with most suffixes having

10^{3}

10^{4}\times

lower perplexity than GCG-meaning published perplexity-filter defenses that collapse GCG (64.7%

\to

1.0%) leave our suffixes nearly intact (76.9%

\to

76.0%). These results demonstrate that current alignment margins, while consistently present, can be thin and efficiently measurable, and that defense strategies must account for in-distribution suffixes.

Tung-Ling Li, Hongliang Liu

Palo Alto Networks

LLM Alignment Language Model Safety Evaluation Adversarial Attacks on LLMs Refusal Behavior in Language Models

Ask PDF

Apr 18, 2026

Alignment Imprint: Zero-Shot AI-Generated Text Detection via Provable Preference Discrepancy

Detecting AI-generated text is an important but challenging problem. Existing likelihood-based detection methods are often sensitive to content complexity and may exhibit unstable performance. In this paper, our key insight is that modern Large Language Models (LLMs) undergo alignment (including fine-tuning and preference tuning), leaving a measurable distributional imprint. We theoretically derive this imprint by abstracting the alignment process as a sequence of constrained optimization steps, showing that the log-likelihood ratio can naturally decompose into implicit instructional biases and preference rewards. We refer to this quantity as the Alignment Imprint. Furthermore, to mitigate the instability in high-entropy regions, we introduce Log-likelihood Alignment Preference Discrepancy (LAPD), a standardized information-weighted statistic based on alignment imprint. We provide statistical guarantee that alignment-based statistics dominate Fast-DetectGPT in performance. We also theoretically show that LAPD strictly improves the unweighted alignment scores when the aligned and base models are close in distribution. Extensive experiments show that LAPD achieves an improvement 45.82% relative to the strongest existing baselines, yielding large and consistent gains across all settings.

Junxi Wu, Kailin Huang, Dongjian Hu +4

Nankai University · Tsinghua University · Wuhan University +3

LLM Alignment AI-Generated Text Detection

Ask PDF

Jun 9, 2026

Where You Inject Diversity Matters: A Unified Framework for Diverse Generation

Open-ended generation tasks often require a set of meaningfully different outputs, yet large language models often produce similar generations. Existing test-time diversity methods operate at different stages of generation with varying effectiveness, but it remains unclear what design choices lead to meaningful diversity in the output. We introduce a framework that characterizes test-time diverse generation methods by the diversity source introduced during generation and provide a transmission score for measuring how effectively variation in the source reaches the final output. Guided by this framework, we propose fully automated specification-level generation methods that first generate diverse intermediate specifications and then condition on them to produce final responses. Across five open-ended tasks and four backbone models, specification-level injection improves output diversity over test-time baselines while maintaining comparable quality. Our analysis shows that successful diversity injection depends on both the diversity of the sources and their transmission to the output, highlighting source design and source-to-output realization as two key levers for building more diverse generation systems.

Cheng Zhang, Rui Xin, Chudi Zhong

UNC Chapel Hill · University of Washington

Open-Ended Generation Controllable Text Generation Diverse Text Generation Output Diversity in Language Models

Ask PDF

Abstract

Diverse Text Generation

Explore similar work

Logit-Gap Steering: A Forward-Pass Diagnostic for Alignment Robustness

Alignment Imprint: Zero-Shot AI-Generated Text Detection via Provable Preference Discrepancy

Where You Inject Diversity Matters: A Unified Framework for Diverse Generation