Do Embedding-Map Distances Guide LLM Ideation? Four Tests Against What Came Next
Organizations: AGILabs (javieraguilar.ai)
Abstract
A popular ideation recipe treats an embedding map of existing work as a guide: pick points far from everything and decode them into ideas. I test four ways such distances could guide ideation against what appeared later. (1) Far random directions are not text: no later item reaches a raw cosine above about 0.10 with one (after centring or whitening, 22 of 24 cells stay out of reach), and an LLM decoder shown a target's neighbours, with score-only feedback, writes those neighbours in all six target sets. A neighbour-free search writes coherent text at median cosine 0.57 and 0.47 to interior gaps, beating the nearest existing item on 15 and 14 of 20, but only 0.07 and 0.06 to far points. (2) On k-means cells of a past-only map, the registered hole score mostly tracks cell size and position; three density-based gap scores forecast which cells grow fastest at or below chance (AUC 0.30 to 0.66) and add nothing to momentum in Steam and arXiv, with patents inconclusive; more isolated new Steam games are slightly better reviewed. (3) LLM continuations of arXiv papers stay closer to the source than papers that cite it, under three user prompts and against proposal rewrites (gpt-4o) and April-December 2023 papers (OLMo). Critique lengthens this step beyond content-free revision, replicated on 150 new papers. Revealing a real follow-up's title turns the ideas by 0.032 along a follow-up-specific direction beyond revision; reviewer, oracle and distance feedback add 0.13, 0.08 and 0.34 of that turn (upper bounds 0.46, 0.46, 0.70), and their turn is not separable from revision or their longer step. (4) For inspiration material graded by distance, no inverted U was detected, on an anticipation margin that tracks format more than content. Original tests were registered before running, follow-ups (thirteen requested in two AI-agent reviews) after the main results; all are dated only by local git commits.
Figures & tables
| Claim (section) | Finding | Verdict | Basis |
|---|---|---|---|
| Far random directions can be decoded into ideas (3) | no later item reaches a raw cosine above about 0.10 with one; 22 of 24 corrected-space cells stay out of reach | not supported | exploratory (raw future check); follow-up (centred and whitened) |
| A decoder guided by distance follows the target (3) | shown a target’s neighbours with score-only feedback, it writes those neighbours in 6 of 6 target sets | not supported for this decoder | original; follow-up (independent target sets) |
| Interior gaps can be written as text (3) | median cosine 0.57 and 0.47; above the nearest existing item on 15 and 14 of 20 targets | supported, by small margins | original (search); exploratory (exact ceiling counts) |
| Sparse cells of the past map grow faster (4) | valid gap scores forecast top-decile growth at or below chance (AUC 0.30 to 0.66); nothing added to momentum in Steam and arXiv; registered criteria also met in two spaces without post-split encoder training (Steam inconclusive in LSA) | not supported; patents inconclusive | original (F2, F3); follow-up (score validity, leak-free spaces) |
| Empty regions fill (4) | new items are slightly more isolated than held-out past ones (AUC 0.61 to 0.64 arXiv, 0.55 to 0.60 Steam, 0.51 patents); preferential filling of the past map’s empty points not shown (Steam and arXiv inconclusive, patents contradicting) | not shown by the registered contrast (a graded association in arXiv); new arXiv items more isolated | original (F6, inconclusive); follow-up (item-level density) |
| Isolated new items are more successful (4) | more isolated new Steam games are slightly better reviewed (ρ +0.11 to +0.13), also net of review count, year, reach and genre; more isolated patent applications are slightly less often granted | slightly, in Steam (registered F5 fails); reversed in patents | original (F5); follow-up (controls) |
| Model | Role | Training cutoff | Sections |
|---|---|---|---|
| gpt-4o-2024-08-06 (OpenAI, 2024 ), temperature 1 | generator | October 2023 | 5 , 6 , A.7 |
| OLMo-7B-0424-Instruct (Groeneveld et al., 2024 ), temperature 1 | generator | February–March 2023 (Dolma; Soldaini et al., 2024 ) | 5 |
| gpt-5.4-mini (2026-03-17), temperature 1 | generator | 31 August 2025 (third-party listings; none published) | 5.4 |
| gpt-6-luna | decoder and generator | after all splits | 3 , A.1 – A.4 |
| Claude Sonnet 5.5 | decoder and generator | after all splits | 3 , A.1 – A.4 |
| gpt-6.1-sol, Claude Opus 5.5, Claude Haiku 5.5 | stress-test generators, coherence and novelty judges, follow-up labellers | after all splits | 6 , 8 , A.1 , A.7 |
| Arm | step | fit | hit |
|---|---|---|---|
| GEO | 0.766 → 0.679 | 0.677 → 0.636 | 0.627 → 0.518 |
| Reviewer | 0.766 → 0.703 | 0.679 → 0.647 | 0.644 → 0.591 |
| ORACLE | 0.761 → 0.729 | 0.677 → 0.666 | 0.664 → 0.611 |
| NEUTRAL (added later) | 0.766 → 0.722 | 0.677 → 0.662 | 0.627 → 0.613 |
| Test | GEO − NEUTRAL | Reviewer − NEUTRAL | ORACLE − NEUTRAL |
|---|---|---|---|
| N1, N2: step lengthened (one-sided) | +0.042, p 2 × 10⁻¹⁶ [+0.013, 2 × 10⁻⁵] | +0.018, p 2 × 10⁻⁴ [+0.014, 6 × 10⁻⁶] | not tested |
| N3: fit | −0.026, p 6 × 10⁻¹¹ [−0.002, 1] | −0.016, p 0.001 [−0.009, 0.001] | +0.0035, p 1 [−0.003, 0.75] |
| N4: hit | −0.096, p 0.001 [−0.033, 0.61] | −0.038, p 0.74 [−0.033, 1] | −0.040, p 0.86 [+0.002, 1] |
| N5: alignment | +0.003, p 1 [+0.012, 0.21] | +0.000, p 1 [−0.003, 1] | −0.016, p 0.12 [−0.014, 0.22] |
| N6: follow-up-specific | −0.003, p 1 [+0.010, 0.48] | −0.013, p 0.14 [−0.008, 0.94] | −0.022, p 0.018 [−0.014, 0.27] |
| N6 as a share of the gap G ( ) | −0.04, CI −0.15 to +0.07 [+0.10, CI +0.00 to +0.19] | −0.13, CI −0.27 to −0.02 [−0.08, CI −0.19 to +0.03] | −0.23, CI −0.39 to −0.09 [−0.13, CI −0.24 to −0.02] |
| Generator / embedder | Arm | alignment r0 → r2 (Holm p ) | contrib. along | contrib. along | change along ( p ; Holm p ) | share of G ( ) |
|---|---|---|---|---|---|---|
| gpt-4o / Qwen3 | GEO | 0.232 → 0.265 (1 × 10⁻⁷) | +0.022 | +0.012 | +0.015 (0.01; 0.11) | +0.16 |
| Reviewer | 0.233 → 0.264 (9 × 10⁻⁹) | +0.026 | +0.004 | +0.006 (0.09; 0.47) | +0.07 | |
| ORACLE | 0.238 → 0.252 (0.06) | +0.017 | −0.002 | −0.003 (0.54; 1) | −0.03 | |
| NEUTRAL ( Section 5.2 ) | 0.232 → 0.262 | — | — | +0.019 (5 × 10⁻⁴; see N6) | +0.20 | |
| gpt-4o / BGE-M3 | GEO | 0.393 → 0.435 (2 × 10⁻⁹) | +0.024 | +0.017 | +0.024 (3 × 10⁻⁴; 0.005) | +0.23 |
| Reviewer | 0.394 → 0.420 (4 × 10⁻⁹) | +0.023 | +0.004 | +0.006 (0.16; 0.62) | +0.06 |
| Test | Qwen3 | Holm p | BGE-M3 | Holm p |
|---|---|---|---|---|
| R1: GEO − NEUTRAL, step | +0.040 [0.032, 0.048] | <10⁻¹⁴ | +0.017 [0.012, 0.022] | <10⁻⁴ |
| R2: Reviewer − NEUTRAL, step | +0.023 [0.016, 0.030] | <10⁻⁶ | +0.014 [0.008, 0.019] | <10⁻⁴ |
| E1: GEO | +0.019 [0.009, 0.029] | 0.003 | +0.020 [0.010, 0.031] | 0.002 |
| E2: Reviewer | +0.012 [0.004, 0.020] | 0.046 | +0.010 [0.001, 0.019] | 0.20 |
| E3: ORACLE | +0.012 [0.003, 0.021] | 0.10 | +0.004 [−0.003, +0.011] | 1 (within ±0.015: 0.022) |
| E4: GEO − NEUTRAL | +0.011 [−0.000, +0.022] | 0.36 | +0.011 [0.002, 0.021] | 0.20 |
| Quantity | Qwen3 | BGE-M3 |
|---|---|---|
| Manipulation check: HINT − NEUTRAL, cosine to the hinted paper | +0.227 [0.206, 0.248] | +0.183 [0.165, 0.200] |
| PC1: HINT − NEUTRAL along | +0.032 [0.018, 0.046], Holm 0.0003 | +0.024 [0.012, 0.036], Holm 0.0003 |
| PC2: HINT − GEO along | +0.021 [0.007, 0.036], Holm 0.018 | +0.014 [0.001, 0.026], Holm 0.074 |
| PC3: HINT − ORACLE along | +0.029 [0.014, 0.045], Holm 0.002 | +0.031 [0.017, 0.044], Holm 0.0001 |
| GEO beyond revision, share of HINT’s turn | 0.34 [0.00, 0.70] | 0.43 [0.04, 0.90] |
| Reviewer beyond revision, share of HINT’s turn | 0.13 [−0.32, 0.46] | −0.02 [−0.68, 0.40] |
| Contrast | GEO | Reviewer | ORACLE |
|---|---|---|---|
| Registered estimand | −0.002 [−0.013, +0.009] | −0.003 [−0.014, +0.008] | +0.010 [−0.004, +0.024] |
| BGE-M3 | −0.000 [−0.010, +0.009] | −0.000 [−0.011, +0.010] | +0.001 [−0.010, +0.013] |
| 133 contexts not in the gpt-4o replication | −0.003 [−0.015, +0.009] | −0.003 [−0.015, +0.010] | +0.006 [−0.008, +0.021] |
| Adjusted for step change | −0.006 [−0.018, +0.006] | −0.004 [−0.016, +0.007] | +0.010 [−0.004, +0.024] |
| Genre direction removed | −0.004 [−0.015, +0.008] | −0.004 [−0.015, +0.008] | +0.009 [−0.006, +0.023] |
| Corrected for the reliability of | −0.004 [−0.021, +0.013] | −0.007 [−0.024, +0.011] | +0.014 [−0.007, +0.035] |
| Generator / arm | Step lengthened | Fit change | Alignment change | Follow-up-specific |
|---|---|---|---|---|
| gpt-4o / ORACLE | 0.032 | −0.012 | +0.014 (0.0052) | −0.003 (0.52) |
| gpt-4o / ORACLE-H | 0.023 | −0.008 | +0.018 (5 × 10⁻⁶) | +0.005 (0.13) |
| gpt-5.4-mini / GEO | 0.061 | −0.021 | +0.025 (6 × 10⁻⁵) | −0.003 (0.37) |
| gpt-5.4-mini / Reviewer | 0.050 | −0.022 | +0.0035 (0.40) | −0.004 (0.59) |
| gpt-5.4-mini / ORACLE | 0.008 | −0.008 | −0.022 (2 × 10⁻⁵) | −0.006 (0.19) |
| gpt-5.4-mini / ORACLE-H | 0.012 | −0.007 | −0.012 (0.03) | +0.001 (0.96) |
| Qwen3 INCIDENTAL | Qwen3 INSTRUCTED | BGE-M3 INCIDENTAL | BGE-M3 INSTRUCTED | |
|---|---|---|---|---|
| P1 slope of margin | +0.0038 (0.034): fails | +0.0053 (0.008): fails | +0.0015 (0.13): holds | −0.0008 (0.72): holds |
| P1, two-sided 95 % interval | [−0.0003, +0.0079] | [+0.0010, +0.0096] | [−0.0011, +0.0042] | [−0.0033, +0.0018] |
| P1 with isolation controlled (exploratory) | +0.0015 [−0.0025, +0.0056] | +0.0027 [−0.0016, +0.0070] | −0.0014 [−0.0040, +0.0012] | −0.0033 [−0.0058, −0.0007] |
| Inverted U (two-lines) | no | no | no | no |
| P2 η² source vs quintile | 0.041 vs 0.004: holds | 0.060 vs 0.007: holds | 0.017 vs 0.007: fails | 0.016 vs 0.008: fails |
| P3 slope of isolation | +0.022 (< 10⁻⁸): holds | +0.024 (< 10⁻⁹): holds | +0.023 (< 10⁻¹⁵): holds | +0.022 (< 10⁻¹⁴): holds |
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Corpus | Past (map) | Future (validation) | Quality signal |
|---|---|---|---|
| Steam games (Steam Games Dataset; Bustos, 2026 ) | release < 2024-01-01; 68,222 after removing adult-tagged rows | 2024 onwards; 40,989 | positive review share, items with ≥ 20 reviews |
| arXiv cs.* (arXiv metadata snapshot; Cornell University, 2026 ), title + abstract | first version < 2023-01-01; 72,000 | 2023-01-01 to 2026-10-15; 41,000 | none |
| US patent applications (HUPD) | filed 2012–2015; 71,521 | filed 2016–2017; 40,342 | granted vs rejected (pending excluded) |
| Steam | arXiv | Patents | |
|---|---|---|---|
| F1: future items with max cos ≤ 0.15 | 0 / 0 | 0 / 0 | 0 / 0 |
| lowest max-cos of a future item to the past | 0.48 / 0.57 | 0.41 / 0.55 | 0.46 / 0.57 |
| median isolation, future − held-out past | +0.012 / +0.015 | +0.029 / +0.018 | +0.004 / +0.002 |
| F2: partial ρ(growth, hole | momentum, size) | +0.20 / +0.33 | +0.17 / +0.34 | +0.07 / +0.07 |
| F3: AUC, momentum alone, scored directly (exploratory) | 0.92 / 0.85 | 0.92 / 0.86 | 0.73 / 0.76 |
| F3: AUC, hole alone, scored directly (exploratory) | 0.45 / 0.54 | 0.49 / 0.53 | 0.50 / 0.48 |
| Score | Planted-hole AUC (cells passing) | ρ with mean density | ρ with centroid cosine to corpus mean | Growth AUC | Gain over momentum |
|---|---|---|---|---|---|
| H0, registered Gaussian hole | 0.45 to 0.59 (0 of 6) | −0.49 to −0.32 | +0.74 to +0.80 | 0.34 to 0.57 | −0.098 to +0.058 |
| H1, centroid kNN density | 0.75 to 0.99 (5 of 6) | −0.95 to −0.89 | +0.06 to +0.46 | 0.33 to 0.62 | −0.037 to +0.049 |
| H2, mean item sparseness | 0.74 to 0.99 (5 of 6) | −1.00 | +0.11 to +0.54 | 0.32 to 0.65 | −0.035 to +0.040 |
| H3, uncovered probe share | 0.83 to 0.93 (6 of 6) | −0.96 to −0.94 | +0.21 to +0.51 | 0.30 to 0.66 | −0.040 to +0.055 |
| Domain | Decoder | Arm | D [interval] | A [interval] | true / swap / none |
|---|---|---|---|---|---|
| Steam | gpt-6-luna | REAL | +0.265 [+0.161, +0.368] | 0.55 [0.42, 0.68] | 0.691 / 0.426 / 0.383 |
| Steam | Sonnet 5.5 | REAL | +0.311 [+0.222, +0.399] | 0.65 [0.50, 0.79] | 0.765 / 0.454 / 0.465 |
| Steam | gpt-6-luna | GAP | +0.142 [+0.085, +0.198] | 0.35 [0.28, 0.42] | 0.470 / 0.328 / 0.329 |
| Steam | Sonnet 5.5 | GAP | +0.133 [+0.084, +0.182] | 0.41 [0.32, 0.49] | 0.526 / 0.393 / 0.373 |
| Steam | gpt-6-luna | FAR | +0.038 [+0.019, +0.057] | 0.30 [0.21, 0.39] | 0.019 / −0.019 / −0.007 |
| Steam | Sonnet 5.5 | FAR | +0.057 [+0.041, +0.072] | 0.41 [0.30, 0.51] | 0.035 / −0.022 / −0.019 |
| Domain | Arm | ceiling | q99 | targets ≥ q99 | targets > ceiling | |
|---|---|---|---|---|---|---|
| Steam | REAL | 0.729 | 0.589 | 0.842 | 10/10 | 10/10 |
| Steam | GAP | 0.533 | 0.457 | 0.567 | 19/20 | 15/20 |
| Steam | FAR | 0.042 | −0.002 | 0.069 | 20/20 | 16/20 |
| arXiv | REAL | 0.739 | 0.527 | 0.874 | 10/10 | 10/10 |
| arXiv | GAP | 0.428 | 0.346 | 0.472 | 19/20 | 14/20 |
| arXiv | FAR | 0.055 | 0.010 | 0.057 | 18/20 | 11/20 |
| Score | Enrichment, all | AUC, all | Novel, full ranking (a) | Novel, within novel (b) | AUC within novel |
|---|---|---|---|---|---|
| POP (static popularity) | 28.2 | 0.961 | 9.6 | 23.5 | 0.953 |
| M (momentum, unfitted) | 29.1 | 0.969 | 0.0 | 30.9 | 0.962 |
| COH (PMI coherence) = NC 0 | 0.0 | 0.164 | 0.0 | 0.0 | 0.140 |
| NC 0.4 (coherence + rarity) | 0.0 | 0.086 | 0.0 | 0.0 | 0.140 |
| HOLE, Qwen3 / BGE-M3 | 1.43 / 2.30 | 0.622 / 0.693 | 1.87 / 2.36 | 1.87 / 2.39 | 0.618 / 0.696 |
| FIT_M (fitted momentum) | 34.2 | 0.970 | 8.3 | 31.0 | 0.961 |
| Arm | step gap | fit | hit | diversity |
|---|---|---|---|---|
| CTX | 0.149 | 0.680 | 0.659 | 0.824 |
| CTX+NB | 0.140 | 0.686 | 0.675 | 0.808 |
| CTX+MEM | 0.128 | 0.668 | 0.611 | 0.767 |
| CTX+MOM | 0.138 | 0.685 | 0.617 | 0.803 |
| Generator / embedder | Step gap (all / S2) | Step lengthened | Fit change | Alignment change | Follow-up-specific |
|---|---|---|---|---|---|
| gpt-4o / Qwen3 | 0.149 / 0.129 | 0.029 to 0.087 | −0.041 to −0.012 | +0.014 to +0.033 | −0.003 to +0.015 |
| gpt-4o / BGE-M3 | 0.168 / 0.159 | 0.028 to 0.048 | −0.017 to −0.007 | +0.015 to +0.041 | +0.001 to +0.024 |
| OLMo / Qwen3 | 0.218 / 0.197 | 0.032 to 0.092 | −0.021 to −0.006 | +0.038 to +0.077 | +0.024 to +0.032 |
| OLMo / BGE-M3 | 0.188 / 0.179 | 0.029 to 0.061 | −0.006 to +0.003 | +0.031 to +0.075 | +0.012 to +0.028 |
| Sample, embedder | G [95 % CI]; ±0.015 / G | Arm | (own change / G) | (beyond NEUTRAL / G) |
|---|---|---|---|---|
| Discovery, gpt-4o, Qwen3 | 0.094 [0.073, 0.116]; 0.16 | GEO | +0.16 [+0.06, +0.28] | −0.04 [−0.15, +0.07] |
| Discovery, gpt-4o, Qwen3 | Reviewer | +0.07 [−0.02, +0.16] | −0.13 [−0.27, −0.02] | |
| Discovery, gpt-4o, Qwen3 | ORACLE | −0.03 [−0.14, +0.07] | −0.23 [−0.39, −0.09] | |
| Discovery, gpt-4o, Qwen3 | NEUTRAL | +0.20 [+0.10, +0.32] | — | |
| Discovery, gpt-4o, Qwen3 | ORACLE-H | +0.07 [−0.02, +0.18] | +0.10 [−0.03, +0.25] (minus ORACLE) | |
| Discovery, OLMo, Qwen3 | 0.128 [0.105, 0.150]; 0.12 | GEO | +0.19 [+0.04, +0.34] | +0.10 [−0.05, +0.25] |
| Embedder, k | n | L | 0.015 / L | GEO − NEUTRAL | Reviewer − NEUTRAL | ORACLE − NEUTRAL |
|---|---|---|---|---|---|---|
| Qwen3, 2 | 83 | 0.228 | 0.066 | −0.005 [−0.020, +0.009] | −0.013 [−0.030, +0.004] | +0.008 [−0.012, +0.028] |
| Qwen3, 3–4 | 44 | 0.290 | 0.052 | −0.007 [−0.028, +0.014] | +0.000 [−0.014, +0.014] | +0.005 [−0.020, +0.031] |
| Qwen3, 5 or more | 23 | 0.362 | 0.041 | +0.020 [−0.015, +0.055] | +0.027 [−0.010, +0.064] | +0.025 [−0.008, +0.058] |
| Qwen3, all | 150 | 0.267 | 0.056 | −0.002 [−0.013, +0.009] | −0.003 [−0.015, +0.009] | +0.010 [−0.004, +0.024] |
| BGE-M3, 2 | 83 | 0.246 | 0.061 | −0.006 [−0.018, +0.006] | −0.004 [−0.016, +0.009] | +0.004 [−0.010, +0.017] |
| BGE-M3, 3–4 | 44 | 0.295 | 0.051 | −0.001 [−0.023, +0.020] | +0.007 [−0.015, +0.029] | +0.004 [−0.023, +0.032] |
| Embedder, k | GEO | Reviewer | ORACLE | NEUTRAL |
|---|---|---|---|---|
| Qwen3, 2 | +0.013 [−0.003, +0.030] | +0.006 [−0.008, +0.020] | +0.027 [+0.010, +0.043] | +0.019 [+0.004, +0.034] |
| Qwen3, 3–4 | −0.017 [−0.037, +0.003] | −0.010 [−0.029, +0.009] | −0.005 [−0.026, +0.016] | −0.010 [−0.026, +0.006] |
| Qwen3, 5 or more | +0.011 [−0.029, +0.051] | +0.018 [−0.016, +0.052] | +0.016 [−0.007, +0.039] | −0.009 [−0.031, +0.013] |
| Qwen3, all (Holm over four) | +0.004 [−0.008, +0.016], Holm 1 | +0.003 [−0.008, +0.014], Holm 1 | +0.016 [+0.004, +0.027], Holm 0.076 | +0.006 [−0.004, +0.016], Holm 1 |
| BGE-M3, 2 | +0.007 [−0.007, +0.022] | +0.010 [−0.003, +0.022] | +0.017 [+0.005, +0.029] | +0.013 [+0.001, +0.025] |
| BGE-M3, 3–4 | −0.013 [−0.032, +0.006] | −0.005 [−0.022, +0.013] | −0.007 [−0.026, +0.012] | −0.011 [−0.033, +0.010] |
| Document | Commit, date | Tests | Smoke calls before the test run | Section |
|---|---|---|---|---|
| arXiv FAR ceiling in design.md | 2aab2ad , 2026-10-24 | parameter-free corpus ceiling of far targets | none | 3 , A.3 |
| memory_probe.md | f4e61f6 , 2026-10-25 | anticipation by generators with known cutoffs | none | A.7 |
| coherent_search.md | 15eeead , 2026-10-26 (rev. 2 the same day) | neighbour-free LLM search with score feedback | rev. 1 and rev. 2 smokes on target k = 0, excluded | 3 , A.4 |
| patents block of design.md | 05c997a , 2026-10-26 | the gaps study on patents, including the FAR ceiling | none | 3 , A.3 |
| fluent_search.md | edfcb5c , 2026-10-27; rev. 2 c643773 , 2026-10-28 | gradient token search with a fluency penalty | 9 + 4 Kaggle runs on target k = 0, excluded | A.4 |
| fluent_search.md , rev. 3 | 3235967 , 2026-10-28 | seed-and-rewrite, tuned on REAL k = 11–20 | tuning targets disjoint from the test set | 3 , A.4 |
| Result | Section | BGE-M3 |
|---|---|---|
| Causal anchoring, 6 independent target sets | 3 | replicated: donor attraction in all 12 decoder cells, also in centred and whitened spaces; target loss in the 4 REAL cells (not defined for GAP and FAR targets) |
| F1, nothing in the far region | 4 | replicated |
| F2 fails: partial association with growth | 4 | replicated (Steam, arXiv); patents not significant in either; with valid gap scores in 12 of 15 configurations |
| F3, hole adds nothing to momentum | 4 | replicated in Steam and arXiv; patents/BGE-M3 +0.031, inconclusive |
| F4, latitude shift of the future | 4 | not replicated |
| F5, isolation and quality | 4 | replicated (fails in Steam, holds in patents) |
| Source | Qwen3 range | Ratio | BGE-M3 range | Ratio |
|---|---|---|---|---|
| Steam games | 0.129 to 0.372 | 0.60 | 0.097 to 0.276 | 0.51 |
| Board games | 0.260 to 0.484 | 0.55 | 0.235 to 0.381 | 0.42 |
| Film plots | 0.441 to 0.583 | 0.35 | 0.352 to 0.460 | 0.31 |
| Sports rules | 0.282 to 0.482 | 0.49 | 0.250 to 0.430 | 0.52 |
| arXiv papers | 0.464 to 0.667 | 0.50 | 0.370 to 0.495 | 0.36 |
| Patents | 0.469 to 0.660 | 0.47 | 0.379 to 0.502 | 0.35 |