GEO-Flag: Detecting and Measuring GEO-Optimized Web Content
Organizations: CISPA Helmholtz Center for Information Security · HPE · University of Waterloo
Abstract
Generative Engine Optimization (GEO) modifies web content to increase its likelihood of being selected and cited by generative search engines. This can give strategically optimized pages visibility disproportionate to their authority or relevance and even make weak or false information appear well supported. Unlike conventional search, generative search synthesizes information into direct answers rather than presenting competing sources, which can further amplify these risks, as assessing source provenance and authority requires additional user interaction. Despite these concerns, systematic methods for detecting GEO-optimized webpages remain underexplored. We introduce GEOFlagBench, a benchmark of 3,200 web content instances spanning 400 queries, four domains, and eight GEO optimizer families, and use it to systematically evaluate existing GEO detection methods. Although the strongest baseline achieves an aggregate F1 of 0.880, method-level and authorship-conditioned evaluations reveal substantial weaknesses and potential reliance on authorship-related shortcuts. We therefore propose Intervention-Paired Training (IPT), which supervises detector responses to GEO interventions and non-GEO AI polishing; on ModernBERT, IPT improves F1 from 0.862 to 0.944 and worst-group accuracy from 0.725 to 0.883. We develop a GEO-gated Agent system for auditing the Source Tier and verifiability of Citation URLs in detected GEO pages. Finally, we deploy the complete pipeline on released Google Search and Gemini-grounded retrieval results for 1,000 real-user queries. Across 10,095 available pages, we estimate an overall GEO prevalence of 8.90%, reaching 16.36% among pages modified in 2026. Our results establish a foundation for systematically detecting, auditing, and measuring GEO in real-world search ecosystems.
Figures & tables
| Method | Acc. | F1 |
|---|---|---|
| Fine-tuned Models | ||
| ModernBERT (1K) | 0.842 | 0.864 |
| ModernBERT (8K) | 0.839 | 0.862 |
| Qwen3-0.6B (SFT) | 0.778 | 0.833 |
| Feature engineering + classical classifiers | ||
| TF-IDF char (2-4) + LR | 0.848 | 0.878 |
| Method | Method-Level GEO Recall | Authorship-Conditioned Diagnostics | ||||
|---|---|---|---|---|---|---|
| PMA | AutoGEO-Light | Human | Worst-Group Acc. | FPR | TPR | |
| Fine-tuned models | ||||||
| ModernBERT-base (1K) | 0.566 | 0.423 | 0.426 | 0.725 | 0.271 | 0.121 |
| ModernBERT-base (8K) | 0.513 | 0.346 | 0.444 | 0.725 | 0.263 | 0.117 |
| Qwen3-0.6B (SFT) | 0.934 | 0.654 | 0.463 | 0.325 | 0.413 | 0.105 |
| Feature engineering + classical classifiers | ||||||
| Overall Performance | Shortcut Diagnostics | ||||
| Method | Acc. | F1 | WGA | FPR | TPR |
| Selected baselines | |||||
| ModernBERT (1K) | 0.842 | 0.864 | 0.725 | 0.271 | 0.121 |
| ModernBERT (8K) | 0.839 | 0.862 | 0.725 | 0.263 | 0.117 |
| Qwen Zero-shot | 0.627 | 0.770 | 0.008 | 0.000 | 0.006 |
| Qwen SFT | 0.778 | 0.833 | 0.325 | 0.413 | 0.105 |
| Method | Acc. | Macro F1 |
|---|---|---|
| TF-IDF Character 2–4 with LR | 0.730 | 0.612 |
| TF-IDF Word 1–2 with LR | 0.710 | 0.562 |
| Ghostbuster features, recalibrated | 0.545 | 0.286 |
| Structural features with XGBoost | 0.648 | 0.573 |
| GPT-2 perplexity features with LR | 0.497 | 0.226 |
| Gemini-3.5-flash | 0.500 | 0.315 |
| Agent Model | URL Source Tier | Citation URL Verifiability |
|---|---|---|
| GLM 5.2 | 84.75 / 84.02 | 83.00 / 83.20 |
| Kimi K3 | 78.46 / 76.43 | 76.65 / 76.78 |
| Qwen 3.6 35B-A3B | 77.37 / 75.69 | 76.10 / 76.54 |
| GPT 5.6 Luna | 76.77 / 75.03 | 75.32 / 75.50 |
| Claude Haiku 4.5 | 73.93 / 71.39 | 72.60 / 72.50 |
| Gemini 3.5 Flash | 62.07 / 58.10 | 62.31 / 63.05 |
| Setting | Google Search | Gemini | Unique Union | Gemini Google | 95% CI |
|---|---|---|---|---|---|
| Page-level estimate | 441/5,421 (8.14%) | 599/6,590 (9.09%) | 898/10,095 (8.90%) | 0.95 pp | N/A |
| Paired-query macro ( ) | 7.91% | 9.34% | N/A | 1.43 pp | [0.46, 2.41] pp |
| Domain | Pages | GEO | GEO Rate |
|---|---|---|---|
| en.wikipedia.org | 613 | 0 | 0.00% |
| youtube.com | 434 | 19 | 4.38% |
| my.clevelandclinic.org | 106 | 0 | 0.00% |
| webmd.com | 78 | 0 | 0.00% |
| healthline.com | 73 | 0 | 0.00% |
| medicalnewstoday.com | 61 | 2 | 3.28% |
| Metric | Google Search | Gemini |
|---|---|---|
| Page and Citation Volume | ||
| Detected GEO pages | 441 | 599 |
| GEO pages with citations | 236 | 335 |
| GEO pages with citations (% of GEO pages) | 53.51% | 55.93% |
| Citation occurrences | 2,162 | 5,272 |
| Citations per detected GEO page | 4.90 | 8.80 |
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Family | Instances | Seeds | Optimization Procedure | Variants and Construction Details |
|---|---|---|---|---|
| GEO Strategy Pool [ 1 , 36 , 5 ] | 400 | 200 human, 200 AI | Samples 3–13 strategies and applies them jointly in one cohesive rewrite. The pool contains 13 strategies covering fluency, lexical diversity, authoritative style, quotations, citations, simpler language, technical terms, search keywords, statistics, content improvement, an llms.txt -style summary, reader questions, and diverse perspectives. | Executor(s): Claude Haiku 4.5; Kimi K2.5; Gemini 3 Flash Preview; GPT-4o. Each processes 50 human and 50 AI seeds. Combines nine GEO strategies with two strategies from C-SEO and two adapted from E-GEO. The E-GEO prompts are adapted from product descriptions to general web articles. |
| AutoGEO [ 47 ] | 400 | 200 human, 200 AI | Applies learned AutoGEO preference rules at three intervention intensities. For each seed type, 50 instances use Light, 100 use Medium, and 50 use Full. | Executor(s): Gemini 3.1 Pro Preview for all instances. Light applies 3 rules, requests limited additions, and requires at least 80% word overlap. Medium applies 6 rules and targets 50–70% overlap. Full preserves the AutoGEO API prompt and applies all 12 learned rules. AutoGEO Mini is excluded. |
| PMA [ 31 ] | 200 | 100 human, 100 AI | Implements three benchmark variants of the Preference Manipulation Attack. Inject places source-prioritization instructions before and after the unchanged seed. Persuade integrates authority claims, social proof, and superlative language into the instance. Combo applies Persuade followed by Inject. | Executor(s): GPT-4o for Persuade and Combo on human seeds; Claude Sonnet 4.6 for Persuade and Combo on AI seeds. Inject requires no LLM. For each seed type, we construct 25 Inject, 25 Persuade, and 50 Combo instances. |
| RAID G-SEO [ 10 ] | 200 | 100 human, 100 AI | Uses our prompt-level implementation of the role-augmented intent pipeline. Four LLM calls extract claims and evidence, infer queries for Student, Professional, Researcher, and Decision-maker roles, construct an intent-aligned content plan, and rewrite the instance. | Executor(s): GLM-5.1 for 199 instances; GLM-5 through Together AI for one AI-seed instance as a fallback. The fallback is used only for one AI-seed output. |
| Meta-Optimization [ 5 ] | 200 | 100 human, 100 AI | Samples 4–6 strategies from the 13-strategy pool, generates an initial rewrite, asks the same model to evaluate it, and revises the instance based on that evaluation. A third rewrite is requested when the intermediate optimization rating is below 7/10. | Executor(s): GPT-4o for 100 human seeds; Claude Sonnet 4.6 for 100 AI seeds. All retained human-seed outputs complete two rewrite rounds. For AI seeds, 4 outputs retain one round, 79 retain two rounds, and 17 retain three rounds. |
| Stealthy GEO Strategy Pool | 200 | 100 human, 100 AI | Uses the same strategy-sampling procedure as GEO Strategy Pool, with an additional instruction to avoid formulaic structure, excessive optimization markers, and unnatural authority signals. | Executor(s): Claude Haiku 4.5; Kimi K2.5; Gemini 3 Flash Preview; GPT-4o. Each processes 25 human and 25 AI seeds. The final revision replaces the original Claude Sonnet 4.6 executor with GPT-4o. |
| Provenance | Non-GEO | GEO | Total |
|---|---|---|---|
| Human | 400 | 1,050 | 1,450 |
| Human + AI Polished | 400 | 0 | 400 |
| AI | 400 | 950 | 1,350 |
| Total | 1,200 | 2,000 | 3,200 |
| Optimizer Family | Human Seed | AI Seed | Median Word-Overlap |
|---|---|---|---|
| GEO Strategy Pool | 200 | 200 | 0.56 |
| AutoGEO | 200 | 200 | 0.71 |
| PMA | 100 | 100 | 0.99 |
| RAID G-SEO | 100 | 100 | 0.58 |
| Meta-Optimization | 100 | 100 | 0.55 |
| Stealthy GEO-SP | 100 | 100 | 0.55 |
| F1 | Accuracy | FPR | Worst-Group Acc. | |
|---|---|---|---|---|
| 0.5 | 0.931 | 0.918 | 0.025 | 0.857 |
| 1 | 0.944 | 0.931 | 0.108 | 0.883 |
| 2 | 0.942 | 0.929 | 0.146 | 0.850 |
| 4 | 0.941 | 0.928 | 0.150 | 0.842 |
| Detector | AutoGEO | GEO Strategy Pool | ||||
|---|---|---|---|---|---|---|
| Full ( ) | Light ( ) | Medium ( ) | Medium-Stealthy ( ) | Standard ( ) | Stealthy ( ) | |
| Fine-tuned Models | ||||||
| ModernBERT (1K) | 1.000 | 0.423 | 0.919 | 0.833 | 0.890 | 0.865 |
| ModernBERT (8K) | 1.000 | 0.346 | 0.919 | 0.819 | 0.941 | 0.846 |
| Qwen3-0.6B (SFT) | 1.000 | 0.654 | 0.887 | 0.847 | 0.966 | 0.923 |
| Feature engineering + classical classifiers | ||||||
| Method | Non-GEO | AutoGEO | Pool | PMA | RAID | Meta | Human |
|---|---|---|---|---|---|---|---|
| TF-IDF Character 2–4 with LR | 0.944 | 0.691 | 0.876 | 0.421 | 0.242 | 0.542 | 0.167 |
| TF-IDF Word 1–2 with LR | 0.975 | 0.511 | 0.900 | 0.645 | 0.182 | 0.438 | 0.019 |
| Ghostbuster features, recalibrated | 0.942 | 0.229 | 0.729 | 0.000 | 0.242 | 0.000 | 0.037 |
| Structural features with XGBoost | 0.769 | 0.739 | 0.582 | 0.066 | 0.636 | 0.667 | 0.537 |
| GPT-2 perplexity features with LR | 0.858 | 0.271 | 0.688 | 0.000 | 0.015 | 0.000 | 0.000 |
| Gemini-3.5-flash | 0.825 | 0.479 | 0.153 | 0.789 | 0.030 | 0.021 | 0.093 |
| Method | Non-GEO | AutoGEO | Pool | PMA | RAID | Meta | Human |
|---|---|---|---|---|---|---|---|
| TF-IDF Character 2–4 with LR | 0.679 | 0.681 | 0.797 | 1.000 | 1.000 | 1.000 | 1.000 |
| TF-IDF Word 1–2 with LR | 0.654 | 0.632 | 0.805 | 1.000 | 1.000 | 1.000 | 1.000 |
| Ghostbuster features, recalibrated | 0.575 | 0.434 | 0.539 | 0.000 | 0.390 | 0.000 | 1.000 |
| Structural features with XGBoost | 0.647 | 0.702 | 0.688 | 0.227 | 0.792 | 0.711 | 0.403 |
| GPT-2 perplexity features with LR | 0.526 | 0.370 | 0.515 | 0.000 | 0.333 | 0.000 | 0.000 |
| Gemini-3.5-flash | 0.692 | 0.287 | 0.310 | 0.556 | 0.400 | 0.125 | 0.357 |
| Tier | Publisher Class | Assignment Rule | Policy Indicators |
|---|---|---|---|
| C1 | Primary or strongly accountable authority | The publisher is the primary institutional source, a peer-reviewed research venue, or an organization with strong expert review, corrections, and editorial accountability. | Primary authority; government or intergovernmental body; peer-reviewed research; strong editorial accountability. |
| C2 | Accountable secondary or curated source | The publisher provides identifiable secondary editorial or community-curated material with meaningful review signals, but is not the primary authority for the claim. | Professional secondary editorial process; named authors and disclosures; aggregation of primary sources; community-curated reference. |
| C3 | Limited-accountability or user-contributed source | Publication is substantially open-contributor, user-generated, advocacy-driven, promotional, anonymous, or lacks stable editorial accountability. This tier does not imply that a specific claim is false. | Open-contributor platform; user-generated content; limited editorial accountability; advocacy or promotion; anonymous or unverifiable authorship. |
| Accessibility Level | Definition | Retrieval Outcomes |
|---|---|---|
| Directly accessible | The citation URL resolves successfully, and its destination content can be directly retrieved and inspected. | LIVE |
| Restricted or archived | The cited source can be established, but its content cannot be directly retrieved because access is restricted or only an archived copy is available. | BLOCKED , ARCHIVED_ONLY |
| Unavailable or unresolved | The cited content cannot be retrieved, or its availability cannot be established at retrieval time. | HTTP_404 , DOMAIN_NXDOMAIN , TIMEOUT |
| Channel | Queries | URLs | Pages | Coverage |
|---|---|---|---|---|
| Google Search | 1,000 | 7,856 | 5,421 | 69.00% |
| Gemini-Grounded | 979 | 8,591 | 6,590 | 76.71% |
| Unique Union | 1,000 | 13,985 | 10,095 | 72.18% |
| Analysis Unit | Google Search | Gemini | Unique Union |
|---|---|---|---|
| Citation occurrences | 2,162 | 5,272 | 6,663 |
| Distinct Citation URLs | 1,443 | 2,647 | 3,551 |
| Agent Model | Time | Estimated Cost |
|---|---|---|
| Gemini 3.5 Flash | 21.47 min | $5.81 |
| Claude Haiku 4.5 | 25.91 min | $3.79 |
| GPT 5.6 Luna | 28.78 min | $3.17 |
| GLM 5.2 | 29.92 min | $3.72 |
| Kimi K3 | 101.76 min | $12.36 |
| Qwen 3.6 35B-A3B | 101.86 min | $2.28 |
| Unique Union | Google Search | Gemini | |||||
| Period | Pages | GEO | Rate | Pages | Rate | Pages | Rate |
| 2023 | 165 | 4 | 2.42% | 65 | 3.08% | 124 | 2.42% |
| 2024 | 299 | 21 | 7.02% | 107 | 7.48% | 243 | 7.41% |
| 2025 | 578 | 74 | 12.80% | 247 | 10.93% | 422 | 13.51% |
| 2026 | 642 | 105 | 16.36% | 318 | 13.52% | 434 | 18.20% |
| Missing | 8,119 | 682 | 8.40% | 4,563 | 7.78% | 5,153 | 8.42% |
| Final Outcome | URLs | Share |
|---|---|---|
| Persistent HTTP 403 | 2,082 | 53.52% |
| Robots policy unavailable or denied | 811 | 20.85% |
| Navigation or network failure | 226 | 5.81% |
| Insufficient or nonmeaningful content | 225 | 5.78% |
| Missing or removed page | 203 | 5.22% |
| Authentication or payment required | 163 | 4.19% |
| Platform | Original Webpage Content | Strong GEO Signals |
|---|---|---|
| YouTube | Title: Think CVTs Don’t Last? Here Are 15 Reliable Cars That Do Description: “15 Cars With CVTs That Actually Last | The Most Reliable CVTs Ever Made” “Many drivers believe every continuously variable transmission is a disaster waiting to happen—but that’s not true. In this video, we highlight 15 cars with CVTs that actually last, backed by real-world reliability. From Toyota and Honda to Subaru and even some Nissan models, these vehicles prove not all CVTs are junk.” [...] “Are CVTs really that bad? The truth is, while some fail early with slipping belts, high repair costs, and short lifespans, others are shockingly durable—lasting well beyond 150,000–200,000 miles.” “We’ll break down: The most reliable CVT cars you can buy today; Which model years to look for (and which to avoid); How CVTs compare to traditional automatics; Simple maintenance tips (like CVT fluid changes) that make a big difference.” [...] Chapter excerpts: “Honda CR-V: 200K Miles and Still Smooth”; “Toyota Camry Hybrid: e-CVT Bulletproof Design”; “Toyota RAV4 Hybrid: The 250K-Mile Workhorse”; “Toyota Prius: Taxi Fleet Proven Reliability.” | The page covers multiple closely related informational intents, including reliable CVT models, model-year selection, transmission lifespan, maintenance, and comparisons with traditional automatics. It also contains dense entity–attribute associations : individual car models are paired with explicit reliability or longevity claims, such as “200K Miles” and “250K-Mile Workhorse.” These concise claims are easy for generative engines to extract and reuse. Moreover, the title, description, and chapter headings repeatedly provide direct, answer-like statements for likely user queries, including “most reliable CVTs,” “cars you can buy today,” and “which model years to look for.” Overall, the page combines broad query coverage with explicit and easily extractable claims, exhibiting strong GEO-style signals. |
| Amazon | Title: Total by Verizon 60 No-Contract Single-Device Monthly Plan Unlimited Talk, Text & Data+20 GB Hotspot About this item: “Get unlimited talk, text & data (1) plus 20 GB of hotspot data (3) and 10 International Calling Credit (7) and enjoy coverage on Verizon 5G Ultra-Wideband (2) with the Total by Verizon Unlimited+ Plan.” “Stay in touch with your people abroad with international talk & text to 5 countries of your choice (4), and call home on your travels with international roaming in Canada & Mexico (6). Screen and automatically block incoming spam calls with Total Spam Filter.” “Continue, upgrade, or stop service easily at any time with a monthly single device plan. Disney Premium+ (No Ads) on us (5).” [...] Product Description: “Go beyond the norm and get a premium wireless experience for less with Total by Verizon’s Unlimited+ Plan: Unlimited talk, text & data (1) monthly plan. Save on the data you crave while exploring without limits on Verizon 5G Ultra Wideband (2). Seize every day with great perks including 20GB of hotspot data (3), international talk & text to five countries of your choice (4), and Disney Premium+ (No Ads) on us (5).” [...] “(3) Use up to 20GB of non-sharable data per line for hotspot usage.” “(6) Unlimited talk and text to the US, Mexico and Canada and unlimited data when traveling in Mexico and Canada.” | The page presents a large number of explicit product–attribute associations , including price, unlimited talk/text/data, 20 GB hotspot data, 5G access, international calling, roaming, spam filtering, and bundled Disney Premium+ access. These attributes are expressed in short, self-contained statements and are repeated across the title, “About this item,” and product description. This makes the page highly suitable for direct extraction when answering specific product questions. The numbered explanations further provide fine-grained factual details , such as hotspot limits, geographic coverage, and roaming conditions. These statements can directly support queries such as how much hotspot data the plan includes or whether it works in Canada and Mexico. Overall, the page has high information density, structured attribute coverage, and many concise answer-ready statements, which together constitute strong GEO-style signals. |
Generative Engine Optimization
Generative Engine Optimization studies how to make web content visible, cited, and accurately represented in AI-generated search answers.
See all 40 papers