cs.AISep 28, 2026

What Drives Citations in Production Large Language Models? An Observational Multi-Method Study of Two Million AI Citations Across Ten Thousand Web Pages

Authors: Ben Moore, Liam Dunne

Organizations: Discovered Labs

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Production large language models retrieve and cite web pages alongside generated answers, yet the page-level features that predict citation frequency remain poorly characterised. We present an observational study of approximately 2 million LLM citations from four commercial engines (ChatGPT, Claude, Google AI, Gemini) over six months, joined to 10,000 crawled pages from nineteen B2B SaaS workspaces. Sixty-plus features are tested using a nine-method consensus framework combining mixed-effects regression with domain fixed effects, FDR correction, stability-selection Lasso, double machine learning, generalised additive models, and temporal hold-out replication. Four findings survive all checks. First, prompt-content alignment (Jaccard overlap between page tokens and the full workspace prompt corpus, including non-citing prompts) is the dominant page-level predictor (beta = +0.37, 95% CI [+0.33, +0.41], q ~ 10^-73). Second, the standard AEO checklist (FAQ blocks, structured data, Core Web Vitals) shows positive effects in pooled data that reverse or collapse to zero once domain fixed effects are applied: Simpson's paradox with practical consequences for the AEO literature. Third, domain-level AI authority exceeds the strongest non-alignment page-level feature by a factor of six in mean absolute SHAP value. We release the analytic pipeline as a methodological contribution.

Figures & tables

Figure 1 : Predictors of citation count y=log⁡2(1+nc)y=\log_{2}(1+n_{c}) from the mixed-effects model in Equation 2 . Points are standardised coefficients; bars are 95% confidence intervals. Only predictors with positive point estimates are shown; the full coefficient table including negatively-signed and non-significant predictors appears in Appendix A . N=4,015N=4{,}015 pages across D=297D=297 domains.
Figure 2 : GAM smooths fj(xj)f_{j}(x_{j}) for the top-five predictors after domain residualisation. The leftmost panel (alignment) is monotone across the data range; the panels for title-prompt similarity, page length, page age, and intro similarity show smaller but still mostly monotone effects.
Figure 3 : Best-matching-paragraph depth distribution. Median =0.36=0.36 ; engines cite from the top third of the page.
Figure 4 : Cumulative share of citations by content age, by engine. Claude reaches 60% within six months; ChatGPT requires twelve. N=101,063N=101{,}063 observations with usable publication dates.
Figure 5 : Brand-controlled versus third-party URL share by engine. ChatGPT cites brand-controlled URLs at ≈3×{\approx}3\times the rate of Gemini.
Figure 6 : Range of median citation age across engines by page type. Pricing pages show the largest engine-to-engine spread ( ≈11{\approx}11 months); comparison content the smallest.
Figure 7 : Rank-biserial effect of binary on-page signals on log-citation count, by funnel stage. Effect sizes grow with funnel depth, suggesting that structural and authority signals matter more on transactional than informational pages.
Figure 8 : Top-20 features by mean absolute SHAP value from the gradient-boosted regressor on y=log⁡2(1+nc)y=\log_{2}(1+n_{c}) . Domain authority (target-encoded mean log-citation) and prompt-content alignment are the two largest contributors; the next strongest non-alignment page-level feature has mean ∣SHAP∣|\text{SHAP}| approximately 6×6\times smaller than domain authority.
Figure 9 : Top-10 third-party domains by citation share, overall and by funnel stage ( 1.27×1061.27\times 10^{6} true third-party citations). No single source exceeds single-digit share at any stage.
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Predictorβ^\hat{\beta}95% CIpp -valueqq (FDR-BH)
Prompt-content alignment+0.369[+0.330,+0.409][+0.330,+0.409]4.87e-751.12e-73
Outbound link count-0.180[−0.239,−0.121][-0.239,-0.121]2.37e-091.82e-08
Page type: pricing+0.423[+0.272,+0.574][+0.272,+0.574]3.92e-082.25e-07
Best paragraph similarity+0.129[+0.074,+0.184][+0.074,+0.184]4.55e-062.09e-05
Title-prompt similarity+0.114[+0.064,+0.164][+0.064,+0.164]7.78e-062.98e-05
Page age+0.056[+0.026,+0.085][+0.026,+0.085]2.03e-046.66e-04
Appendix
Table 1 : Full coefficient table from the mixed-effects model in Equation 2 ( N=4,015N=4{,}015 pages across D=297D=297 domains, HC1-robust standard errors). Sorted by raw pp -value. Rows with q<0.05q<0.05 are classified as “robustly identified”; rows with 0.05≤q<0.200.05\leq q<0.20 are “borderline”; rows with q≥0.20q\geq 0.20 are “not identified”.
EngineCitations in feature corpus
Claude24,333
Gemini52,244
Google AI69,824
ChatGPT12,121
Appendix
Table 2 : Per-engine citation volume and median citation age.
EngineMedian citation age (months)
Claude5.1
Google AI6.0
Gemini7.8
ChatGPT8.0
Appendix
Table 3 : Cumulative citation share by content age, per engine.
Feature familyPage coverage
Alignment (Jaccard, cosine, paragraph depth)100.0%
Structural (length, links, FAQ, TLDR, author)100.0%
Recency (page age, pub date)80.6%
Real-user CWV (LCP, INP, CLS)90.6%
Page type100.0%
Appendix
Table 4 : Non-missing rate per feature family on the page corpus.
Predictorβ^train\hat{\beta}_{\text{train}}β^hold\hat{\beta}_{\text{hold}}∣Δ∣|\Delta|
Prompt-content alignment+0.605+0.3420.263
Page length+0.004+0.0600.056
Title-prompt similarity+0.115+0.1210.006
Page age+0.323-0.1360.459
Intro similarity+0.135+0.1340.001
Appendix
Table 5 : Top-five predictor coefficients refit independently on training and hold targets ( N=4,015N=4{,}015 URLs in both). Alignment, intro similarity, and title similarity preserve sign and approximate magnitude. Page age flips sign ( +0.323→−0.136+0.323\to-0.136 ) and is treated as temporally non-robust.
ModelIn-sample R2R^{2}
Intercept-only0.000
Word-count only0.004
Domain-only (target-encoded)0.320
Full feature set0.388
Appendix
Table 6 : In-sample R2R^{2} of four nested models. Word count alone: R2=0.004R^{2}=0.004 . Domain-only: R2=0.320R^{2}=0.320 . Full feature set: R2=0.450R^{2}=0.450 . The ΔR2=0.130\Delta R^{2}=0.130 increment is the upper bound on marginal page-level predictive value.

Explore similar work

CardsList

  1. How Large Language Models Source Brand Reputation Across Languages and Markets

    Jun 24, 2026Dmitrij ZatuchinBrand-As-MemoryCredibility

    Department of Information Technologies, Estonian Entrepreneurship University of Applied Sciences (EUAS), Tallinn, Estonia. · Rankfor.AI, Tallinn, Estonia.

  2. Authority, Truth, and Citation Bias: A Large-Scale Multi-Domain Benchmark for Studying Epistemic Susceptibility in Large Language Models

    Jun 11, 2026Aryan Khurana, Aravind Ramana RN, Dhruv KumarCitationsLarge Language Model Reliability

    BITS Pilani, Pilani Campus, Rajasthan, India.

  3. Abstention vs. Hallucination: Benchmarking LLM Source Attribution for Scientific Citations

    May 3, 2024Deepa Tilwani, Yash Saxena, Seyedali Mohammadi +5CitationsLarge Language Model Reliability

    Artificial Intelligence Institute, University of South Carolina, Columbia, South Carolina, USA · University of Maryland, Baltimore County, Baltimore, Maryland, USA · IAIRO, India +1