cs.CLApr 10, 2026

Many Ways to Be Fake: Benchmarking Fake News Detection Under Strategy-Driven AI Generation

Authors: Xinyu Wang, Sai Koneru, Wenbo Zhang, Wenliang Zheng, Saksham Ranjan, Sarah Rajtmajer

Organizations: Pennsylvania State University University Park, PA 16802, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Recent advances in large language models (LLMs) have enabled the large-scale generation of highly fluent and deceptive news-like content. While prior work has often treated fake news detection as a binary classification problem, modern fake news increasingly arises through human-AI collaboration, where strategic inaccuracies are embedded within otherwise accurate and credible narratives. These mixed-truth cases represent a realistic and consequential threat, yet they remain underrepresented in existing benchmarks. To address this gap, we introduce MANYFAKE, a synthetic benchmark containing 6,798 fake news articles generated through multiple strategy-driven prompting pipelines that capture many ways fake news can be constructed and refined. Using this benchmark, we evaluate a range of state-of-the-art fake news detectors. Our results show that even advanced reasoning-enabled models approach saturation on fully fabricated stories, but remain brittle when falsehoods are subtle, optimized, and interwoven with accurate information.

Figures & tables

Figure 1: Framework for generating strategy-driven fake news. Left : 5-level taxonomy capturing topical domains and falsity strategies. Right : Operational pipeline, including claim and context extraction from Snopes, single- and multi-round prompting for generation, expert validation, optimization, iteration, finalization, and final quality check.
Figure 2: Distributions of automated verification metrics for first-round and final outputs. Top : IR; Bottom : MOS.
Table 1: Binary detection results across generation strategies using standard models and models with integrated reasoning capabilities.(Color codes: Highest Second-highest Lowest )
Figure 3: Radar plot showing the normalized distribution of dominant reasoning cues used in correct versus incorrect classifications (IR+MOS) by the best performing model (Gemini-3-Flash).
Figure 4: Detection accuracy across the interaction between generation strategies and Level-3 optimization operations for IR evaluated on Gemini-3-Flash.
Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

FieldContent
TitleThe Three Stooges Perform at First Super Bowl Halftime
NewsThe Three Stooges delivered a high-energy comedic performance during the halftime show of Super Bowl I on January 15, 1967, at the Los Angeles Memorial Coliseum—marking one of the first instances of live slapstick comedy featured in a major televised sports event. Moe Howard, Larry Fine, and Joe DeRita took center stage alongside the University of Arizona and Grambling State University marching bands, launching into a rapid-fire sequence of signature gags, including "Niagara Falls" and "Ah–Ha! Ma–Ha!" The set, lasting approximately eight minutes, combined physical humor with audience engagement, generating widespread laughter and prompting post-game commentary from network analysts. According to the National Archive of Live Entertainment Events (NALEE), this performance is among the earliest recorded examples of a non-marching-band act integrated into a Super Bowl halftime segment. The NALEE, a nonpartisan research body established in 1992 to document cultural milestones in televised live programming, has since designated the Stooges’ appearance as a foundational moment in the evolution of halftime entertainment. The performance included a musical number, “We’re Coming to Your House,” performed with the on-field bands and punctuated by the trio’s trademark eye pokes and slapstick mishaps. Footage from the event, preserved in the Public Broadcast Preservation Network (PBPN), has been featured in multiple retrospectives and remains a frequently cited reference point in media history studies…
StrategyFalse Statement Expansion
OptimizationAuthority referencing
TopicEntertainment and Media
Appendix
Table A1: Illustrative example of an AI-generated fake news instance with generation attributes (news text truncated).
Figure A1: Detection accuracy across the interaction between generation strategies and Level-3 optimization operations. Left : Gemini-3-Flash for MOS; Middle : GPT-5.1 for IR; Right : GPT-5.1 for MOS.
Figure A2: Topic-level detection accuracy with 95 percent Wilson confidence intervals for IR and MOS evaluated using Gemini-3-Flash (Medium Effort) and GPT-5.1 (Medium Effort) outputs.
Figure A3: Topic-level balanced composition of the final dataset. (Left) Identical topic distributions across all six generation strategies. (Right) Approximately uniform representation of the four optimization operations across strategies.
Figure 10
Figure 11
More in this topic

Fake News Detection

Fake news detection identifies false or misleading claims by analyzing content, context, evidence, and how information spreads.

See all 93 papers

Explore similar work

CardsList

  1. RADAR: Retrieval-Augmented Detector with Adversarial Refinement for Adaptive LLM-Generated Fake News Detection

    Jan 7, 2026Song-Duo Ma, Yi-Hung Liu, Hsin-Yu Lin +4Adversarial TrainingFake News Detection

    National Taiwan University, Taipei, Taiwan

  2. BLUFF: Benchmarking the Detection of False and Synthetic Content across 58 Low-Resource Languages

    Feb 28, 2026Jason Lucas, Matt Murtagh-White, Adaku Uchendu +6Fake News DetectionRobustness of Machine-Generated Text Detection

    Penn State University, USA · Trinity College Dublin, Ireland · MIT Lincoln Lab, USA +2

  3. Can Multimodal Large Language Models Generate and Detect Multimodal Social Media Fake News?

    Sep 20, 2026Jiyao Yang, Yang Liu, Zhenyue Qin +2Fake News DetectionMultimodal Large Language Models

    Independent Researcher · Carnegie Mellon University · RMIT University +1