cs.CYJan 29, 2026

Industrialized Deception: The Collateral Effects of LLM-Generated Misinformation on Digital Ecosystems

Authors: Alexander Loth, Martin Kappes, Marc-Oliver Pahl

Organizations: Frankfurt University of Applied Sciences Frankfurt am Main, Germany · IMT Atlantique, UMR IRISA, Chaire Cyber CNI Rennes, France

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Generative AI and misinformation research have evolved since our 2024 survey. This paper presents an updated perspective, transitioning from literature review to practical countermeasures. We report on changes in the threat landscape, including improved AI-generated content through Large Language Models (LLMs) and multimodal systems - addressing the adverse impacts of AI on information quality. This work focuses on our practical contributions: JudgeGPT, a platform to evaluate human perception of AI-generated news, and RogueGPT, a controlled stimulus generation engine for research. Together, these tools form an experimental pipeline for studying how humans perceive and detect AI-generated misinformation. Our findings show that detection capabilities have improved, but competition between generation and detection continues. We discuss mitigation strategies such as LLM-based detection, inoculation approaches, and the dual-use nature of generative AI.

Figures & tables

Figure 1. Structural overview of Generative AI’s impact on Fake News. The dual-use nature (purple dashed arrow) illustrates how the same technologies enable both creation and detection of synthetic content. Diagram: A central Generative AI node connects to four theme nodes (Creation, Detection, Mitigation, Ethics), each with subnodes (e.g., Text/Image/Audio/Video; Verification/Social Media/Crowdsource; Awareness/Policy/Prebunking; Privacy/Bias/Trust). Additional technology nodes (Transformers, GPT/LLM, GANs, VAEs, Diffusion) are linked; a dashed bidirectional arrow highlights the dual-use relationship between Creation and Detection.
Figure 2. GPT architecture: tokens are embedded with positional encoding, processed through NN transformer blocks (self-attention →\rightarrow layer norm →\rightarrow FFN →\rightarrow layer norm, with residual connections at each stage), then projected to output vocabulary. Block diagram of the GPT pipeline: token embeddings are combined with positional encoding and fed into a repeated transformer block. Inside the block, self-attention and a feed-forward network (FFN) are each followed by LayerNorm, with residual connections. Finally, the output is projected to the vocabulary and passed through a softmax to produce next-token probabilities.
More in this topic

Fake News Detection

Fake news detection identifies false or misleading claims by analyzing content, context, evidence, and how information spreads.

See all 100 papers

Explore similar work

CardsList

  1. The Verification Crisis: Expert Perceptions of GenAI Disinformation and the Case for Reproducible Provenance

    Feb 2, 2026Alexander Loth, Martin Kappes, Marc-Oliver PahlGenerative AI EvaluationScientific Reproducibility

    Frankfurt University of Applied Sciences Frankfurt am Main, Germany · IMT Atlantique, UMR IRISA, Chaire Cyber CNI Rennes, France

  2. Can Humans Tell? A Dual-Axis Study of Human Perception of LLM-Generated News

    Apr 4, 2026Alexander Loth, Martin Kappes, Marc-Oliver PahlAI-Generated Text DetectionLLM Evaluation

    IMT Atlantique, UMR IRISA, Chaire Cyber CNI Rennes, France

  3. Adversarial Robustness of AI-Generated Image Detectors in the Real World

    Oct 2, 2024Sina Mavali, Jonas Ricker, David Pape +2Adversarial AttacksAdversarial Robustness

    CISPA Helmholtz Center for Information Security, Saarbrücken, Germany · Ruhr University Bochum, Bochum, Germany