With the rapid diffusion of AI-generated content, AI-driven misinformation is becoming increasingly pervasive and difficult to govern, undermining information credibility and social trust. This study models the strategic interdependence among a government regulator, an AI enterprise, and users through a three-party evolutionary game that incorporates heterogeneous rewards and punishments. From the resulting replicator equations, we characterize the evolutionary stability of competing governance and production strategies. The analysis indicates that neither unilateral regulation nor market incentives alone can effectively curb misinformation. Instead, an evolutionarily stable regime of real-information production arises only when regulatory rewards and punishment intensity, enterprise reputation loss, and user adoption incentives collectively surpass critical thresholds. The findings highlight the need for coordinated and adaptive policy mixes that align regulatory instruments with enterprise behavior and user uptake while managing governance costs.
The rapid proliferation of AI applications has intensified debate on effective regulation of these black-box services. Effective regulation must balance two competing goals: (1) deterring non-compliant providers from entering the market, while (2) retaining compliant ones. We call this ideal the perfect market outcome (PMO). Regulators face two compounding obstacles that make PMO difficult to achieve: providers hold private information and can act strategically to evade compliance, while any evidence drawn or derived from a finite sample carries statistical uncertainty in proving non-compliance. As this information asymmetry and statistical uncertainty is inherent to any effective regulation, we formalise them through a mechanism design framework that explicitly accounts for such statistical uncertainty. This yields a sharp characterisation: a mechanism achieves PMO if and only if the set of non-compliant evidence distributions forms a closed, convex set of probability measures, known in imprecise probability as a credal set. This result serves as a diagnostic tool to determine whether PMO is achievable under a given regulation. We further show that PMO-achieving mechanisms can be constructed from a collection of hypothesis tests, and validate our theoretical contributions through experiments on spurious-feature and fairness-based regulations.
This paper develops a variational framework for regulated language generation. Starting from autoregressive token sampling, we derive the induced distribution over complete messages and relate it to an entropy-regularized Gibbs law. Regulation is modeled as an optimal discriminator whose convex-dual value is an f-divergence, and the generator-regulator interaction is formulated as a saddle-point problem. The framework applies to moderation, censorship, AI deception detection, compliance auditing, phishing defense, and manipulation control, where regulation concerns a distribution over possible messages rather than a single output. The equilibrium clarifies the tradeoff among utility, entropy, regulatory alignment, and finite-length detectability. Two finite-vocabulary case studies, censorship filtering and phishing defense, illustrate how the theory can be evaluated through utility, entropy, divergence, receiver-side scores, and detection probability.
Generative AI (GenAI) search systems are transforming how users access information. Unlike ranking-based search systems, where users observe a ranked list of documents, GenAI search systems, given a user's question, generate an answer, often accompanied by external sources (e.g., in the form of citations). Content creators (publishers) seeking to increase exposure might behave strategically and compete with other creators for users' attention. While publishers in ranking-based systems might strategically modify their content to improve its ranking, the incentives in generative systems take on a new form. Publishers may now gain exposure through generated responses and attributions to those responses. We introduce a novel game-theoretic model of the emerging GenAI ecosystem in which publishers compete for attribution-based exposure. We study the learning dynamics of strategic content creators under better-response dynamics. We associate the convergence of learning dynamics to equilibrium with ecosystem stability. Employing the notion of potential games, we study the stability of GenAI ecosystems under several known content selection mechanisms. We demonstrate the instability of mechanisms representing real-world modern systems and characterize a mechanism that induces a stable ecosystem. We conduct extensive simulations to analyze the stability and welfare of GenAI ecosystems under various mechanisms. The simulations support our theoretical findings and reveal an interplay among stability, publisher welfare, and user welfare. In particular, stable mechanisms do not necessarily maximize welfare, demonstrating an important trade-off for platform designers. We then introduce a study illustrating that the proper selection of the GenAI mechanism enables the manifestation of desired trade-offs between publisher welfare and the different sources of user welfare.