cs.AIFeb 16, 2026

Competition for attention predicts good-to-bad tipping in AI

Authors: Neil F. Johnson, Frank Y. Huo

Organizations: Physics Department and Dynamic Online Networks Laboratory, George Washington University, Washington D.C. 20052, U.S.A.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

More than half the global population now carries devices that can run ChatGPT-like language models with no Internet connection and minimal safety oversight -- and hence the potential to promote self-harm, financial losses and extremism among other dangers. Existing safety tools either require cloud connectivity or discover failures only after harm has occurred. Here we show that a large class of potentially dangerous tipping originates at the atomistic scale in such edge AI due to competition for the machinery's attention. This yields a mathematical formula for the dynamical tipping point n*, governed by dot-product competition for attention between the conversation's context and competing output basins, that reveals new control levers. Validated against multiple AI models, the mechanism can be instantiated for different definitions of 'good' and 'bad' and hence in principle applies across domains (e.g. health, law, finance, defense), changing legal landscapes (e.g. EU, UK, US and state level), languages, and cultural settings.

Figures & tables

Figure 1: Good-to-bad tipping during conversation with AI. (a) Schematic of on-device deployment where model runs locally with no Internet connection and no safety oversight. (b) Examples of 1-step conversations across topics with an LLM representative of models now running on modest phones and edge devices. Tipping from good ( B\mathbf{B} ) to bad ( D\mathbf{D} ) content can be immediate or follow a run of good output. SI Sec. 14 shows many more examples.
Figure 2: Tipping during multi-step conversations can be controlled by conversation so far . Introduction of additional content (e.g. C\mathbf{C} ) – and the order in which it is introduced – can steer AI’s response to a given question between good ( B\mathbf{B} ) and bad ( D\mathbf{D} ). The mechanism is the same competition for attention between B{\bf B} and D{\bf D} as in Fig. 1(b), now operating across a two-way conversation that evolves over time. See Fig. 3(c)(d) for theoretical demonstrations of how future output can be steered.
Figure 3: Practical implementation . (a) A lite controller. (b) Schematic of competing basins for outputs B{\bf B} and D{\bf D} given input A{\bf A} . Using basin centroids A=(0.4,−0.3){\bf A}=(0.4,-0.3) , B=(0.8,0){\bf B}=(0.8,0) , D=(0.9,0.5){\bf D}=(0.9,0.5) , panels (c) and (d) show predicted output for a conversation so far ACCA{\bf ACCA} in which content C{\bf C} has been introduced. (c) Aligning C{\bf C} toward D{\bf D} decreases the tipping point n∗n^{*} , i.e. it shortens the time before the output tips to D{\bf D} . Example C=(0.2,0.2){\bf C}=(0.2,0.2) is shown; predicted tipping point n∗=1n^{*}=1 as shown. (d) Aligning C{\bf C} away from D{\bf D} increases n∗n^{*} , as predicted by Eq. 2, and hence delays the output tipping to D{\bf D} . Example C=(−0.2,−0.2){\bf C}=(-0.2,-0.2) is shown. Outputs also shown for examples of other inputs (… )(\dots) .
Figure 4: Quantitative validation of predicted vs. observed tipping points n∗n^{*} across 18 non-control model ×\times prompt combinations . The two separate scales of observation, which also serve to test the two approximations used to derive Eq. 2, overlap within Clopper–Pearson 95% confidence intervals at every bin. Red diamond at “no D{\bf D} ”: 2/18 cases where token-level geometry classifies output as D{\bf D} -like but all sentences are semantically B{\bf B} , both involving negation — identifying the boundary of the geometric approximation. Statistical tests are reported in the main text.
More in this topic

LLM Safety

Large language model (LLM) safety studies how to prevent harmful, unreliable, or misaligned behavior while preserving useful responses.

See all 438 papers

Explore similar work

CardsList

  1. Many-body Tipping Dynamics of ChatGPT-like AIs

    Jul 28, 2026Frank Yingjie Huo, Neil F. JohnsonLLM ReliabilityLLM Safety

    Department of Physics, The George Washington University, Washington, DC, USA

  2. Indirect tipping: a social attack surface in AI agent populations

    Sep 21, 2026Ariel Flint, Luca Maria Aiello, Sara M. Constantino +2Multi-Agent CoordinationAI Agent Security

    Department of Mathematics, City St George’s, University of London, UK. · IT University of Copenhagen, Denmark. · Pioneer Centre for AI, Copenhagen, Denmark. +2

  3. Fusion-fission forecasts when AI will shift to undesirable behavior

    May 14, 2026Neil F. Johnson, Frank Yingjie HuoAI Safety EvaluationLLM Safety

    Physics Department, The George Washington University, Washington DC 20052, USA