Claims of emergence in AI should be grounded in information decomposition

Christoph Riedl; Fernando E. Rosas

cs.AIOct 20, 2026

Claims of emergence in AI should be grounded in information decomposition

Authors: Christoph Riedl, Fernando E. Rosas

Organizations: Khoury College of Computer Sciences Northeastern University Boston, MA 02115 · Department of Informatics University of Sussex Brighton, BN1 9RH

Abstract

When a language model develops capabilities not traceable to its individual components, when a multi-agent AI system outperforms each agent on its own, or when a human-AI pair succeeds where either fails in isolation, AI researchers describe it as a case of synergy'' or emergence''. However, these are usually informal assessments not supported by a theoretical framework, making these claims impossible to compare or even meaningfully dispute. Here we argue that information decomposition can serve as a unifying formal framework to operationalize assessments about synergy in AI systems. Information decomposition provides a way to distinguish qualitatively different types of information, including unique information available from individual components, redundant information shared across multiple components, and synergistic information accessible only from parts taken together. This gives claims of emergence a precise, falsifiable criterion: a system's behavior is emergent only if it carries synergistic information about task-relevant outcomes. We apply this lens to attention heads, autonomous agents, and human-AI teams, showing how it organizes existing knowledge across scales and substrates, while also generating cross-level conjectures that siloed fields would miss.

Figures & tables

Figure 1: a) Information decomposition with two source variables

X_{1}

and

X_{2}

, and target variable

Y

. b) The PID conceptual triangle resolves ambiguities in system dynamics. Traditional dichotomies in complex systems literature often conflate mathematically distinct properties. Integration conflates redundancy (integration-as-oneness) with synergy (complementary cooperation). Differentiation conflates unique information (strict independence) with synergy (interactive differentiation). Furthermore, locality conflates unique (singularly local) with redundant (multiply local) information, whereas synergistic representations are strictly de-localized (adapted from Luppi et al. [55] ).

Figure 2: LLMs exhibit a synergistic core surrounded by a redundant periphery. a) Heatmaps of synergy and redundancy between pairs of attention heads in Gemma 3 4B. b) Heatmap of the synergy-redundancy ranks in Gemma 3 4B. Figures adapted from Urbina-Rodriguez et al. [101] .

Figure 3: Emergent synergy in multi-agent LLM systems. a) Practical emergence criterion (bias corrected). b) Emergence capacity dynamical synergy (bias corrected). c) Total Stability (time-delayed mutual information of triplets). Data were Winsorized at the 1st and 99th percentiles for visual clarity. Stars indicate significance level of Wilcoxon test. Figures adapted from Riedl [82] . Notes: *** $p<0.001$ ; ** $p<0.01$ ; * $p<0.05$ .

Figure 4: Viewing human-AI synergy through an information decomposition lens unifies existing theories about substitution, linear and non-linear complementarity.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

	Question / stage	Recommended approach	Key caveat
Decide	What are the parts, target, and timescale?	Fix parts, target, timescale $\ell$ , and redundancy measure on theoretical grounds	Not given by the theory; claims are bounded by these choices. Substantiate claims by demonstrating robustness across granularities (e.g., heads and experts), orders (e.g., pairs/triplets), and targets (e.g., group error and principal component)
Estimate	Small system ( $n\leq\sim 5$ ), full structure needed	Exact PID, named redundancy measure (BROJA bivariate; $I_{ccs}$ / $I_{min}$ discrete)	Report the measure; check $\geq 2$ for robustness
	Large system, net balance suffices	Localized/gradient O-information	Conflates Red/Syn; use per-node, not one global number; report strength measure (S-information or total correlation) to distinguish more synergy from less redundancy; size-matched random baseline to address bias.
	“Is there any emergence?” screen	Emergence-capacity (pairwise $\mathrm{Syn}>0$ ); $\Psi_{\ell}(V)>0$ (Eq. 4)	Negative $\Psi$ inconclusive (redundancy subtracted up to $n$ times); consider redundancy-corrected $\Psi(k,q)$ [ 92 ]
	LLM circuits—which units	Pairwise/subset synergy $\rightarrow$ per-node ranks (as in the core work)	Blind to high-order; needs null; add ablation for causal claims
	Dynamical / multi-agent over time	TDMI-based causal-emergence / $\Phi$ ID	Requires block-shuffled nulls; timescale $\ell$ is a choice; state $\ell$ and report at least one alternative $\ell$ .

Appendix

Table S1: Practical guidelines for applying information decomposition across the research workflow.

PID Principle	Observable Signature	Design Levers
Synergy requires differentiation	Identical experts / agents produce only redundancy	Sparsity and routing in MoE systems (larger $k$ in top- $k$ gating $\uparrow$ synergy [ 101 , 29 ] ); data augmentation ( $\uparrow$ redundancy [ 98 ] ); personas/roles ( $\uparrow$ synergy [ 82 ] ); division of labor ( $\uparrow$ uniqueness) vs. co-creation ( $\uparrow$ synergy)
Synergy is fragile, redundancy is robust	Sharp failure modes vs. graceful degradation; flat-gradient stalling on synergistic tasks; robust OOD when structure is stable vs. brittle if composition shifts; path-dependent collapse	Dropout ( $\uparrow$ redundancy [ 2 , 74 ] ); BatchNorm / LayerNorm ( $\uparrow$ synergy potential [ 20 ] ); early stopping ( $\uparrow$ redundancy) vs. extended training, warmup schedules ( $\uparrow$ synergy); individual rewards ( $\uparrow$ independence & uniqueness) vs. shared rewards ( $\uparrow$ synergy); shaped rewards for coordination ( $\uparrow\uparrow$ synergy)
Synergistic information is non-local	Not locally attributable; modular interpretability fails	Attention mechanism ( $\uparrow$ synergy [ 101 ] ); data curation in multi-modal training ( $\uparrow$ synergy [ 52 ] ); centralized ( $\uparrow$ synergy) vs. decentralized training execution ( $\uparrow$ redundancy); adaptivity & personalization ( $\uparrow$ synergy [ 14 , 49 , 43 ] )
Synergy provides superlinear capacity	Superlinear performance gains but data-hungry estimation	Network width ( $\downarrow$ synergy) vs. depth ( $\uparrow$ synergy); ToM in multi-agent systems & human-AI ( $\uparrow$ synergy; $\uparrow$ differentiation [ 78 , 80 , 82 ] ); full observability ( $\downarrow$ synergy) vs. partial observability ( $\uparrow$ synergy)

Appendix

Table S2: Overview of control parameters and their effects across layers.

Rung	Synergistic core [ 101 ]	Multi-agent [ 82 ]	Human-AI [ 80 ]
Detect	Pairwise $\Phi$ ID synergy ranks	$\Psi>0$ ; emergence capacity $>0$	Bounded residual; formal PID $\mathrm{Syn}>0$
Localize	Middle layers; heads and experts	Agent pairs, not triplets; identity-linked roles	Hard and AI-unsolvable items; $\kappa$ trait
Identify causally	Ablation of synergistic vs. redundant heads	Randomized Plain / Persona / ToM conditions	Randomized assignment; users $\times$ items crossed; phases rule out learning
Control	RL fine-tuning of synergistic regions	ToM prompt as control parameter	Open: ToM is measured, not manipulated
Predict	Conjectures 1–4	Conjecture 5	Conjecture 5

Appendix

Table S3: Evidence ladder across the three empirical settings.

Name	License	Language	Cont.	Disc.	Framework	Ref.
Partial Information Decomposition
IDTxl	GPLv3	Python	✓	✓	PID, PED	[ 108 ]
dit	BSD-3	Python	×	✓	PID	[ 39 ]
BROJA_2PID	Apache 2	Python	×	✓	PID (bivariate)	[ 57 ]
SxPID	GPLv3	Python	×	✓	PID	[ 56 ]
infotheory	MIT	Python/C++	✓	✓	PID	[ 11 ]

Appendix

Table S4: Non-exhaustive list of available software packages for computing information decomposition and related quantities. Support spans popular environments including Python and R. Runtime and scalability depend on both the entropy estimator (e.g., Gaussian approximation, non-parametric methods) and the specific decomposition. Packages are grouped by scope: full PID implementations, and scalar higher-order interaction summaries (e.g., O-information).

Hypothesis
Grokking as redundancy-to-synergy transition:
Track O-information $\Omega$ of the hidden representation across training; confirmed if $\Omega$ flips from positive to negative within the test-accuracy jump (and an $\Omega$ -penalty delays it), falsified if $\Omega$ drifts smoothly with train accuracy.
Jagged intelligence as a factored-representation boundary:
Score each task by synergistic depth $\mathrm{Syn}(F_{1},...,F_{k};Y)$ ; confirmed if accuracy falls with depth after partialling out human-rated difficulty, falsified if accuracy is unrelated to $\mathrm{Syn}$ or difficulty dominates once depth is controlled.
Test-time compute as synergy unrolling:
Measure per-pass synergy (synergy-redundancy rank) for a reasoning vs. base model; confirmed if the reasoning model shows lower per-pass synergy and compute gains shrink with synergistic depth, falsified if its per-pass synergy is equal or higher.

Appendix

Table S5: Hypotheses in testable form.

Emergent Abilities in Language Models

See all 12 papers

Human-AI Co-Evolution and Epistemic Collapse: A Dynamical Systems Perspective
May 7, 2026Xuening Wu, Yanlan Kang, Qianya Xu +5Model Collapse Human-AI Interaction
Fudan University, Shanghai, China · Pfizer China, Shanghai, China · University of California San Diego, CA, USA +4
Gradual Cognitive Externalization: From Modeling Cognition to Constituting It
Apr 6, 2026Zhimin ZhaoCognitive Offloading Human-AI Collaboration
Software Analysis and Intelligence Lab (SAIL), Queen’s University
TeamGym: A Gym for Team Decision-Making by AI Agents with Distributed Information
Oct 27, 2026Chanwoo Park, Jaywoong Jeong, Zachary Wojtowicz +4Multi-Agent Collaboration Multi-Agent Systems
MIT · KAIST · UT Austin +1

May 7, 2026J/K move · Enter open · S save

Human-AI Co-Evolution and Epistemic Collapse: A Dynamical Systems Perspective

Xuening Wu, Yanlan Kang, Qianya Xu, Kexuan Xie+4

Fudan University, Shanghai, China · Pfizer China, Shanghai, China · University of California San Diego, CA, USA+4

Model Collapse Human-AI Interaction Human-AI Collaboration

Open paper AskPDF arXiv

Large language models (LLMs) are reshaping how knowledge is produced, with increasing reliance on AI systems for generation, summarization, and reasoning. While prior work has studied cognitive offloading in humans and model collapse in recursive training, these effects are typically considered in isolation. We propose a unified perspective: humans and language models form a coupled dynamical system linked by a feedback loop of usage, generation, and retraining. We introduce a minimal model with three variables -- human cognition, data quality, and model capability -- and show that this feedback can give rise to distinct dynamical regimes. Our analysis identifies three regimes: co-evolutionary enhancement, fragile equilibrium, and degenerative convergence. Through a simple simulation, we demonstrate that increasing reliance on AI can induce a transition toward a low-diversity, suboptimal equilibrium. From an information-theoretic perspective, this transition corresponds to an emergent information bottleneck in the human-AI loop, where entropy reduction reflects loss of diversity and support under closed-loop feedback rather than beneficial compression. These results suggest that the trajectory of AI systems is shaped not only by model design, but by the dynamics of human-AI co-evolution.

Abstract

Emergent Abilities in Language Models

Explore similar work

Human-AI Co-Evolution and Epistemic Collapse: A Dynamical Systems Perspective

Gradual Cognitive Externalization: From Modeling Cognition to Constituting It

TeamGym: A Gym for Team Decision-Making by AI Agents with Distributed Information