Agent-based imitation dynamics can yield efficiently compressed population-level vocabularies

Nathaniel Imel; Richard Futrell; Michael Franke; Noga Zaslavsky

cs.CLMar 16, 2026

Agent-based imitation dynamics can yield efficiently compressed population-level vocabularies

Authors: Nathaniel Imel, Richard Futrell, Michael Franke, Noga Zaslavsky

Organizations: Department of Psychology, New York University · Department of Language Science, University of California, Irvine · Department of Linguistics, University of Tübingen

Abstract

Natural languages have been argued to evolve under pressure to efficiently compress meanings into words by optimizing the Information Bottleneck (IB) complexity-accuracy tradeoff. However, the underlying social dynamics that could drive the optimization of a language's vocabulary towards efficiency remain largely unknown. In parallel, evolutionary game theory has been invoked to explain the emergence of language from rudimentary agent-level dynamics, but it has not yet been tested whether such an approach can lead to efficient compression in the IB sense. Here, we provide a unified model integrating evolutionary game theory with the IB framework and show how near-optimal compression can arise in a population through an independently motivated dynamic of imprecise strategy imitation in signaling games. We find that key parameters of the model -- namely, those that regulate precision in these games, as well as players' tendency to confuse similar states -- lead to constrained variation of the tradeoffs achieved by emergent vocabularies. Our results suggest that evolutionary game dynamics could potentially provide a mechanistic basis for the evolution of vocabularies with information-theoretically optimal and empirically attested properties.

Figures & tables

Figure 1: A. The IB communication model of Zaslavsky et al. (2018) , see Section 2.1 for details. B. Communication model unifying noisy sim-max games with the IB framework. We illustrate our model on a semantic domain similar to numerals, and depict states as cards with varying quantities of dots, and meanings as (conditional) belief distributions about these states. We denote the (symmetric) noisy perception and interpretation of states as $p$ . C. The imprecise conditional imitation dynamic ( Franke and Correia, 2018 ) . See main text and Appendix A for details.

Figure 2: Emergent efficient semantic systems. A : Converged systems on the information plane, colored by the pragmatic standard of precision $\gamma$ in the sim-max game. Circles denote systems evolved from our model, derived from Franke and Correia (2018) , abbreviated as FC18. See Movie 1 visualizing the dynamics on the plane. Gray circles mark the result of permuting the systems evolved from the FC18 dynamics (see Appendix D ). Gray triangles correspond to emergent systems evolved from a dynamic proposed in Nowak and Krakauer (1999) , abbreviated as NK99, which we consider as another baseline. B : Game pragmatic standard of precision ( $\gamma$ , Equation 6 ) vs. efficiency loss ( $\epsilon$ , Equation 17 ) of converged systems. Mean scores for systems emerging from our model are depicted in blue while mean scores for permuted counterparts are in gray. Shaded regions correspond to the $95\%$ confidence interval. The inset shows a zoomed-in view of the emergent systems’ scores only.

Figure 3: Sample of converged emergent systems and their optimal counterparts. A - C : 1-dimensional mode maps of converged system for $\gamma=0.001,0.01,10$ ( Equation 6 ). In each plot, the $x$ -axis denotes the meaning space $\mathcal{X}_{o}=\{0,\dots,99\}$ , the $y$ -axis denotes probability $S(w|m_{o})$ , and lines are the modal word $w$ (out of $100$ possible words) used for each meaning $m_{o}=x_{o}$ . The color of each line corresponds to the average meaning that the word is used to communicate, with black corresponding to $x_{1}$ and bright orange corresponding to $x_{100}$ . D - F : Mode maps for the optimal counterparts of the emergent systems in D - F , fitted by efficiency loss $\epsilon$ ( Equation 17 ). See Movie 2 for the evolution of the system in A .

Figure 4: Complexity (A), accuracy (B), population fitness (C), and efficiency loss (D), relative to game pragmatic standard of precision ( $\gamma$ ) across discrete time step of simulated evolution. Lines in each plot show mean (with 95% CIs) across eight random seeds. Color corresponds to $\gamma$ ( Equation 6 ).

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: The imprecise conditional imitation dynamic ( Franke and Correia, 2018 ) . The frequency of word $w$ being used to communicate state $x^{\mathrm{im}}_{o}$ at the next time step grows in proportion to (i) the expected frequency of $w$ given that $x^{\mathrm{im}}_{o}$ was observed by some (randomly sampled) imitating Sender, times (ii) the expected utility of this word relative to the population of Receivers. Likewise, the frequency of interpreting $w$ as meaning $\hat{x}^{\mathrm{im}}_{o}$ grows in proportion to (i) the probability that $\hat{x}^{\mathrm{im}}_{o}$ is observed by a randomly sampled agent imitating a randomly sampled Receiver that actually realized $\hat{x}_{a}$ after receiving word $w$ , times (ii) the expected utility of the interpretation $\hat{x}^{\mathrm{im}}_{o}$ given $w$ . See Equations 11 , 12 , 13 , 14 , 15 and 16 for the details of these conditional expectations.

Notation	Interpretation
$x_{a}$	actual state
$x_{o}$	Sender’s observed state
$x^{im}_{o}$	an imitating Sender’s observed state
$m_{o}(x_{a})\equiv p(x_{a}\|x_{o})$	observation noise / belief state
$p(m_{o})$	communicative need distribution
$w$	signal

Appendix

Table 1: Mapping between our notation, Franke and Correia (2018) ’s notation, and Zaslavsky et al. (2018) ’s notation.

Figure 6: Monotonic relationship between

\gamma

and

\beta

S_{i,mw}^{(t+1)}\leftarrow\text{Sample from }\{S_{p,mw}^{(t)}\},\quad R_{i,wm}^{(t+1)}\leftarrow\text{Sample from }\{R_{p,wm}^{(t)}\}

Appendix

Algorithm 1 Evolutionary Dynamics for Signal-Object Associations

Information Bottleneck

Information Bottleneck studies how to compress representations while retaining information needed for prediction, control, or communication.

See all 56 papers

From Signals to Structure: How Memory Architecture Drives Language Emergence in LLM Agents
Jun 30, 2026Yashar Talebirad, Eden Redman, Ali Parsaee +1Multi-Agent LLM Systems Multi-Agent Coordination
Alberta Machine Intelligence Institute, University of Alberta, Edmonton, Canada · Network for Applied Technology, Edmonton, Canada
On the Optimality of Kinship Naming: an Information-theoretic Approach
Nov 24, 2025Phong Le, Mees Lindeman, Raquel G. AlhamaRate-Distortion Optimization
University of St Andrews · Institute for Logic, Language and Computation; University of Amsterdam.
zip2zip++: Towards Inference-Time Token Compression with Minimal Quality Loss
Oct 9, 2026Saibo Geng, Andrea Gentilini, Xinxian Ma +3Token Compression LLM Compression
EPFL · Technical University of Munich · Zuse School ELIZA +1

Jun 30, 2026J/K move · Enter open · S save

From Signals to Structure: How Memory Architecture Drives Language Emergence in LLM Agents

Yashar Talebirad, Eden Redman, Ali Parsaee, Osmar R. Zaiane

Alberta Machine Intelligence Institute, University of Alberta, Edmonton, Canada · Network for Applied Technology, Edmonton, Canada

Multi-Agent LLM Systems Multi-Agent Coordination Memory-Augmented Agents Emergent Communication

Open paper AskPDF arXiv

How do two agents invent a shared language from scratch? In a Lewis signaling game, a sender and receiver must coordinate on a code using only their interaction history. We study five memory architectures across varying channel configurations with LLM agents and find that memory architecture matters more than channel capacity. Agents with a persistent private notebook benefit from surplus channel capacity and avoid the high-capacity collapse seen in stateless agents, achieving the most reliable coordination (

0.867 \pm 0.023

at capacity = 25). Stateless agents peak at moderate capacity and then degrade as the vocabulary grows beyond what a rolling context window can track The notebook externalizes learned conventions, freeing agents from having to re-derive codes each round. An information bottleneck-inspired argument predicts an optimal capacity equal to the number of objects. Instead, the bottleneck (capacity = 8) proves to be a fragility point, and surplus capacity is generally better. We show that channel capacity alone cannot predict coordination; memory architecture determines whether agents turn interaction history into stable conventions, and both dimensions are needed to understand how signals become language.

Abstract

Information Bottleneck

Explore similar work

From Signals to Structure: How Memory Architecture Drives Language Emergence in LLM Agents

On the Optimality of Kinship Naming: an Information-theoretic Approach

zip2zip++: Towards Inference-Time Token Compression with Minimal Quality Loss