cs.CLMar 6, 2025

VQEL: Enabling Self-Play in Emergent Language Games via Agent-Internal Vector Quantization

Authors: Mohammad Mahdi Samiei Paqaleh, Mehdi Jamalkhah, Mahdieh Soleymani Baghshah

Organizations: Department of Computer Engineering, Sharif University of Technology · School of Electrical and Computer Engineering, University of Tehran

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Emergent Language (EL) focuses on the emergence of communication among artificial agents. Although symbolic communication channels more closely mirror the discrete nature of human language, learning such protocols remains fundamentally difficult due to the non-differentiability of symbol sampling. Existing approaches typically rely on high-variance gradient estimators such as REINFORCE or on continuous relaxations such as Gumbel-Softmax, both of which suffer from limitations in training stability and scalability. Motivated by cognitive theories that emphasize intrapersonal processes preceding communication, we explore self-play as a substrate for language emergence prior to mutual interaction. We introduce Vector Quantized Emergent Language (VQEL), a novel architecture that incorporates vector quantization into the message generation process. VQEL enables agents to perform self-play using discrete internal representations derived from a learned codebook while preserving end-to-end differentiability. Moreover, the resulting vector-quantized codebook naturally induces a symbolic vocabulary that can be directly transferred and aligned during subsequent mutual play with other agents. Empirical results show that agents pretrained via VQEL self-play achieve more consistent symbol alignment and higher task success when later engaged in mutual interaction. These findings position self-play as a principled and effective mechanism for learning discrete communication protocols, addressing key optimization and representational challenges in emergent language systems.

Figures & tables

Figure 1: Overview of self-play and mutual play. In self-play, the sender independently develops a symbolic language using a contrastive loss. The sender then interacts with the receiver in mutual play, where the self-developed language is refined through communication using the same contrastive loss.
Figure 2: Left: Overview of the message generator and the corresponding messages produced during self-play and mutual play. Right: Visualization of the embedding space, where the GRU output z1\mathbf{z}_{1} is mapped to its nearest codebook embedding e7\mathbf{e}_{7} .
DatasetMethodSender UpdateACC ↑\uparrowAW ↑\uparrowTopSim ↑\uparrowH(C|M) ↓\downarrow
O BJECTSGS-ST-0.78±0.010.78_{\pm 0.01}0.93±0.060.93_{\pm 0.06}0.21±0.020.21_{\pm 0.02}1.04±0.031.04_{\pm 0.03}
REINFORCE-0.51±0.210.51_{\pm 0.21}0.47±0.120.47_{\pm 0.12}0.14±0.070.14_{\pm 0.07}2.21±1.282.21_{\pm 1.28}
VQEL-SP S{}_{\text{S}}-0.82±0.010.82_{\pm 0.01}1.00±0.001.00_{\pm 0.00}0.19±0.010.19_{\pm 0.01}0.12±0.02{0.12_{\pm 0.02}}
VQEL-SP S{}_{\text{S}} +MPFrozen0.85±0.01‾\underline{0.85_{\pm 0.01}}1.00±0.001.00_{\pm 0.00}0.19±0.010.19_{\pm 0.01}0.12±0.02{0.12_{\pm 0.02}}
VQEL-SP S{}_{\text{S}} +MPRL0.86±0.01\mathbf{0.86_{\pm 0.01}}1.00±0.001.00_{\pm 0.00}0.19±0.010.19_{\pm 0.01}0.12±0.02{0.12_{\pm 0.02}}
VQEL-SP S{}_{\text{S}} +MPRL+Pres0.86±0.01\mathbf{0.86_{\pm 0.01}}1.00±0.001.00_{\pm 0.00}0.19±0.010.19_{\pm 0.01}0.12±0.02{0.12_{\pm 0.02}}
Table 1: Performance comparison across datasets and evaluation metrics for the sender self-play game.
Figure 3: Accuracy of the sender self-play game compared to baseline methods for varying numbers of candidates at test time.
Figure 4: Comparison of the number of unique messages produced by VQEL, during the sender self-play game, and baseline models.
DatasetMethodACC ↑\uparrowAW ↑\uparrowTopSim ↑\uparrowH(C|M) ↓\downarrow
O BJECTSGS-ST0.78±0.010.78_{\pm 0.01}0.93±0.060.93_{\pm 0.06}0.21±0.020.21_{\pm 0.02}1.04±0.031.04_{\pm 0.03}
REINFORCE0.51±0.210.51_{\pm 0.21}0.47±0.120.47_{\pm 0.12}0.14±0.070.14_{\pm 0.07}2.21±1.282.21_{\pm 1.28}
VQEL-SP S{}_{\text{S}}0.82±0.010.82_{\pm 0.01}1.00±0.001.00_{\pm 0.00}0.19±0.010.19_{\pm 0.01}0.12±0.020.12_{\pm 0.02}
VQEL-SP R{}_{\text{R}}0.81±0.010.81_{\pm 0.01}1.00±0.001.00_{\pm 0.00}0.19±0.010.19_{\pm 0.01}0.12±0.000.12_{\pm 0.00}
VQEL-SP S,R{}_{\text{S,R}} +MP0.90±0.01\mathbf{0.90_{\pm 0.01}}1.00±0.001.00_{\pm 0.00}0.17±0.020.17_{\pm 0.02}0.14±0.020.14_{\pm 0.02}
S HAPEGS-ST0.82±0.010.82_{\pm 0.01}1.00±0.001.00_{\pm 0.00}0.02±0.010.02_{\pm 0.01}1.21±0.111.21_{\pm 0.11}
Table 2: Performance comparison across datasets and evaluation metrics for the sender and receiver self-play game.
Figure 5: Accuracy of the sender and receiver self-play game compared to baseline methods for varying numbers of candidates at test time.
DatasetMethodSender UpdateACC ↑\uparrowAW ↑\uparrowTopSim ↑\uparrowH(C|M) ↓\downarrow
O BJECTSGS-ST-0.78±0.010.78_{\pm 0.01}0.93±0.060.93_{\pm 0.06}0.21±0.020.21_{\pm 0.02}1.04±0.031.04_{\pm 0.03}
REINFORCE-0.51±0.210.51_{\pm 0.21}0.47±0.120.47_{\pm 0.12}0.14±0.070.14_{\pm 0.07}2.21±1.282.21_{\pm 1.28}
VQEL-SP S{}_{\text{S}} +MPRL0.86±0.01\mathbf{0.86_{\pm 0.01}}1.00±0.001.00_{\pm 0.00}0.19±0.010.19_{\pm 0.01}0.12±0.020.12_{\pm 0.02}
VQEL-MPRL0.28±0.180.28_{\pm 0.18}1.00±0.001.00_{\pm 0.00}0.22±0.040.22_{\pm 0.04}2.64±0.322.64_{\pm 0.32}
S HAPEGS-ST-0.82±0.010.82_{\pm 0.01}1.00±0.001.00_{\pm 0.00}0.02±0.010.02_{\pm 0.01}1.21±0.111.21_{\pm 0.11}
REINFORCE-0.86±0.000.86_{\pm 0.00}0.83±0.150.83_{\pm 0.15}0.01±0.010.01_{\pm 0.01}0.88±0.110.88_{\pm 0.11}
Table 3: Effect of self-play on the agents’ communication performance.
Figure 6: Training stability across four datasets. Curves show the mean accuracy over multiple random seeds, and shaded regions denote one standard deviation. Our method demonstrates reduced variance on Objects and Shape, while exhibiting comparable stability to the baselines on DSprites and CelebA.
DatasetMethodSP TrainingACC ↑\uparrowAW ↑\uparrowTopSim ↑\uparrowH(C|M) ↓\downarrow
O BJECTSGS-ST-0.78±0.010.78_{\pm 0.01}0.93±0.060.93_{\pm 0.06}0.21±0.020.21_{\pm 0.02}1.04±0.031.04_{\pm 0.03}
REINFORCE-0.51±0.210.51_{\pm 0.21}0.47±0.120.47_{\pm 0.12}0.14±0.070.14_{\pm 0.07}2.21±1.282.21_{\pm 1.28}
VQEL-SP S{}_{\text{S}} +MPVQ-ST0.86±0.01\mathbf{0.86_{\pm 0.01}}1.00±0.001.00_{\pm 0.00}0.19±0.010.19_{\pm 0.01}0.12±0.020.12_{\pm 0.02}
GS-SP S{}_{\text{S}} +MPGS0.77±0.010.77_{\pm 0.01}1.00±0.001.00_{\pm 0.00}0.24±0.030.24_{\pm 0.03}1.15±0.181.15_{\pm 0.18}
GS-SP S{}_{\text{S}} +MPGS-ST0.74±0.060.74_{\pm 0.06}1.00±0.001.00_{\pm 0.00}0.22±0.030.22_{\pm 0.03}1.25±0.251.25_{\pm 0.25}
S HAPEGS-ST-0.82±0.010.82_{\pm 0.01}1.00±0.001.00_{\pm 0.00}0.02±0.010.02_{\pm 0.01}1.21±0.111.21_{\pm 0.11}
Table 4: Comparison of GS with self-play and VQEL in the sender self-play setting. Despite using the same self-play protocol, VQEL consistently achieves higher accuracy than GS with self-play, highlighting the importance of the VQ module in enabling effective self-play.
Figure 7: Effect of communication capacity on test accuracy. Test accuracy on ShapeWorld under different vocabulary sizes (10, 20, 30, 50, and 100) and message lengths (4, 8, and 12). VQEL consistently outperforms GS-ST and REINFORCE across all communication capacities, demonstrating robustness to both vocabulary size and message length.
Figure 8: Effect of communication capacity on Active Words (AW). While the AW of GS-ST and REINFORCE decreases as the vocabulary size increases, VQEL maintains an AW of 1 across all settings, indicating effective utilization of the available vocabulary regardless of communication capacity.
Figure 9: Effect of communication capacity on number of unique messages. VQEL consistently produces more unique messages than GS-ST and REINFORCE across all communication capacities, indicating a richer and more expressive communication protocol.
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

DatasetMethodSender UpdateACC ↑\uparrowAW ↑\uparrowTopSim ↑\uparrowH(C|M) ↓\downarrow
O BJECTSGS-ST-0.78±0.010.78_{\pm 0.01}0.93±0.060.93_{\pm 0.06}0.21±0.020.21_{\pm 0.02}1.04±0.031.04_{\pm 0.03}
REINFORCE-0.51±0.210.51_{\pm 0.21}0.47±0.120.47_{\pm 0.12}0.14±0.070.14_{\pm 0.07}2.21±1.282.21_{\pm 1.28}
VQEL-SP S{}_{\text{S}}-0.75±0.020.75_{\pm 0.02}1.00±0.001.00_{\pm 0.00}0.17±0.020.17_{\pm 0.02}0.30±0.030.30_{\pm 0.03}
VQEL-SP S{}_{\text{S}} +MPFrozen0.81±0.020.81_{\pm 0.02}1.00±0.001.00_{\pm 0.00}0.17±0.020.17_{\pm 0.02}0.30±0.030.30_{\pm 0.03}
VQEL-SP S{}_{\text{S}} +MPRL0.84±0.010.84_{\pm 0.01}1.00±0.001.00_{\pm 0.00}0.17±0.020.17_{\pm 0.02}0.29±0.030.29_{\pm 0.03}
VQEL-SP S{}_{\text{S}} +MPRL+Pres0.84±0.010.84_{\pm 0.01}1.00±0.001.00_{\pm 0.00}0.18±0.010.18_{\pm 0.01}0.28±0.030.28_{\pm 0.03}
Appendix
Table 5: Performance comparison across datasets and evaluation metrics for the sender self-play game using Euclidean distance in the codebook.
DatasetMethodReceiver UpdateACC ↑\uparrowAW ↑\uparrowTopSim ↑\uparrowH(C|M) ↓\downarrow
O BJECTSGS-ST-0.78±0.010.78_{\pm 0.01}0.93±0.060.93_{\pm 0.06}0.21±0.020.21_{\pm 0.02}1.04±0.031.04_{\pm 0.03}
REINFORCE-0.51±0.210.51_{\pm 0.21}0.47±0.120.47_{\pm 0.12}0.14±0.070.14_{\pm 0.07}2.21±1.282.21_{\pm 1.28}
VQEL-SP R{}_{\text{R}}-0.82±0.010.82_{\pm 0.01}1.00±0.001.00_{\pm 0.00}0.19±0.010.19_{\pm 0.01}0.12±0.020.12_{\pm 0.02}
VQEL-SP R{}_{\text{R}} +MPFrozen0.14±0.020.14_{\pm 0.02}0.70±0.100.70_{\pm 0.10}0.14±0.030.14_{\pm 0.03}3.97±0.263.97_{\pm 0.26}
VQEL-SP R{}_{\text{R}} +MPFine-tuned0.43±0.120.43_{\pm 0.12}1.00±0.001.00_{\pm 0.00}0.18±0.020.18_{\pm 0.02}2.17±0.402.17_{\pm 0.40}
S HAPEGS-ST-0.82±0.010.82_{\pm 0.01}1.00±0.001.00_{\pm 0.00}0.02±0.010.02_{\pm 0.01}1.21±0.111.21_{\pm 0.11}
Appendix
Table 6: Performance comparison across datasets and evaluation metrics for the receiver self-play game.
Figure 10: Token distributions conditioned on shape and color attributes for messages generated by VQEL with sender self-play. Each heatmap shows P(token∣attribute value,position)P(\text{token}\mid\text{attribute value},\text{position}) , where rows correspond to attribute values and columns correspond to message tokens.
Figure 11: Token distributions conditioned on shape and color attributes for messages generated by REINFORCE. Each heatmap shows P(token∣attribute value,position)P(\text{token}\mid\text{attribute value},\text{position}) , where rows correspond to attribute values and columns correspond to message tokens.
Figure 12: Token distributions conditioned on shape and color attributes for messages generated by GS-ST. Each heatmap shows P(token∣attribute value,position)P(\text{token}\mid\text{attribute value},\text{position}) , where rows correspond to attribute values and columns correspond to message tokens.
More in this topic

Emergent Communication

Emergent communication studies how agents develop shared signals or languages through interaction to coordinate, convey information, and build conventions.

See all 30 papers

Explore similar work

CardsList

  1. Emergent Language as an Approach to Conscious AI

    Jun 4, 2026Zengqing Wu, Chuan XiaoMachine ConsciousnessEmergent Communication

    University of Osaka

  2. Self-Emergence Agent Architecture:Behavior-Inertia HMM, Reflexive Metacognition,and Social-Contrastive Self-Modeling

    Sep 15, 2026Xiaoyang LiuSelf-Evolving AgentsCognitive Architectures for AI Agents

    Independent Researcher

  3. Vocabulary Dropout for Curriculum Diversity in LLM Co-Evolution

    Apr 3, 2026Jacob Dineen, Aswin RRV, Zhikun Xu +1Curriculum RLRL for Language Model Reasoning

    Arizona State University