Cultural Evolution Drives the Emergence of Conventions in Neural Emergent Communication
Organizations: Artificial Intelligence Laboratory, Vrije Universiteit Brussel, Belgium · Faculté d’Informatique, Université de Namur, Belgium
Abstract
Recent studies of language evolution with neural agents often use models where agents can either speak or listen, where there are only two agents in the population, or where agents can share internal representations. These choices prevent the study of how language emerges through cultural evolution, i.e. through the cumulative adaptation of individual behaviours to fit the communicative needs of the community. We tackle this issue with a new model that trains populations of bi-directional neural agents in a fully distributed fashion to solve a referential game. We demonstrate the effectiveness of this approach on multiple image datasets and show that it is able to scale to large populations. We also show that more conventional languages are easier to learn, providing a direct advantage for cultural transmission. Overall, this work provides a novel approach to neural emergent communication that opens up new directions for evolutionary linguistics.
Figures & tables
| Method | Success (%) | Conv. (%) | |
| ColorReference ( , ) | |||
| Ours | 99.13 ±0.15 | 93.18 ±0.59 | 8.8 ±0.42 |
| No-Imitation | 99.54 ±0.05 | 76.46 ±1.72 | 20.5 ±1.51 |
| Similarity | 85.46 ±0.78 | 52.4 ±9.17 | 21.9 ±4.25 |
| S-L | 97.36 ±0.35 | 5.84 ±2.6 | 30.4 ±2.41 |
| ManyNames ( , ) | |||
| Hyperparameter | Method | |||
| Ours | No-Imitation | Speaker-Listener | Similarity | |
| Learning rate | 0.0001 | |||
| Vocabulary size | 100 | |||
| Hidden dimension | 64 | |||
| Embedding dimension | 64 | |||
| LogSumExp temperature | 1.0 | 1.0 | 1.0 | - |
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
| Success | Conventionality | ||
| Bi-directional | 0 (No-Imitation) | 98.63 ±0.28 | 68.42 ±1.8 |
| 0.2 (Ours) | 97.41 ±0.66 | 82.74 ±1.17 | |
| 1 | 89.25 ±1.38 | 87.3 ±1.09 | |
| Speaker-Listener | 0 | 88.54 ±1.46 | 13.83 ±2.7 |
| 0.2 | 86.14 ±1.01 | 18.94 ±2.25 | |
| 1 | 76.62 ±2.22 | 33.83 ±5.8 |
| Total interactions | Success | Conv. | |
| 10 | 98.97 ±0.19 | 91.23 ±0.50 | |
| 100 | 97.40 ±1.11 | 91.62 ±0.80 | |
| 1000 | 95.89 ±0.16 | 92.01 ±0.28 |
Emergent Communication
Emergent communication studies how agents develop shared signals or languages through interaction to coordinate, convey information, and build conventions.
See all 32 papers