cs.CLSep 4, 2026

Self-Supervised Lexical Representation Learning for Fast, Large-Scale Phylogenetic Inference

Authors: Tim Wientzek

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Computational phylogenetics has become an essential tool in historical linguistics, yet its application at a global scale remains constrained by two factors: the labor-intensive manual annotation of cognacy judgments required for character-based methods and the substantial computational cost of inference on large datasets. This paper introduces a fully self-supervised contrastive learning framework that learns lexical representations directly from raw IPA-transcribed wordlists, without requiring cognacy annotations, alignments, or additional expert input. The model employs a dual contrastive objective: a word-level loss that organizes phonetically similar forms into a coherent space, and an auxiliary language-level loss that encourages the lexical space to reflect broader phonological properties of languages. From the resulting word representations, pairwise language distances are derived and used to infer a global phylogenetic tree of 3,399 language varieties. The inferred tree achieves a generalized quartet distance (GQD) to the Glottolog reference tree competitive with multiple baselines, while requiring only minutes of computation on a standard notebook GPU. Furthermore, the same representations capture diachronic concept stability: variance in pairwise distances across languages yields stability rankings that correlate significantly with established rankings. Ablation studies confirm that both the language-level objective and the use of phonetic feature vectors improved the inferred trees topology with regards to GQD. The framework thus provides a computationally efficient and fully automatic alternative for large-scale phylogenetic inference and offers a unified representation supporting downstream analyses at both the language and concept level.

Figures & tables

Figure 1: Schematic model overview. a) shows the internal architecture of both the word and language encoder (adapted from Wientzek (2025a) ). For the word encoder, each batch item xx , a word, is passed through the model twice to get two augmented views, which are then processed by the encoder, pooled and passed through a projection head. The model objective (loss function) maximises agreement between the two projected augmented views of an item (attract), while minimising agreement between these and the projections from all other items (repel). For the language encoder, xx constitutes the concatenated lexical representations with the augmentations applied in the word encoder. b) shows the overall architecture. Batches of word forms are encoded into lexical representations and concatenated for the language encoder. Language embeddings are added to each position of the sequence. Both the language encoder and the evaluation operate on the outputs of the word encoder, before the projection layer.
ParameterValue
Embedding dimension256
Encoder layers1
Encoder heads4
Learning rate0.001
Number of languages32
Number of concepts30
Table 1: Model hyperparameters.
Figure 2: Representative phylogenetic tree inferred from the learned lexical representations, rooted with MAD and ultrametricised via penalised likelihood. The tree was visualised with the R package ggtree ( Yu et al., 2017 ) .
MethodGQDF1
LDN0.04680.9639
pHMM0.06920.9636
MSA ( Jäger, 2026 )0.04330.959
DualCWE (own)0.0346 (0.003)0.955
Table 2: Generalised quartet distance (GQD) of trees inferred from normalised Levenshtein distance (LDN), a pair-Hidden Markov Model (pHMM), multiple sequence alignment (MSA) and lexical representations (DualCWE). LDN, pHMM and DualCWE are distance-based methods with trees inferred via BIONJ, MSA is a character-based method using maximum likelihood inference.
RankingOverlapSpearman rp-value
Holman et al. (2008) +69 (100)−0.4907\mathbf{-0.4907}0.0000\mathbf{0.0000}
Pagel et al. (2007) -97 (200)0.3172\mathbf{0.3172}0.0015\mathbf{0.0015}
Petroni and Serva (2011) +61 (100)−0.2801\mathbf{-0.2801}0.0288\mathbf{0.0288}
Peust (2013) -101 (185)0.3449\mathbf{0.3449}0.0004\mathbf{0.0004}
Rama and Borin (2014) -69 (100)0.7204\mathbf{0.7204}0.0000\mathbf{0.0000}
Tadmor (2009) +59 (100)−0.1980-0.19800.13270.1327
Table 3: Spearman rank correlations between the model-derived concept stability ranking (with higher variance resulting in ranks closer to 1) and established stability rankings. Superscript + indicates rankings where higher values correspond to greater stability; - indicates rankings where lower values correspond to greater stability. Column “Overlap” shows the number of concepts from the ranking that overlap with the model’s 122 ranked concepts (total size of the ranking in parentheses). Significant correlations (p << 0.05) are shown in bold.
ConceptAvg. RankHolmanPagelP & SPeustRamainfTadmorTad. (rep.)Mean Disagr.
FATHER1.4-562634-27-10549.6
MOTHER1.6-59148-20 8442.4
I3.42-2-1-211062.29
AND3.6 79----3-6547.0
HORSE6.2-----64-11790.5
FIVE6.2--420-4-50-9631.6
Table 4: Average rank of the 20 highest variance concepts across the 5 seeded runs (“Avg. Rank”), with per-ranking deviations from the resulting placement. A negative deviation indicates the concept is ranked higher (more stable) in that specific ranking; a positive deviation indicates it is ranked lower (less stable). The three WOLD scores are omitted for readability and because they are not direct stability rankings; a weighted composition of these scores is captured in the two Tadmor rankings.
Figure 3: Distribution of cosine distances for representations of concepts of different ranks from a single model run. The ranks are based on the average rank across seeds.
SettingGQD
Original0.0346
No language encoder0.044
No language encoder, 4 epochs, 0.05 temp.0.071
No language embeddings0.054
ASJP encoding0.053
IPA encoding0.049
Table 5: Ablation results showing GQD for different model configurations.
More in this topic

Phylogenetic Inference

Phylogenetic inference reconstructs evolutionary relationships and divergence histories from genetic, morphological, linguistic, or other inherited traits.

See all 12 papers

Explore similar work

CardsList
Mar 7, 2026

Scaling Self-Supervised Speech Models Uncovers Deep Linguistic Relationships: Evidence from the Pacific Cluster

Similarities between language representations derived from Self-Supervised Speech Models (S3Ms) have been observed to primarily reflect geographic proximity or surface typological similarities driven by recent expansion or contact, potentially missing deeper genealogical signals. We investigate how scaling an S3M-based language identification system from 126 to 4,017 languages reshapes this topology, and find a non-linear effect: phylogenetic recovery stays flat up to the 1K scale, but the 4K model undergoes a qualitative shift, resolving both clear lineages and long-term linguistic contact. Most strikingly, a robust Pacific macro-cluster emerges, grouping genealogically unrelated Papuan, Oceanic, and Australian languages, and we trace its driver to a concentrated encoding that captures shared acoustic signatures such as global energy dynamics. These results suggest that massive S3Ms internalize multiple layers of language history, offering a promising perspective for computational phylogenetics and the study of language contact.

Minu Kim, Hoirin Kim, David R. Mortensen

Thomas Lord Department of Computer Science, University of Southern California, USA · School of Electrical Engineering, KAIST, Republic of Korea · Language Technologies Institute, Carnegie Mellon University, USA

Self-Supervised Speech Representation LearningLanguage IdentificationLinguistic Typology
AskPDF
Oct 13, 2026

Deep Generative Models for Phylogenetic Inference with Complex Evolutionary Processes

Inference of phylogenetic trees from sequences plays a central role in understanding evolutionary relationships, with applications ranging from tracking pathogen spread to reconstructing the history of life. Conventionally, practitioners obtain posteriors over trees using likelihood-based methods like MCMC. However, likelihoods are only available for simple evolutionary models with restrictive assumptions. For realistic evolutionary models, inference using conventional methods is prohibitively expensive, inaccurate, or even impossible. In practice, practitioners settle for using as flexible a conventional method as possible, even though they expect it is misspecified. In principle, we could use simulation-based inference: simulate tree and sequence data from an evolutionary model and train a neural network to predict the posterior over trees given sequences. However it is a mathematical challenge to represent and train complex posterior distributions over trees. Here we present a flexible family of conditional generative models over trees. Our models iteratively build trees using three natural paradigms -- top-down, middle-out, or bottom-up -- and are trained using a discrete diffusion framework. For real and synthetic datasets of up to 6565 taxa and >105>10^5 tokens, our models recover MCMC posteriors much more accurately than the previous state-of-the-art simulation-based method Phyloformer 2. Finally, we compare to IQ-Tree, a popular flexible-but-misspecified conventional method. Our models match its tree estimates on simple data, but on more realistic data, our method is reliably more accurate.

Ethan Baron, Alan N. Amin, Andrew Gordon Wilson

New York University

Phylogenetic InferenceGenerative ModelingSimulation-Based InferenceConditional Diffusion Models
AskPDF
Apr 7, 2026

A Modular Architecture for Typologically Controlled Lexicon Generation

Constructing artificial lexicons that are pronounceable, typologically plausible, and semantically structured remains an open challenge in computational linguistics. Existing conlang generators either lack formal phonotactic guarantees or delegate generation to opaque, non-reproducible LLM-based pipelines. We propose a modular framework that samples phoneme inventories from PHOIBLE, generates word forms under interchangeable phonological grammars (deterministic, OT, and MaxEnt), and assigns meanings via a Swadesh--Leipzig--Jakarta ontology with explicit form--meaning alignment. Evaluation on character nn-gram perplexity, log-likelihood, and KL divergence against PHOIBLE across lexicon sizes of 100-5,000 forms shows that probabilistic grammars consistently outperform deterministic and random baselines on both phonotactic coherence and typological realism.

Sankalp Tattwadarshi Swain, Dhruv Kumar

Birla Institute of Technology and Science, Pilani Pilani, Rajasthan 333031, India

Linguistic Typology
AskPDF