From Parallels to Divergence: Language Models' Learning of Action Nominal Constructions

Iskar Deng; Shane Steinert-Threlkeld

cs.CLOct 15, 2026

From Parallels to Divergence: Language Models' Learning of Action Nominal Constructions

Authors: Iskar Deng, Shane Steinert-Threlkeld

Organizations: University of Washington

Abstract

Action nominal constructions (ANCs) express events in nominal form, with their argument marking following patterns found in clauses or ordinary noun phrases. In this paper, we investigate how different grammatical configurations affect language models' learning of ANCs and related constructions, and how these learning preferences compare with typological distributions in natural languages. We use a grammar-engineered method for synthetic corpus construction that combines shared semantic input from natural corpora with computational grammars. We construct 96 target languages varying in clause word order, noun-phrase word order, alignment, complement system, and ANC strategy. For each language, we train GPT-2-small models from scratch and evaluate them using targeted minimal pairs covering 34 grammatical phenomena. The results show typological parallels in basic clause configurations but divergences in complement systems and ANC strategies, with models also learning combinations rare or unattested in natural languages. This partial alignment suggests that natural language distributions may reflect both general learning pressures and functional demands not fully captured by model learning experiments.

Figures & tables

Type	Transitive ANC	Intransitive ANC	Example language
SENT	John- nom destruction city- acc	John- nom running	Korean
POSS-ACC	John- gen destruction city- acc	John- gen running	Selkup
ERG-POSS	John- instr destruction city- gen	John- gen running	Russian
NOMN	John- gen destruction city- obl	John- gen running	Icelandic

Table 1: Four major nominalization strategies illustrated with pseudo-examples. Case markers are simplified.

Parameter	Values
Clause word order	SOV / SVO / VOS
NP word order	GN / NG
Alignment	NOM-ACC / ERG-ABS
Complement system	Balancing / Deranking
ANC strategy	SENT / POSS-ACC / ERG-POSS / NOMN

Table 2: Five typological parameters and their values.

Original sentence	Pseudo-English	Generated form
Mary is sleeping.	Mary sleeps	Mary sleeps
I eat an apple.	I eats apple	I eats appleca
He wants to eat an apple.	He wants that he eats apple	He wants he eats appleca
I think Mary sleeps.	I thinks that mary sleeps	I thinks mary sleeps
He is looking for a book.	He lookfors book	He lookfors bookca
Why don’t you get up a collection?	You getups collectnmz	You getups collectingca

Table 3: Examples of pseudo-English and target-language realization. The target configuration uses SVO, GN, NOM-ACC, Balancing, and NOMN.

Figure 1: Overview of the corpus-generation pipeline.

ID	Phenomenon	Acc. (%)
1.1	Intransitive finite verb form	99.4
1.2	Intransitive S marker	87.7
1.3	Intransitive S–V order	100.0
1.4	Transitive finite verb form	99.7
1.5	Transitive A marker	96.6
1.6	Transitive P marker	97.9

Table 4: Targeted minimal-pair evaluation results. Accuracies are averaged across the 96 configurations and three random seeds. Darker shading indicates higher accuracy.

Panel of Figure 2 — Figure 2: Mean accuracy across three seeds for each grammatical configuration, averaged over (a) all 34 phenomena and (b) the 15 ANC-related phenomena. Darker shading indicates higher accuracy.

Clause WO	Alignment	IV frame	TV frame	Overlap	Model acc. (%)	Language count
SVO	NOM-ACC	NP V-s	NP V-s NP-ca	2	94.1	9
SVO	ERG-ABS	NP V-s	NP-ca V-s NP	0	89.3	1
SOV	NOM-ACC	NP V-s	NP NP-ca V-s	1	87.0	29
SOV	ERG-ABS	NP V-s	NP-ca NP V-s	0	84.6	16
VOS	NOM-ACC	V-s NP	V-s NP-ca NP	1	86.2	1
VOS	ERG-ABS	V-s NP	V-s NP NP-ca	2	89.0	2

Table 5: Finite intransitive (IV) and transitive (TV) frames, mean accuracy across all 34 phenomena, and counts of natural languages by word order and alignment. Overlap counts consecutive units with identical morphological forms shared from the beginning of the two frames.

Word-order profile	Bal.	Der.	Bal.	Der.
	Model acc. (%)		Language count
SVO/GN	92.2	91.2	5	1
SVO/NG	92.5	90.8	13	1
SOV/GN	86.2	88.4	7	24
SOV/NG	83.3	85.3	4	0

Table 6: Mean accuracy across all 34 phenomena and natural language counts by word-order profile and complement system.

ANC strategy	Bal.	Der.	Bal.	Der.
	Model acc. (%)		Language count
SENT	73.7	84.0	3	14
POSS-ACC	82.6	85.9	13	12
ERG-POSS	86.2	87.5	22	3
NOMN	88.2	89.7	14	0

Table 7: Mean accuracy across the 15 ANC-related phenomena and natural language counts by ANC strategy and complement system. Languages may occur in more than one ANC strategy category.

Appendix figures & tables41 assets

Supplementary material from the paper’s appendix.

Appendix

Variable	Meaning	Possible values	Example configuration
CLAUSE_WO	finite clause order of A/S, P, and V	SOV, SVO, VOS	SVO
FIN_S_MARK	dependent marking of S in finite intransitive clauses	0	0
FIN_A_MARK	dependent marking of A in finite transitive clauses	0, -ca	0
FIN_P_MARK	dependent marking of P in finite transitive clauses	0, -ca	-ca
FIN_V_MARK	verbal morphology on finite verbs	-s	-s
NP_WO	order of genitive dependent and nominal head	GN, NG	GN

Appendix

Table 8: Possible morphosyntactic settings for the target languages and an example configuration with SVO, GN, NOM-ACC, Balancing, and NOMN. Here, 0 indicates zero marking.

Alignment	S	A	P
NOM-ACC	0	0	-ca
ERG-ABS	0	-ca	0

Appendix

Table 9: Finite-clause argument marking under the two alignment systems.

System	Main clause	ANC	Complement
Balancing	-s	-ing	-s
Deranking	-s	-ing	-ing

Appendix

Table 10: Verb-form profiles under the two complement-system settings.

ANC strategy	S	A	P
SENT	$S_{\text{fin}}$	$A_{\text{fin}}$	$P_{\text{fin}}$
POSS-ACC	-ge	-ge	$P_{\text{fin}}$
ERG-POSS	-ge	-ob	-ge
NOMN	-ge	-ge	-ob

Appendix

Table 11: Dependent-marking profiles of the four ANC strategies. The subscript fin indicates the corresponding finite-clause marker.

Word Order	SENT	POSS-ACC	ERG-POSS	NOMN
SOV + GN	SV/APV	SV/APV	SV/APV	SV/APV
SVO + GN	SV/AVP	SV/AVP	SV/PVA	SV/AVP
VOS + GN	VS/VPA	SV/AVP	SV/PVA	SV/AVP
SOV + NG	SV/APV	VS/PVA	VS/AVP	VS/PVA
SVO + NG	SV/AVP	VS/VPA	VS/VPA	VS/VPA
VOS + NG	VS/VPA	VS/VPA	VS/VPA	VS/VPA

Appendix

Table 12: ANC-internal word orders determined by the interaction of clause word order, NP word order, and ANC strategy.

English Parsing		Shared Lexicon
Category	Count	Category	Count
Parsed sentences	12,441,836	Nouns	225,939
Retained sentences	4,095,236	Intransitive verbs	12,252
iv constructions	883,158	Transitive verbs	35,312
tv constructions	1,812,451	Clausal-complement verbs	5,220
cv constructions	674,525	Action-nominal candidates	6,298

Appendix

Table 13: Summary of English parsing, shared-lexicon construction, and action-nominal distribution for the training split.

Parameter	Value
Model configuration	GPT-2-small
Tokenizer	GPT-2 tokenizer
Initialization	Random
Seeds	42, 43, 44
Block size	32
Training steps	70,000

Appendix

Table 14: Training configuration for each target-language model.

Figure 4: Accuracy and BAD parse rate for 1.1 Intransitive finite verb form.

Figure 5: Accuracy and BAD parse rate for 1.2 Intransitive S marker.

Figure 6: Accuracy and BAD parse rate for 1.3 Intransitive S–V order.

Figure 7: Accuracy and BAD parse rate for 1.4 Transitive finite verb form.

Figure 8: Accuracy and BAD parse rate for 1.5 Transitive A marker.

Figure 9: Accuracy and BAD parse rate for 1.6 Transitive P marker.

Figure 10: Accuracy and BAD parse rate for 1.7 Transitive A–V order.

Figure 11: Accuracy and BAD parse rate for 1.8 Transitive V–P order.

Figure 12: Accuracy and BAD parse rate for 1.9 Intransitive V valency.

Figure 13: Accuracy and BAD parse rate for 1.10 Transitive V valency.

Figure 14: Accuracy and BAD parse rate for 2.1 Intransitive genitive marker.

Figure 15: Accuracy and BAD parse rate for 2.2 Transitive genitive marker.

Figure 16: Accuracy and BAD parse rate for 2.3 Genitive possessor–head order.

Figure 17: Accuracy and BAD parse rate for 3.1 Clausal complement verb form.

Figure 18: Accuracy and BAD parse rate for 3.2 Clausal complement S marker.

Figure 19: Accuracy and BAD parse rate for 3.3 Clausal complement A marker.

Figure 20: Accuracy and BAD parse rate for 3.4 Clausal complement P marker.

Figure 21: Accuracy and BAD parse rate for 3.5 Clausal complement–V order.

Figure 22: Accuracy and BAD parse rate for 3.6 Clausal complement CV valency.

Figure 23: Accuracy and BAD parse rate for 4.1 ANC verb form.

Figure 24: Accuracy and BAD parse rate for 4.2 ANC S marker.

Figure 25: Accuracy and BAD parse rate for 4.3 ANC A marker.

Figure 26: Accuracy and BAD parse rate for 4.4 ANC P marker.

Figure 27: Accuracy and BAD parse rate for 4.5 ANC S–V order.

Figure 28: Accuracy and BAD parse rate for 4.6 ANC A–V order.

Figure 29: Accuracy and BAD parse rate for 4.7 ANC V–P order.

Figure 30: Accuracy and BAD parse rate for 5.1 ANC external S marker.

Figure 31: Accuracy and BAD parse rate for 5.2 ANC external A marker.

Figure 32: Accuracy and BAD parse rate for 5.3 ANC external P marker.

Figure 33: Accuracy and BAD parse rate for 5.4 ANC external genitive marker.

Figure 34: Accuracy and BAD parse rate for 6.1 ANC omitted S.

Figure 35: Accuracy and BAD parse rate for 6.2 ANC omitted A.

Figure 36: Accuracy and BAD parse rate for 6.3 ANC omitted P.

Figure 37: Accuracy and BAD parse rate for 6.4 ANC omitted A+P.

Syntactic Generalization in Language Models

See all 13 papers

What Kind of Language is Easy to Language-Model Under Curriculum Learning?
Apr 29, 2026Nadine El-Naggar, Tatsuki Kuribayashi, Ted BriscoeInductive Bias Linguistic Bias in Language Models
Mohamed bin Zayed University of Artificial Intelligence
Differences in Typological Alignment in Language Models' Treatment of Differential Argument Marking
Feb 19, 2026Iskar Deng, Nathalia Xu, Shane Steinert-Threlkeld
University of Washington
Neural Language Models Learn the Contextual Distributions of Dependency Structures: a statistical learning theory to compositionality
Sep 28, 2026Wang Bojun, Junjie Chen, Holly Jenkins +1Language Modeling Computational Psycholinguistics
Department of Education, University of Oxford, Oxford, United Kingdom · Department of Statistics, University of Oxford, Oxford, United Kingdom

Apr 29, 2026J/K move · Enter open · S save

What Kind of Language is Easy to Language-Model Under Curriculum Learning?

Nadine El-Naggar, Tatsuki Kuribayashi, Ted Briscoe

Mohamed bin Zayed University of Artificial Intelligence

Inductive Bias Linguistic Bias in Language Models Curriculum Learning Linguistic Typology LLM Training

Open paper AskPDF arXiv

Many of the thousands of attested languages share common configurations of features, creating a spectrum from typologically very rare (e.g., object-verb-subject word order) or impossible languages to very common combinations of features (e.g., subject-object-verb word order). One central question is under what conditions such typological tendencies can be predicted, and specifically whether the learning bias of language models (LMs) is sufficient to reproduce such patterns. In this study, we add one dimensionality to such analysis -- the learning scenario for LMs -- to explore its interaction with the inductive bias of LMs. Specifically, as a first study, we examine the effect of curriculum learning (CL), as a developmentally motivated learning scenario, i.e., starting with simpler sentences rather than randomly-ordered input. We expand existing LM-based exploration (El-Naggar et al., 2025a,b) with a simple CL variant and find that CL substantially impacts the apparent inductive bias of LMs.

Abstract

Syntactic Generalization in Language Models

Explore similar work

What Kind of Language is Easy to Language-Model Under Curriculum Learning?

Differences in Typological Alignment in Language Models' Treatment of Differential Argument Marking

Neural Language Models Learn the Contextual Distributions of Dependency Structures: a statistical learning theory to compositionality