Patterns of Priming in Production: Lexical, Semantic and Structural Alignment in Language Model Generation

Giulia Pucci; Ruizhe Li; Arabella Sinclair

cs.CLSep 3, 2026

Patterns of Priming in Production: Lexical, Semantic and Structural Alignment in Language Model Generation

Authors: Giulia Pucci, Ruizhe Li, Arabella Sinclair

Organizations: School of Natural and Computing Science, University of Aberdeen · School of Computer Science, University of Birmingham · Department of Information Studies, University College London

Abstract

This paper investigates structural priming in language model (LM) production, examining how preceding structural context influences sentence completion. While prior work has demonstrated priming effects in comprehension of structural alternations, it remained unclear whether these persist in production, where, when generating, an LM samples from many possible continuations at each step. We address this question through a series of controlled sentence-completion experiments on dative constructions. In line with prior work, we find that LMs are susceptible to structural priming, particularly in sentences that are semantically coherent. In terms of priming magnitude, we find that while there is a greater relative increase of double-object datives against our baselines, in line with inverse frequency effects, there is a larger absolute increase in prepositional-objects, the more frequently produced construction. Finally, we not only observe that structural priming is boosted by lexico-semantic coherence, but that structurally primed completions display greater levels of lexico-semantic repetition. Taken together, our evidence supports the view that structural priming in LMs operates across multiple levels of linguistic representation, facilitating, and facilitated by syntactic, lexical, and semantic alignment. Code: https://github.com/the-context-lab/primedproduction.

Figures & tables

Condition	Example prompt
	prime target stem
No context	The student sent the
Non-dative	Roger is lucky. The student sent the
Unrelated	…gave a toy to a man. The student sent the
Coherence	…gave a toy to a man . The man sent the

Table 1: Sentence-completion examples. In the No context condition, the target stem with a dative verb is presented without a preceding prime. For Non-dative the prime is any non-dative construction, for Unrelated , the prime and target are semantically independent, with no lexical overlap and

<0.5

semantic similarity, for Coherence the target reuses last animate noun from the prime, creating local discourse coherence.

Model	Test Set	Accuracy	Precision			Recall			F1-score
Model			PO	DO	Non-dative	PO	DO	Non-dative	PO	DO	Non-dative
Logistic Regression	PrimeLMDative+	0.96	0.99	0.98	0.91	1.00	0.88	0.98	1.00	0.93	0.95
	Gemma-2-2B	0.56	0.67	0.20	0.85	0.67	0.67	0.50	0.67	0.30	0.63
	EuroLLM-1.7B	0.59	0.56	0.09	0.72	0.48	0.17	0.69	0.51	0.12	0.71
	Phi-2-2.7B	0.52	0.43	0.21	0.92	0.70	0.79	0.45	0.53	0.34	0.60
BERT	PrimeLMDative+	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00	1.00

Table 2: Classifier comparison on the in-domain PrimeLM+ test set and on hand-annotated generated completions from different language models. We compare the final BERT classifier against a linguistically motivated Logistic Regression baseline using hand-engineered syntactic features. Accuracy, precision, recall, and F1-scores are reported for each class (PO, DO, and Non-dative). BERT shows more robust performance across generated test sets and is therefore used for structural labelling in the main experiments. We provide additional details in Section D.1 .

Panel of Figure 1 — Figure 1: Structure-matching completion rates for DO (left) and PO (right) structures under four conditions: No Context , Non-dative , Dative Non-match , and Primed . The Dative Non-match condition contains a dative prime whose structure does not match the target structure. Dotted horizontal lines indicate the average rate across models for the Dative Mon-match and Primed conditions. Across both target structures, the matching primed condition produces the highest rates of structure-matching completions, indicating structural priming beyond both non-dative and non-matching dative controls. Full results are reported in Appendix I , Table 13 .

Panel of Figure 2 — Figure 2: Structure-matching completion rates for DO (left) and PO (right) targets in the unrelated and coherence conditions. For both target structures, the coherence condition tends to produce higher rates of structure-matching completions than the unrelated condition, showing that semantic coherence amplifies structural priming. Lighter bars mark PO comparisons for which the coherence–unrelated difference is not significant after Benjamini–Hochberg FDR correction. Full results are reported in Appendix I , Table 14 .

Figure 3: Priming strength relative to No context, Non dative and Dative (non-matching) baselines measured by i) absolute difference and ii) relative increase in production rate of PO and DO constructions. Results averaged across LMs, * indicates significant differences for t-tests comparing PO and DO priming strength.

Metric	Prim.	Phi-2			Gemma-2-2B			Gemma-2-9B			EuroLLM-1.7B			EuroLLM-9B			Qwen-3-1.7B			Qwen-3-8B
		$\overline{P}$	$P$	t	$\overline{P}$	$P$	t	$\overline{P}$	$P$	t	$\overline{P}$	$P$	t	$\overline{P}$	$P$	t	$\overline{P}$	$P$	t	$\overline{P}$	$P$	t
MiniLM	U	0.26	0.36	8.78	0.28	0.38	8.65	0.27	0.36	8.18	0.23	0.35	11.20	0.23	0.33	9.80	0.27	0.35	6.96	0.26	0.35	7.86
	C	0.36	0.48	12.54	0.38	0.49	13.22	0.39	0.53	15.05	0.32	0.44	13.29	0.36	0.48	14.64	0.36	0.46	10.68	0.37	0.48	12.35
MPNet	U	0.26	0.35	8.40	0.27	0.37	8.52	0.26	0.34	7.76	0.23	0.34	11.03	0.23	0.32	9.41	0.26	0.34	6.53	0.26	0.33	6.83
	C	0.34	0.44	11.67	0.36	0.46	12.05	0.37	0.49	13.52	0.31	0.42	12.56	0.34	0.44	12.80	0.35	0.43	9.68	0.35	0.45	11.09
BERTScore F1	U	0.21	0.26	8.55	0.18	0.22	6.14	0.18	0.23	7.76	0.15	0.20	7.08	0.17	0.21	6.24	0.15	0.21	8.42	0.16	0.21	7.49

Table 3: RQ3 results. Mean similarity scores for non-primed (

\overline{P}

) and primed (

P

) completions in the unrelated (U) and coherent (C) conditions. Reported t -values come from Welch’s t -tests comparing

\overline{P}

and

P

completions within each context. The same results are shown in Figure 7 . All comparisons are statistically significant after Benjamini–Hochberg FDR correction (

p<.001

in all cases), except for the EuroLLM-9B in coherence . In the Appendix we report the breaking down for PO (see Table 15 ), and DO (see Table 16 ). The results show a less consistent effect for DO in the Unrelated condition.

Figure 4: Lexico-semantic alignment boost in primed completions across U nrelated and C oherent settings. Results are an average across LMs. * Indicates independent t-tests comparing differences are significant.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

Augmentation	PO Sentence	DO Sentence
Original	The teacher sent the book to the student.	She gave her mother a present.
Vocabulary	Jasmine sent the book to the student.	Stacy gave her mother a present.
Temporal	A couple of days ago, the teacher sent the book to the student.	Last month, she gave her mother a present.
Indirect Speech	Joana told Susan that the teacher sent the book to the student.	Angela said that she gave her mother a present.
Negation	The teacher did not send the book to the students.	She didn’t give her mother a present.

Appendix

Table 4: Examples of PO and DO sentences and their augmentations (Vocabulary, Temporal, Indirect Speech, and Negation).

Augmentation Templates
Temporal Complement
a couple of days ago, a few weeks ago, a few hours ago, a long time ago, at dawn, at dusk, early this morning, five minutes ago, last autumn, last summer, last spring, last winter, last week, last month, last year, yesterday.
Indirect Speech verbs
told, said, saw, heard, found out, read, wanted to say, wanted to tell, wanted to show, could have said, could have told, could have shown, should have said, should have told, should have shown
Vocabulary: ROC Names
Abe, Allen, Allie, Andy, Angela, Anita, Ann, Anna, Annie, Arthur, Barry, Bea, Ben, Beverly, Bill, Billy, Bob, Bobby, Bonnie, Boston, Boy, Brad, Cam, Cara, Carl, Carly, Charles, Chris, Clint, Cora, Crystal, Dan, Daniel, David, Diana, Diane, Eliza, Ella, Ellie, Francine, Francis, Fred, Gary, Genie, Glenda, Greg, Hank, Heather, Jaime, Jake, James, Jane, Janet, Janie, Jasmin, Jasmine, Jason, Jean, Jeff, Jenn, Jenny, Jessica, Jill, Jo, Joan, Joanne, Jody, Joe, John, Johnnie, Joshua, Josie, Jude, Julie, July, June, Justin, Karl, Kaya, Kayla, Kelly, Ken, Kerry, Kira, Kyle, Lance, Larry, Laura, Leah, Lenny, Linda, Lisa, Lola, Lucy, Ludo, Luke, Marc, Marcus, Mario, Mark, Marlene, Martin, Mary, Matt, Meg, Megan, Melody, Meredith, Mia, Mike, Mikey, Naomi, Ned, Nick, Nicole, Niels, Noah, Oscar, Phil, Richard, Robbie, Roger, Rosie, Sam, Sammy, Sarah, Shane, Shannon, Shawn, Sonia, Sophia, Stacy, Stephanie, Stuart, Sue, Susan, Theo, Theodore, Tim, Timmy, Tom, Vanessa, Vera, Vivian, Walter, Wendy,

Appendix

Table 5: Augmentation Templates. Complements for Temporal augmentation, verbs for Indirect Speech augmentation,

Method	Accuracy
spacy’s Senticizer	67%
hybrid (cleaning + spacy)	70%
ours	98%

Appendix

Table 6: Comparison of accuracy across sentence extraction methods.

Condition	Prime Structure	Prime sentence	Target stem	Completion	Pred.
No Context	–	–	The student sent a	letter.	non-dative
Unrelated	Non-dative	The princess opened the old book.	The student sent a	message to the professor.	PO
	PO	The princess gave a book to the man.	The student sent a	book to the teacher.	PO
	DO	The princess gave the man a book.	The student sent a	letter to his teacher.	PO
Coherence	Non-dative	The princess opened the old book.	They sent a	letter to the prince.	PO
	PO	The princess gave a book to the man.	The man gave	a book to a girl.	PO

Appendix

Table 7: Illustrative model completions across prompt conditions. Boldface in the Prime Structure and Pred. columns marks cases where the generated completion matches the structure of the prime, indicating structural priming. Boldface within sentences highlights lexical material repeated between the prime and the generated completion. The examples show that structurally primed completions can also contain greater lexical overlap with the prime, especially in coherent contexts.

Comparison	Structure	Phi-2	Gemma2-2B	Gemma2-9B	EuroLLM-1.7B	EuroLLM-9B	Qwen3-1.7B	Qwen3-8B
Unrelated vs. No Prime	PO	0.38***	0.47***	0.57***	0.56***	0.49***	0.41***	0.52***
Unrelated vs. No Prime	DO	0.15***	0.29***	0.47***	0.24***	0.34***	0.21***	0.29***
Unrelated vs. Non-Dative	PO	0.46***	0.47***	0.51***	0.54***	0.51***	0.43***	0.56***
Unrelated vs. Non-Dative	DO	0.21***	0.26***	0.40***	0.21***	0.31***	0.29***	0.28***
Unrelated vs. Dative Non-match	PO	0.16***	0.09*	0.17***	0.22***	0.23***	0.09*	0.26***
Unrelated vs. Dative Non-match	DO	0.12**	0.15**	0.28***	0.13**	0.23***	0.17***	0.24***

Appendix

Table 8: Cohen’s

h

effect sizes for the RQ1/RQ2 two-proportion

z

-tests, across all seven models. Stars denote Benjamini–Hochberg FDR-corrected significance (

{}^{*}p<.05

{}^{**}p<.01

{}^{***}p<.001

Metric	Context	Phi-2	Gemma2-2B	Gemma2-9B	EuroLLM-1.7B	EuroLLM-9B	Qwen3-1.7B	Qwen3-8B
MiniLM	Unrelated	0.60***	0.55***	0.48***	0.70***	0.57***	0.44***	0.47***
MiniLM	Coherence	0.64***	0.65***	0.77***	0.69***	0.70***	0.55***	0.60***
MPNet	Unrelated	0.55***	0.53***	0.45***	0.68***	0.54***	0.41***	0.40***
MPNet	Coherence	0.59***	0.60***	0.68***	0.64***	0.61***	0.49***	0.53***
BERTScore F1	Unrelated	0.53***	0.38***	0.44***	0.41***	0.34***	0.49***	0.43***
BERTScore F1	Coherence	0.48***	0.44***	0.59***	0.40***	0.44***	0.36***	0.46***

Appendix

Table 9: Cohen’s

d

effect sizes for the RQ3 Welch

t

-tests (primed vs. non-primed completions) on lexico-semantic similarity metrics, across all seven models. Stars denote Benjamini–Hochberg FDR-corrected significance (

{}^{*}p<.05

{}^{**}p<.01

{}^{***}p<.001

Panel of Figure 5 — Figure 5: Priming strength across models and baseline contexts, measured as (a) the absolute percentage-point difference and (b) the relative percentage increase.

	Baseline	$t$	$p$
Percent difference	No Context	8.516	$<.0001$
	Non-Dative	11.420	$<.0001$
	Dative	2.117	.0599
Percent increase	No Context	-1.005	.3500
	Non-Dative	-0.919	.3887
	Dative	-4.440	.0024

Appendix

Table 10: Priming Strength: Independent-samples

t

-tests comparing priming strength for PO and DO constructions under each baseline condition, using absolute percentage-point difference and relative percentage increase as effect measures.

Group	Struct.	Mean Prime	Mean Generated	% Same
$\neg$ Primed	DO	7.00	11.93 ∗	5.42
$\neg$ Primed	PO	8.00	12.60 ∗	8.75
Primed	DO	7.00	10.20 ∗	37.37
Primed	PO	8.00	12.40 ∗	26.17

Appendix

Table 11: Length Analysis : Average prime length, generated-completion length, and percentage of same-length completions, by priming group. ∗ denotes that the mean generated completion length is significantly larger than the average prime length with

p<.001

Figure 6: Lexico-semantic alignment boost in primed completions across U nrelated and C oherent settings. Results are an average across LMs. Independent t-tests compare differences, * indicates $p<0.05$ .

Metric	Mean U	Mean C	$t$	$p$
Jaccard	0.0367	0.0933	5.126	.0083
BLEU	1.4533	4.0700	3.998	.0327
BERTScore F1	0.0433	0.0333	-0.671	.5512
MPNet	0.0567	0.0767	0.616	.5825
MiniLM	0.0700	0.0667	-0.100	.9252

Appendix

Table 12: Lexico-semantic Alignment: Comparison of lexical and semantic similarity between unrelated (U) and coherent (C) prime–target pairs. Comparisons performed grouping LM results.

Model	Target	$\overline{P}$			$P$	Contrasts
Model	Target	$\neg$ N	$\neg$ O	D	P	$\Delta^{N}$	$z^{N}$	$\Delta^{O}$	$z^{O}$	$\Delta^{D}$	$z^{D}$
EuroLLM-1.7B	PO	7.20	7.81	18.40	27.60	20.40	16.37	19.79	14.11	9.20	4.89
EuroLLM-1.7B	DO	2.12	2.49	3.90	6.90	4.78	7.04	4.41	5.66	3.00	2.97
EuroLLM-9B	PO	10.47	9.70	19.70	29.40	18.93	13.97	19.70	13.40	9.70	5.04
EuroLLM-9B	DO	3.16	3.60	5.40	11.60	8.44	9.95	8.00	8.26	6.20	4.97
Gemma-2-2B	PO	7.59	7.53	20.30	24.20	16.61	13.64	16.67	12.38	3.90	2.10

Appendix

Table 13: RQ1. Presence of priming on the full set of generations: summary of PO and DO completions after three

\overline{P}

contexts, and a structure-matching prime.

\neg

N denotes the no-context baseline;

\neg

O denotes the non-dative prime baseline; D denotes the dative non-matching baseline, i.e., PO completions after DO primes or DO completions after PO primes.

P

denotes the matching dative-prime condition, i.e., PO completions after PO primes or DO completions after DO primes.

\Delta^{N}

\Delta^{O}

, and

\Delta^{D}

report the difference between

P

and each baseline;

z^{N}

z^{O}

, and

z^{D}

report the corresponding two-proportion z-test statistics. All percentage columns report structure-matching completions. All results are significant after Benjamini–Hochberg FDR correction at

\alpha=.05

. This table shows full numerical results corresponding to Figure 1 .

	% PO priming					% DO priming
Model	Coher.	Unrel.	$\Delta^{C-U}$	$z$	$p_{\mathrm{BH}}$	Coher.	Unrel.	$\Delta^{C-U}$	$z$	$p_{\mathrm{BH}}$
EuroLLM-1.7B	31.80	27.60	4.20	2.06	$.043$	17.20	6.90	10.30	7.07	$<.001$
EuroLLM-9B	38.10	29.40	8.70	4.11	$<.001$	26.00	11.60	14.40	8.24	$<.001$
Gemma-2-2B	32.90	24.20	8.70	4.31	$<.001$	17.60	7.50	10.10	6.82	$<.001$
Gemma-2-9B	31.10	28.90	2.20	1.07	$.283$	21.40	11.50	9.90	5.97	$<.001$
Phi-2-2.7B	32.40	24.70	7.70	3.81	$<.001$	18.50	6.90	11.60	7.79	$<.001$

Appendix

Table 14: RQ2 results comparing the coherence and unrelated conditions on the full set of generations. PO priming indicates PO incidence following a PO prime, DO computed analogously.

\Delta^{C-U}

denotes the difference between coherent and unrelated percentages. Statistical reliability is assessed with two-sample proportion

z

-tests, with

p

-values corrected using Benjamini–Hochberg FDR correction. Gray text indicates insignificant differences p

>

0.001 . This table shows full numerical results corresponding to Figure 2 .

Metric	Prim.	Phi-2			Gemma-2-2B			Gemma-2-9B			EuroLLM-1.7B			EuroLLM-9B			Qwen-3-1.7B			Qwen-3-8B
		$\overline{P}$	$P$	t	$\overline{P}$	$P$	t	$\overline{P}$	$P$	t	$\overline{P}$	$P$	t	$\overline{P}$	$P$	t	$\overline{P}$	$P$	t	$\overline{P}$	$P$	t
MiniLM	U	0.26	0.38	8.29	0.29	0.39	7.08	0.28	0.36	6.37	0.24	0.36	9.57	0.24	0.35	8.49	0.26	0.35	6.12	0.27	0.36	7.09
	C	0.38	0.49	9.74	0.38	0.50	10.89	0.39	0.52	11.02	0.34	0.47	11.13	0.36	0.48	11.27	0.36	0.46	8.94	0.35	0.45	8.37
MPNet	U	0.25	0.36	8.38	0.27	0.38	7.67	0.27	0.35	6.13	0.23	0.36	9.75	0.23	0.33	8.44	0.26	0.33	5.54	0.27	0.35	6.41
	C	0.36	0.46	8.90	0.36	0.47	10.38	0.36	0.48	10.20	0.32	0.44	11.37	0.34	0.44	9.73	0.34	0.44	8.45	0.33	0.41	7.85
BERTScore F1	U	0.23	0.28	6.39	0.20	0.23	4.05	0.20	0.25	7.45	0.16	0.21	6.01	0.19	0.22	4.31	0.17	0.22	6.39	0.18	0.22	6.06

Appendix

Table 15: RQ3. PO-specific similarity analysis on the full set of generations. Mean similarity scores for completions that do not realise the primed PO structure (

\overline{P}

) and completions that realise it (

P

), shown separately for the unrelated (U) and coherent (C) conditions. Reported t -values come from Welch’s t -tests comparing PO-

P

and PO-

\overline{P}

completions within each context. An asterisk indicates statistical significance after Benjamini–Hochberg FDR correction. All comparisons are statistically significant. This Table shows the PO-specific analysis of Table 3 .

Metric	Prim.	Phi-2			Gemma-2-2B			Gemma-2-9B			EuroLLM-1.7B			EuroLLM-9B			Qwen-3-1.7B			Qwen-3-8B
		$\overline{P}$	$P$	t	$\overline{P}$	$P$	t	$\overline{P}$	$P$	t	$\overline{P}$	$P$	t	$\overline{P}$	$P$	t	$\overline{P}$	$P$	t	$\overline{P}$	$P$	t
MiniLM	U	0.25	0.30	2.29	0.27	0.35	3.52	0.26	0.35	4.24	0.22	0.30	3.60	0.22	0.30	3.83	0.27	0.34	3.18	0.26	0.30	2.01
	C	0.34	0.45	6.84	0.37	0.47	6.61	0.39	0.55	10.38	0.30	0.40	6.17	0.35	0.49	9.06	0.36	0.46	5.99	0.39	0.53	10.04
MPNet	U	0.26	0.30	1.97	0.27	0.34	2.97	0.26	0.34	4.11	0.23	0.30	3.53	0.22	0.28	3.36	0.27	0.35	3.71	0.26	0.29	1.50
	C	0.33	0.42	6.53	0.36	0.44	5.84	0.37	0.50	9.12	0.30	0.38	5.13	0.34	0.45	8.14	0.35	0.43	5.11	0.37	0.49	8.74
BERTScore F1	U	0.19	0.22	2.21	0.16	0.18	1.34	0.17	0.17	0.20	0.14	0.14	0.19	0.16	0.18	1.99	0.14	0.17	2.40	0.14	0.15	0.90

Appendix

Table 16: RQ3. DO-specific similarity analysis on the full set of generations . Mean similarity scores for completions that do not realise the primed DO structure (

\overline{P}

) and completions that realise it (

P

), shown separately for the unrelated (U) and coherent (C) conditions. Reported t -values come from Welch’s t -tests comparing DO-

p

and DO-

\overline{P}

completions within each context. The results in black indicate statistical significance after Benjamini–Hochberg FDR correction. Gray text indicates insignificant differences. This table shows the DO-specific analysis of Table 3 .

Panel of Figure 7 — Figure 7: Target-specific similarity analysis for PO and DO priming. Each heatmap reports $\Delta=P-\overline{P}$ , where $P$ denotes completions that realise the primed structure and $\overline{P}$ denotes completions that do not. Rows correspond to similarity metrics under unrelated (U) and coherence (C) conditions; columns correspond to models. Positive values indicate that structurally primed completions are more similar to the prime than non-primed completions. The effect is consistently positive for PO and becomes stronger for DO in the coherence condition, suggesting that structural priming is accompanied by increased semantic and lexical alignment with the prime. We report the Welch’s t -tests results in Table 3 .

Model	Prompt	LM completion	Label
EuroLLM	A winner threw a juice to a husband. A husband kept the	A husband kept the juice and drank it.	non-dative
EuroLLM	A lawyer saved a tea for a buddy. A buddy gave the	A buddy gave the scruffy-looking solo lawyer a drink since they tasted funny.	DO
EuroLLM	A husband brought a meal to a writer. A writer designed the	A writer designed the meal by placing between each serving of meat and each vegetable a single small, leafy, green vegetable.	non-dative
EuroLLM	An attorney made a camera for an opponent. An opponent gave the	An opponent gave the attorney 10 yellow cameras.	DO
Gemma	A man promised a husband a coffee. A man purchased the	A man purchased the goods, but instead put coffee on sugar.	non-dative
Gemma	A husband purchased the	A husband purchased the house his wife grew up in.	non-dative

Appendix

Table 17: Illustrative LM completions from the manual annotation process. The examples include cases where generated continuations superficially resemble dative constructions, but require manual inspection to determine whether they realise a genuine PO, DO, or non-dative structure.

Model	Target	No Context	Non-dative	Unrelated	$\Delta^{N}$	$z^{N}$	$p^{N}_{\mathrm{BH}}$	$\Delta^{O}$	$z^{O}$	$p^{O}_{\mathrm{BH}}$
EuroLLM-1.7B	PO	6.00	14.00	15.00	9.00	2.08	.114	1.00	0.20	.841
	DO	5.00	9.00	7.00	2.00	0.60	.662	-2.00	-0.52	.841
Gemma-2-2B	PO	11.00	8.00	19.00	8.00	1.58	.226	11.00	2.28	.137
	DO	2.00	5.00	4.00	2.00	0.83	.611	-1.00	-0.34	.841
Phi-2-2.7B	PO	6.00	12.00	22.00	16.00	3.26	.007	10.00	1.88	.179
	DO	8.00	5.00	9.00	1.00	0.25	.800	4.00	1.11	.535

Appendix

Table 18: RQ1 results on the hand-annotated set. For each target structure, Unrelated reports the percentage of structure-matching completions after unrelated dative primes. No Context is the no-context baseline, and Non-Dative is the non-dative-prime baseline.

\Delta^{N}

and

z^{N}

compare unrelated dative primes against the no-context baseline;

\Delta^{O}

and

z^{O}

compare them against the non-dative -prime baseline.

p

-values are Benjamini–Hochberg corrected.

	PO priming					DO priming
Model	Coher.	Unrel.	$\Delta^{C-U}$	$z$	$p_{\mathrm{BH}}$	Coher.	Unrel.	$\Delta^{C-U}$	$z$	$p_{\mathrm{BH}}$
EuroLLM-1.7B	20.00	15.00	5.00	0.93	.423	21.00	7.00	14.00	2.85	.013
Gemma-2-2B	25.00	19.00	6.00	1.02	.423	18.00	4.00	14.00	3.16	.009
Phi-2-2.7B	21.00	22.00	-1.00	-0.17	.863	19.00	9.00	10.00	2.04	.083

Appendix

Table 19: RQ2 results on the hand-annotated set, comparing coherent and unrelated prime conditions. Values are percentages except for

z

and corrected

p

-values.

\Delta^{C-U}

denotes the difference between coherent and unrelated conditions. Reliability is assessed with two-sample proportion

z

-tests with Benjamini–Hochberg FDR correction.

Metric	Cond.	EuroLLM-1.7B			Gemma-2-2B			Phi-2-2.7B
		$\overline{P}$	$P$	$t$	$\overline{P}$	$P$	$t$	$\overline{P}$	$P$	$t$
MiniLM	U	0.28	0.31	0.64	0.29	0.40	2.55 ∗	0.25	0.32	2.06
	C	0.36	0.38	0.62	0.39	0.47	2.46 ∗	0.37	0.47	3.64 ∗
MPNet	U	0.27	0.30	0.94	0.28	0.36	1.75	0.25	0.31	2.06
	C	0.34	0.37	0.79	0.37	0.44	2.38 ∗	0.35	0.48	4.62 ∗
BERTScore F1	U	0.17	0.20	1.43	0.18	0.23	2.56 ∗	0.21	0.26	2.44 ∗

Appendix

Table 20: RQ3 results on the hand-annotated set. Mean similarity scores are reported for non-primed (

\overline{P}

) and primed (

P

) completions in the unrelated (U) and coherent (C) conditions. Reported

t

values come from Welch’s

t

-tests comparing primed and non-primed completions within each condition and model. Asterisks indicate significance after Benjamini–Hochberg FDR correction (

p_{\mathrm{BH}}<.05

Linguistic Productivity in Large Language Models: Models Coerce, but do not Preempt
Jun 1, 2026Claire Bonial, Claire Benet Post, Laura Michaelis +1LLM Evaluation Language Modeling
Georgetown University · University of Colorado Boulder · University of Bath
Production and Perception in LLMs: A Token Probability Approach
Jul 13, 2026Anna Marklová, Jiří Milička, Martina Vokáčová +1Prompt Sensitivity
Faculty of Arts, Charles University, Prague · Faculty of Mathematics and Physics, Charles University, Prague
Implicit Representations of Grammaticality in Language Models
May 6, 2026Yingshan Susan Wang, Linlu Qiu, Zhaofeng Wu +2Language Model Probing
Massachusetts Institute of Technology

Jun 1, 2026J/K move · Enter open · S save

Linguistic Productivity in Large Language Models: Models Coerce, but do not Preempt

Claire Bonial, Claire Benet Post, Laura Michaelis, Harish Tayyar Madabushi

Georgetown University · University of Colorado Boulder · University of Bath

LLM Evaluation Language Modeling

Open paper AskPDF arXiv

Usage-based theories of grammar explain language users' ability to interpret and produce novel combinations of words through two complementary inferential strategies. In cases of coercion, a language user, when encountering a word that is mismatched to its grammatical context, reinterprets that word in a way that resolves the conflict. In cases of preemption, the user infers that an otherwise plausible structure is unavailable or dispreferred because it fails to occur in contexts where it would be expected. Large Language Models are also usage-based, in the sense that the structures of language are learned through exposure to vast amounts of text. Here, we test whether or not the opposing forces of coercion and preemption also encourage and constrain linguistic productivity in LLMs. We demonstrate across models of various sizes and architectures that larger models recognize constructional productivity in cases of coercion, even when these involve novel words. However, we also show that even the largest models do not replicate the conservative language strategy of preemption for novel words: in particular, they do not appear to draw on negative evidence when making choices among competing grammatical options. We conclude that coercion is detectable because it is both surprising and semantically motivated: when a word is mismatched to its syntactic context, the word's meaning can be adjusted to fit the context. Preemption has no such signature. There is no record of the lexeme-construction pairings that were avoided--only of those that were produced. This asymmetry helps to explain why models coerce (create mismatched sign combinations) but do not preempt (avoid unconventional yet semantically defensible ones).

Abstract

Explore similar work

Linguistic Productivity in Large Language Models: Models Coerce, but do not Preempt

Production and Perception in LLMs: A Token Probability Approach

Implicit Representations of Grammaticality in Language Models