Polish phonology and morphology through the lens of distributional semantics

Paula Orzechowska; R. Harald Baayen

cs.CLMar 31, 2026

Polish phonology and morphology through the lens of distributional semantics

Authors: Paula Orzechowska, R. Harald Baayen

Organizations: Adam Mickiewicz University · University of Tübingen

Abstract

This study investigates the relationship between the phonological and morphological structure of Polish words and their meanings using Distributional Semantics. In the present analysis, we ask whether there is a relationship between the form properties of words containing consonant clusters and their meanings. Is the phonological and morphonological structure of complex words mirrored in semantic space? We address these questions for Polish, a language characterized by non-trivial morphology and an impressive inventory of morphologically-motivated consonant clusters. We use statistical and computational techniques, such as t-SNE, Linear Discriminant Analysis and Linear Discriminative Learning, and demonstrate that -- apart from encoding rich morphosyntactic information (e.g. tense, number, case) -- semantic vectors capture information on sub-lexical linguistic units such as phoneme strings. First, phonotactic complexity, morphotactic transparency, and a wide range of morphosyntactic categories available in Polish (case, gender, aspect, tense, number) can be predicted from embeddings without requiring any information about the forms of words. Second, we argue that computational modelling with the discriminative lexicon model using embeddings can provide highly accurate predictions for comprehension and production, exactly because of the existence of extensive information in semantic space that is to a considerable extent isomorphic with structure in the form space.

Figures & tables

1. most parsable	2. less parsable	3. least parsable
clear parsing	active stem	part-of-speech transfer
prefix +	prefix +	prefix +
free base	active bound root	bound root from another part of speech
s+kłamać ‘to lie’	w+płacać ’to pay in’	s+padochron ’parachute’

Table 2: Degrees of parsability of a prefix. Categories (1) through (3) indicate an increasing number of steps necessary to determine the presence of a morpheme boundary in an initial cluster.

case	number	fem	neut	masc.pers.	masc.anim.	masc.inanim.
Nom.	sg.	matka, radość	dziecko, liceum	pan	wilk	miecz
Gen.	sg.	matki, radości	dziecka, liceum	pana	wilka	miecza
Dat.	sg.	matce, radości	dziecku, liceum	panu	wilkowi	mieczowi
Acc.	sg.	matkę, radość	dziecko, liceum	pana	wilka	miecz
Instr.	sg.	matką, radością	dzieckiem, liceum	panem	wilkiem	mieczem
Loc.	sg.	matce, radości	dziecku, liceum	panu	wilku	mieczu

Table 3: Case by number paradigm for three genders; feminine, neuter and masculine. The latter gender is further divided into three animacy subgenders; masculine personal, masculine animate and masculine inanimate.

verb form	mood	person	number	gender	tense	aspect
chowa-m	indicative	1	sg		pres	-
chowa-sz	indicative	2	sg		pres	-
chowa	indicative	3	sg		pres	-
chowa-my	indicative	1	pl		pres	-
chowa-cie	indicative	2	pl		pres	-
chowa-ją	indicative	3	pl		pres	-

Table 4: Basic forms of the verb chować ‘to hide’ (n=81). The imperfective paradigm features 51 unique forms; the perfective paradigm - 38 unique forms.

case	singular	gender	plural	gender
passive adjectival participle
Nom.	(s)chowa-ny/na/ne	m/f/n	(s)chowa-ni/ne	m.p/m.np
Gen.	(s)chowa-nego/nej	m,n/f	(s)chowa-nych	m.p,m.np
Dat.	(s)chowa-nemu/nej	m,n/f	(s)chowa-nym	m.p,m.np
Acc.	(s)chowa-nego/ną/ne	m/f/n	(s)chowa-nych/ne	m.p/m.np
Instr.	(s)chowa-nym/ną	m,n/f	(s)chowa-nymi	m.p,m.np

Table 5: Participle forms of the verb chować ‘to hide’ (n=83). Out of 27 available passive adjectival participle imperfective forms, 16 (=60%) are syncretic. The same proportion holds for the perfective counterparts. Out of 27 available active adjectival participle imperfective forms, 17 (=63%) are syncretic.

case	singular	plural
Nom.	(s)chowa-nie	(s)chowa-nia
Gen.	(s)chowa-nia	(s)chowa-ń
Dat.	(s)chowa-niu	(s)chowa-niom
Acc.	(s)chowa-nie	(s)chowa-nia
Instr.	(s)chowa-niem	(s)chowa-niami
Loc.	(s)chowa-niu	(s)chowa-niach

Table 6: Gerund forms of the verb chować ‘to hide’ (n=28). Out of 14 available imperfective gerund forms, six (=43%) are syncretic. The same proportion holds for the perfective counterparts.

size	rise (unmarked)	plateau	fall (marked)
CC	vj vl vm vr vw zj	f \textipa \textctc fs fx s \textipa \textctc sf ss	\textipa \textctc \textipa \texttctclig fp f \textipa \texttctclig sk sp st zb zd z \textipa \textdctzlig zg
	z \textipa \textObardotlessj zl zm zn z \textipa \textltailn zr zw	s \textipa \textesh sx vz v \textipa \textyogh zv z \textipa \textyogh
CCC	-	fsx	fkw fpr fpw fsk fsp fst skj skl skr sk \textipa \textesh
			skw spj spl spr sp \textipa \textesh spw stf str stw vzb
			vzd vzg zbj zbl zbr zgj zg \textipa \textltailn zgr zgw
CCCC	-	-	fstr fst \textipa \textesh stfj vzdw vzgl

Table 7: Word-initial morphonotactic clusters found in the data.

	full data	training data	test data
comprehension	97.2	97.5	98.3
production	97.3	97.6	77.8

Table 8: Accuracy (in percentages) for comprehension and production for the full dataset, a training dataset and a held-out test data set.

change in one feature	no of forms	change in two features	no. of forms
aspect	11	aspect+tense	3
case	8	number+person	2
number	7	number+case	2
tense	6	tense+person	1
gender	3	case+gender	1
person	2

Table 9: Counts of errors cross-classified by inflectional features for words that are produced by the model with the correct stem.

Panel of Figure 1 — Figure 1: Scatterplots of words in a 2-dimensional t-SNE space, colour-coded for six parts of speech (top) and 14 parts of speech (bottom). Further details can be traced in the corresponding interactive maps for the [ 6 parts of speech ] and [ 14 parts of speech ].

	adjective	adverb	gerund	noun	participle	verb
adjective	381 (86%)	0	1	19	34	6
adverb	2	14 (74%)	0	1	2	0
gerund	3	0	977 (94%)	61	3	0
noun	43	2	55	574 (83%)	11	9
participle	18	1	1	0	2487 (99%)	6
verb	10	6	9	18	14	3229 (98%)

Table 10: A confusion matrix for six parts of speech; model accuracy =

96\text{\,}\mathrm{\%}

	active.adj.P	adj	adv	fut.V	ger	imperatV	impersV	infV	noun	passive.adj.P	past.V	perf.adv.P	presV	pres.adv.P
active.adj.particip	633 (99,5%)	0	1	0	0	0	0	0	0	2	0	0	0	0
adj	4	387 (88%)	0	0	1	0	0	0	22	22	3	2	0	0
adv	0	3	13 (68%)	0	0	0	0	0	1	1	0	0	0	1
fut.verb	4	1	1	735 (97%)	5	0	0	1	2	1	1	1	5	1
ger	0	2	0	0	980 (94%)	0	0	0	61	1	0	0	0	0
imperat.verb	4	2	0	8	5	331 (89%)	3	4	5	3	3	2	1	1

Table 11: A confusion matrix for 14 parts of speech; model accuracy =

95\text{\,}\mathrm{\%}

Figure 2: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for three tenses. An interactive map is found [ here ].

	future	past	present
future	750 (99%)	4	4
past	0	1221(99,9%)	1
present	3	2	496 (99,9%)

Table 12: A confusion matrix for tense; model accuracy =

99\text{\,}\mathrm{\%}

Figure 3: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for three persons. An interactive map is found [ here ].

	first	second	third
first	494 (99%)	2	5
second	15	611 (97%)	4
third	1	8	1713 (99%)

Table 13: A confusion matrix for person; model accuracy =

99\text{\,}\mathrm{\%}

Figure 4: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for seven cases. An interactive map is found [ here ].

	accusative	dative	genitive	instrumental	locative	nominative	vocative
accusative	321 (64%)	1	21	1	1	158	1
dative	0	170 (81%)	0	22	18	0	0
genitive	5	0	1244 (98%)	2	2	6	5
instrumental	32	0	1	752 (94%)	12	2	0
locative	1	2	27	65	392 (80%)	1	1
nominative	69	4	12	4	2	1176 (93%)	2

Table 14: A confusion matrix for case; model accuracy =

89\text{\,}\mathrm{\%}

Panel of Figure 5 — Figure 5: Scatterplots of words in a 2-dimensional t-SNE space, colour-coded for three genders (top) and five genders (bottom). An interactive map for three genders is found [ here ]; for five genders [ here ].

	feminine	masculine	neuter
feminine	1607 (85%)	211	70
masculine	165	1938 (88%)	101
neuter	60	163	1446 (87%)

Table 15: A confusion matrix for three genders; model accuracy =

87\text{\,}\mathrm{\%}

	feminine	m.animate	m.inanimate	m.personal	neuter
feminine	1621(86%)	0	192	4	71
m.animate	1	1(20%)	2	1	0
m.inanimate	133	5	1007 (79%)	52	84
m.personal	42	2	155	705 (77%)	14
neuter	63	0	152	4	1450 (87%)

Table 16: A confusion matrix for five genders; model accuracy =

83\text{\,}\mathrm{\%}

Figure 6: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for two numbers. An interactive map is found [ here ].

	singular	plural
singular	4692 (97%)	134
plural	157	2409 (94%)

Table 17: Confusion matrix for number; model accuracy =

96\text{\,}\mathrm{\%}

Figure 7: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for case, and symbol-coded for number. Location of plural and singular forms differs by case.

Figure 8: Scatterplot of the plural shift vectors of words in t-SNE space, colour-coded for case. Data points clearly cluster by case. Sub-clusters within each case comprise different genders and parts of speech.

Figure 9: Scatterplots of words in a 2-dimensional t-SNE space, colour-coded for two aspects. An interactive map is found [ here ].

	imperfective	perfective
imperfective	2643 (96%)	105
perfective	112	3982 (97%)

Table 18: A confusion matrix for aspect; model accuracy =

97\text{\,}\mathrm{\%}

Figure 10: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for seven prefixes. An interactive map is found [ here ].

	/ \textipa \textctc /	/f/	/fs/	/s/	/v/	/vz/	/z/
/ \textipa \textctc /	207 (96%)	0	0	8	0	0	0
/f/	0	728 (96%)	10	5	14	0	4
/fs/	0	0	218 (90%)	4	1	4	15
/s/	5	6	7	2799 (97%)	5	4	55
/v/	0	20	3	9	348 (88%)	1	3
/vz/	0	0	7	0	0	336 (95%)	12

Table 19: A confusion matrix for prefix type; model accuracy =

97\text{\,}\mathrm{\%}

Figure 11: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for three degrees of morphotactic parsability. An interactive map is found [ here ].

observed/predicted	most parsable	less parsable	least parsable
most parsable	3955 (89%)	439	70
less parsable	608	2154 (76%)	88
least parsable	132	83	468 (67%)

Table 20: Confusion matrix for the degrees of morphotactic parsability; model accuracy:

82\text{\,}\mathrm{\%}

Figure 12: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for nine sonority profile types. An interactive map is found [ here ].

	F	F+R	F+R+R	plat	plat+F	plat+F+R	plat+R	R	R+plat
F	1906 (91%)	235	3	55	18	5	27	121	9
F+R	317	1583 (79%)	1	23	2	2	16	55	2
F+R+R	1	0	66 (99%)	0	0	0	0	0	0
plat	60	66	1	519 (71%)	7	1	13	59	1
plat+F	15	9	0	11	372 (89%)	0	8	9	0
plat+F+R	4	1	0	1	0	40 (83%)	0	2	0

Table 21: A confusion matrix for sonority profile types; model accuracy =

82\text{\,}\mathrm{\%}

. F = fall, R = rise, plat = plateau.

Figure 13: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for three cluster sizes. An interactive map is found [ here ].

	CC	CCC	CCCC
CC	3877 (85%)	635	31
CCC	804	2528 (76%)	10
CCCC	12	7	96 (83%)

Table 22: A confusion matrix for cluster size; model accuracy =

81\text{\,}\mathrm{\%}

Figure 14: A comparison of accuracy results for two sets of embeddings: fastText and Word2vec .

category	fastText			Word2vec
	LDA	baseline	N	LDA	baseline	N
person	99%	60%	2853	93%	84%	1596
tense	99%	49%	2481	90%	65%	1505
prefix type	97%	40%	7997	90%	41%	5936
aspect	97%	60%	6842	85%	59%	4905
number	96%	65%	7392	90%	68%	5393

Table 23: A comparison of the results (model accuracy and majority baseline) for two sets of embeddings. The data are ordered according to the descending accuracy results for fastText , followed by Word2vec embeddings.

N

:total number of observations.

Figure 15: The representation of the spread of the data points in principal components. The left panel shows the proportion of variance as explained by components PC=1 through PC=300. The right panel shows how the total variance explained increases as more components are added.

Figure 16: Accuracy of linear discriminant analyses applied to the first n = 2, 5, 10, 50, 100, 150, 200, 250, 300 dimensions of PCA semantic vectors.

Figure 17: The representation of the spread of the data points in principal components for Word2vec . The left panel shows the proportion of variance as explained by components PC=1 through PC=300. The right panel shows how the total variance explained increases as more components are added.

Figure 18: Accuracy of linear discriminant analyses for Word2vec applied to the first n = 2, 5, 10, 50, 100, 150, 200, 250, 300 dimensions of PCA semantic vectors.

Figure 19: Difference in accuracy between LDA accuracy using fastText and Word2vec . This difference quantifies the advantage of having sublexical co-occurrence information over and above lexical co-occurrence information.

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

	full data	training data	test data
comprehension	77	79	75.6
production	85.6	87	53

Appendix

Table 24: Accuracy (in percentages) for comprehension and production for the full dataset, a training dataset and a held-out test data set for Word2vec embeddings.

Figure 20: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for six parts of speech.

	adjective	adverb	gerund	noun	participle	verb
adjective	230 (58%)	0	14	27	110	17
adverb	1	7 (47%)	0	2	5	0
gerund	3	0	693 (79%)	63	108	11
noun	32	2	70	406 (66%)	101	6
participle	62	0	33	38	1810 (89%)	93
verb	4	0	3	6	152	1827 (92%)

Appendix

Table 25: A confusion matrix for six parts of speech; model accuracy =

84\text{\,}\mathrm{\%}

Figure 21: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for 14 parts of speech.

	active.adj.particip	adj	adv	fut.verb	ger	imperat.verb	inf.verb	noun	passive.adj.particip	past.verb	perf.adv.particip	pres.verb	pres.adv.particip
active.adj.particip	353 (68%)	23	0	1	25	2	0	20	85	5	0	1	4
adj	32	247 (62%)	0	1	13	4	0	33	61	4	2	1	0
adv	3	1	7 (47%)	0	0	0	0	2	2	0	0	0	0
fut.verb	4	0	0	193 (76%)	3	24	1	0	10	13	0	6	0
ger	22	7	0	1	728 (83%)	3	0	73	41	2	1	0	0
imperat.verb	6	1	0	8	2	64 (70%)	1	3	6	0	0	0	0

Appendix

Table 26: A confusion matrix for 14 parts of speech; model accuracy =

79\text{\,}\mathrm{\%}

Figure 22: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for three tenses.

	future	past	present
future	214 (84%)	28	12
past	16	958 (97%)	11
present	46	32	188 (71%)

Appendix

Table 27: A confusion matrix for tense; model accuracy =

90\text{\,}\mathrm{\%}

Figure 23: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for three persons.

	first	second	third
first	95 (67%)	18	20
second	31	71 (63%)	10
third	25	7	1311 (98%)

Appendix

Table 28: A confusion matrix for person; model accuracy =

93\text{\,}\mathrm{\%}

Figure 24: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for seven cases.

	accusative	dative	genitive	instrumental	locative	nominative	vocative
accusative	255 (62%)	0	28	7	2	117	1
dative	2	119 (77%)	3	11	10	9	0
genitive	16	3	996 (91%)	8	11	55	0
instrumental	31	6	13	528 (84%)	22	25	0
locative	6	14	35	39	299 (72%)	19	1
nominative	83	3	34	15	4	964 (87%)	1

Appendix

Table 29: A confusion matrix for case; model accuracy =

83\text{\,}\mathrm{\%}

Figure 25: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for three genders.

	feminine	masculine	neuter
feminine	1177 (77%)	209	148
masculine	175	1477 (79%)	213
neuter	122	225	1036 (75%)

Appendix

Table 30: A confusion matrix for three genders; model accuracy =

77\text{\,}\mathrm{\%}

Figure 26: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for five genders.

	feminine	m.animate	m.inanimate	m.personal	neuter
feminine	1206 (79%)	0	150	15	163
m.animate	2 (67%)	0	0	1	0
m.inanimate	157	1	694 (63%)	53	199
m.personal	47	0	104	572 (75%)	35
neuter	127	0	169	25	1062 (77%)

Appendix

Table 31: A confusion matrix for five genders; model accuracy =

74\text{\,}\mathrm{\%}

Figure 27: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for two numbers.

	singular	plural
singular	1401 (81%)	330
plural	215	3447 (94%)

Appendix

Table 32: A confusion matrix for number; model accuracy =

90\text{\,}\mathrm{\%}

Figure 28: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for two aspects.

	imperfective	perfective
imperfective	1572 (78%)	445
perfective	309	2579 (89%)

Appendix

Table 33: A confusion matrix for aspect; model accuracy =

85\text{\,}\mathrm{\%}

Figure 29: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for three degrees of morphotactic parsability.

	most parsable	less parsable	least parsable
most parsable	2625 (80%)	480	89
less parsable	796	1314 (60%)	77
least parsable	183	150	222 (40%)

Appendix

Table 34: A confusion matrix for the degrees of morphotactic parsability; model accuracy =

70\text{\,}\mathrm{\%}

Figure 30: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for nine sonority profile types.

	F	F+R	F+R+R	plat	plat+F	plat+F+R	plat+R	R	R+plat
F	1081 (63%)	305	3	62	25	5	30	194	1
F+R	437	899 (59%)	4	45	28	2	23	86	0
F+R+R	11	2	44 (77%)	0	0	0	0	0	0
plat	98	65	0	159 (32%)	11	0	14	143	0
plat+F	59	36	4	12	180 (55%)	1	5	29	0
plat+F+R	11	3	0	0	4	19 (48%)	1	2	0

Appendix

Table 35: A confusion matrix for sonority profile types; model accuracy =

61\text{\,}\mathrm{\%}

Figure 31: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for three cluster sizes.

	CC	CCC	CCCC
CC	2584 (80%)	649	15
CCC	985	1586 (61%)	20
CCCC	24	8	65 (67%)

Appendix

Table 36: A confusion matrix for cluster size; model accuracy =

71\text{\,}\mathrm{\%}

Figure 32: Scatterplot of words in a 2-dimensional t-SNE space, colour-coded for seven prefixe types.

	\textipa \textctc	/f/	/fs/	/s/	/v/	/vz/	/z/
\textipa \textctc	96 (63%)	0	0	57	0	0	0
/f/	1	406 (77%)	3	55	43	3	14
/fs/	0	14	156 (74%)	10	3	8	20
/s/	18	17	14	2015 (96%)	3	5	30
/v/	0	62	5	55	136 (51%)	2	6
/vz/	0	10	7	3	1	199 (73%)	52

Appendix

Table 37: A confusion matrix for prefix type; model accuracy =

90\text{\,}\mathrm{\%}

Figure 33: Scatterplot of words in a 2-dimensional t-SNE space for Word2vec , colour-coded for case, and symbol-coded for number. Location of plural and singular forms differs by case.

Figure 34: Scatterplot of the plural shift vectors of words in t-SNE space for Word2vec , colour-coded for case. Data points cluster by case. Sub-clusters within the nominative cluster are differentiated by gender and part of speech.

Distributional Semantics

Distributional semantics models word meaning from patterns of co-occurrence and context in language.

See all 22 papers

Semantic Space of Parts of Speech
Aug 15, 2026Jiří Milička, Ivan Kraus, Arnold Stanovský +5Morphosyntactic Tagging Representation Geometry in Language Models
Charles University, Prague, Czech Republic · University of Potsdam, Potsdam, Germany
PACUTE: Phonology-, Affix-, and Character-level Understanding of Tokens for Filipino
Jun 13, 2026Jann Railey Montalan, David Demitri Africa, Jimson Paulo Layacan +3LLM Evaluation Language Modeling
AI Singapore · Nanyang Technological University · UK AI Security Institute +2
Word meaning co-determines vowel-inherent spectral change. A corpus-based investigation of conversational Mandarin
Jul 23, 2026Xiaoyun Jin, Mirjam Ernestus, R. Harald BaayenSpeech Production
Quantitative Linguistics, Eberhard Karls Universität Tübingen, 72074 Tübingen, Germany · Center for Language Studies, Radboud University, 6525 HT Nijmegen, The Netherlands

Aug 15, 2026J/K move · Enter open · S save

Semantic Space of Parts of Speech

Jiří Milička, Ivan Kraus, Arnold Stanovský, Anna Vysloužilová+4

Charles University, Prague, Czech Republic · University of Potsdam, Potsdam, Germany

Morphosyntactic Tagging Representation Geometry in Language Models

Open paper AskPDF arXiv

Parts of speech categorization is understood in the European linguistic tradition as crisp categorization, which is also reflected in corpus linguistics, where each disambiguated token is assigned exactly one POS. However, the assigned categories are largely determined by arbitrary decisions distilled into annotation manuals. Since some words stand between parts of speech in their semantics or typical syntax, and some parts of speech are closer to each other than others, POS categorization seems inherently fuzzy. We analyze this fuzziness using word2vec embeddings, training a neural network to reduce their high dimensionality to three dimensions relevant for determining parts of speech. This creates a three-dimensional space onto which we map several thousand words, revealing which are prototypical and which lie on the boundaries, and visualizing relationships between parts of speech. The study uses Universal Dependencies POS tags for French, Czech, Finnish, Russian, and English.

Abstract

Distributional Semantics

Explore similar work

Semantic Space of Parts of Speech

PACUTE: Phonology-, Affix-, and Character-level Understanding of Tokens for Filipino

Word meaning co-determines vowel-inherent spectral change. A corpus-based investigation of conversational Mandarin