Quantifying Cultural-Psychological Constructs in Four Centuries of Ottoman Court Registers
Authors: Sila Mutaf, Enes Yılandiloğlu, Mohammad Atari
Organizations: Department of Psychology, University of Michigan · Department of Digital Humanities, University of Helsinki · Department of Psychological and Brain Sciences, University of Massachusetts Amherst
Populations vary along cultural dimensions such as individualism, collectivism, and honor. But how can these dimensions be studied in historical societies for which conventional psychological data are unavailable? Here, we introduce a computational text-analytic framework for quantifying psychological constructs in 54,670 Ottoman Turkish court records from 21 Istanbul courts spanning 1513-1911 CE. We develop sentence-embedding models adapted to Ottoman legal language and extend a theory-driven, text-analytic pipeline to capture cultural variation along the dimensions of individualism, collectivism, and honor in this historical corpus. We evaluate our models through multiple methods: document retrieval, semantic textual similarity, questionnaire-item classification, and validation against expert annotations. Our adapted sentence transformer (OTA-BERTurk) achieved a Recall@1 of .843 on the test set for document retrieval, a Mean Reciprocal Rank (MRR) of .806 for semantic textual similarity (meaning the best answer usually appeared as a top result), and classified questionnaire items with 89.58% accuracy. Document-level performance varied across constructs and methods. Qualitative analysis further showed that high scores reflected historically specific legal and institutional patterns. We publicly release the models, construct representations, scoring pipeline, as well as an interactive web interface to support transparent, reproducible historical-psychological analyses of Ottoman Turkish court records spanning approximately four centuries. These computational tools enable the study of psychological variation across centuries of cultural and institutional change in a diverse empire.
Figures & tables
Figure 1: Temporal Coverage and Corpus Size Across Ottoman Courts
Model
Stage
NDCG@10
MRR@10
R@1
R@5
R@10
OTA XLM-RoBERTa
Base
0.003
0.002
0.002
0.003
0.004
BERTurk
Base
0.011
0.009
0.006
0.013
0.019
EmbeddingGemma-2
Base
0.566
0.528
0.449
0.627
0.692
SimCSE (XLM-RoBERTa)
Stage 1
0.005
0.003
0.001
0.005
0.012
SimCSE (BERTurk)
Stage 1
0.024
0.018
0.011
0.029
0.041
OTA-XLM-RoBERTa
Stage 2
0.889
0.873
0.831
0.920
0.939
Table 1: Document Retrieval Performance on the Test Set
Model
Accuracy (%)
F1-Macro (%)
F1-Weighted (%)
Fill-Mask / MLM Encoders
XLM-RoBERTa-base
47.17 ± 14.42
24.49 ± 5.86
33.21 ± 12.42
OTA XLM-RoBERTa
47.17 ± 13.80
24.38 ± 4.47
33.12 ± 12.20
BERTurk
85.08 ± 7.74
77.74 ± 12.37
83.96 ± 8.51
SerBERT
85.25 ± 8.35
78.10 ± 12.74
84.06 ± 9.20
Domain-adapted Models (ours)
Table 2: Model performance on the QIC task.
Model
MRR [95% CI]
R@1
Cross-reg. AUC [95% CI]
AUC (all)
PR-AUC
MLM encoders
XLM-RoBERTa-base
0.139 [0.105, 0.177]
0.045
0.623 [0.603, 0.648]
0.486
0.005
OTA XLM-RoBERTa
0.351 [0.290, 0.408]
0.240
0.771 [0.733, 0.809]
0.679
0.016
BERTurk
0.331 [0.276, 0.389]
0.235
0.683 [0.657, 0.713]
0.590
0.009
SerBERT
0.566 [0.502, 0.632]
0.465
0.870 [0.841, 0.900]
0.820
0.075
Domain-adapted models
Table 3: Performance on the Semantic Textual Similarity Task
Construct
Gwet’s AC1
Krippendorff’s α
Fleiss’ κ
PABAK
% Agreement
Honor
0.89 [0.86, 0.92]
0.37 [0.27, 0.47]
0.40
0.81
86%
Individualism
0.89 [0.85, 0.91]
0.71 [0.63, 0.78]
0.71
0.83
88%
Collectivism
0.53 [0.46, 0.59]
0.17 [0.11, 0.23]
0.17
0.40
55%
Table 4: Inter-Annotator Agreement Scores
Model
Honor
Individualism
Collectivism
MLM encoders (ROC-AUC)
XLM-RoBERTa-base
0.572 [0.451, 0.682]
0.519 [0.427, 0.606]
0.350 [0.288, 0.417]
OTA XLM-RoBERTa
0.697 [0.599, 0.790]
0.280 [0.212, 0.356]
0.606 [0.542, 0.672]
BERTurk
0.349 [0.252, 0.460]
0.712 [0.639, 0.783]
0.627 [0.558, 0.694]
SerBERT
0.424 [0.312, 0.536]
0.705 [0.634, 0.778]
0.500 [0.419, 0.578]
Domain-adapted models (ours; ROC-AUC)
Table 5: Document-Level Validation Against Expert Annotations
Figure 2: An Example of Cultural-Psychological Construct Loadings, Displayed on the Web Interface
Figure 3: An Example of Psychological Variation Over Time, Displayed on the Web Interface.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Identifier or base model
Params
Max. length
Adaptation
MLM encoders
XLM-RoBERTa-base
FacebookAI/xlm-roberta-base
278M
256
None
OTA XLM-RoBERTa
enesyila/ota-roberta-base-kadisicilleri
278M
256
MLM on corpus
BERTurk
dbmdz/bert-base-turkish-cased
111M
256
None
SerBERT
sumeyyeakca/serbert-ottoman-turkish-bert-model
111M
256
None
Off-the-shelf embedding models
Appendix
Table 6: Embedding Models Evaluated
Construct
Representation
Illustrative text
Honor
Original item
"… your family had a bad reputation?"
Ottoman Turkish adaptation
"… aileniz kötü bir şöhrete sahip olsaydı?"
Aligned proxy
Hânedanımızın nâm-ı nişanına halel gelmesi, cümlemiz için ölümden beter bir züldür.
Opposed proxy
Ailenin mazideki hataları, ferdin istikbaline mani teşkil etmemelidir.
Expert-definition excerpt
Şeref; şahsiyet, etvâr, hissiyât ve saâdet-i şahsiyyedeki ihtilâfâtı îzâh içün istimâl olunan dîğer bir temel mefhûmdur.
Individualism
Original item
"I am unique, different from everyone else."
Appendix
Table 7: Illustrative Examples of Construct Representations
Rank
SD above mean
Court
Date
Source locator
1
3.56
Galata Court
1575–1576
Register 5; vol. 32, p. 51; judgment 32; original text 17-3
2
3.48
Istanbul Court
1661–1663
Register 10; vol. 51, p. 438; judgment 473; original text 59a-2
3
3.48
Balat Court
1563
Register 2; vol. 11, p. 263; judgment 412; original text 73a-2
4
3.44
Galata Court
1596–1599
Register 20; vol. 35, p. 393; judgment 521; original text 78a-2
5
3.41
Istanbul Court
1591–1617
Register 191; vol. 44, p. 180; judgment 167; original text 34b-4
6
3.40
Rumeli Chief Judge Office Court
1633
Register 56; vol. 14, p. 63; judgment 24; original text 7b-1
Appendix
Table 8: Highest-Scoring Individualism Records Examined Qualitatively
Rank
SD above mean
Court
Date
Source locator
1
4.98
Rumeli Chief Judge Office Court
1656–1658
Register 106; vol. 50, p. 599; judgment 626; original text 95b-1
2
4.97
Rumeli Chief Judge Office Court
1623–1624
Register 40; vol. 46, p. 120; judgment 94; original text 17a-1
3
4.76
Evkaf-ı Hümâyûn Inspectorate Court
1608–1626
Register 1; vol. 45, p. 358; judgment 311; original text 96a-1
4
4.63
Eyüp Court
1585–1587
Register 3; vol. 22, p. 181; judgment 282; original text 30b-1 (Arabic)
5
4.58
Rumeli Chief Judge Office Court
1633
Register 56; vol. 14, p. 39; judgment 1; original text 1b-1
6
4.58
Eyüp Court
1644
Register 49; vol. 26, p. 114; judgment 103; original text 19a-1
Appendix
Table 9: Highest-Scoring Collectivism Records Examined Qualitatively
Rank
SD above mean
Court
Date
Source locator
1
4.13
Bab Court
1670–1671
Register 11; vol. 53, p. 354; judgment 425; original text 80a-2
2
4.08
Bab Court
1670–1671
Register 11; vol. 53, p. 176; judgment 160; original text 33a-4
3
4.08
Bab Court
1670–1671
Register 11; vol. 53, p. 176; judgment 158; original text 33a-2
4
4.01
Rumeli Chief Judge Office Court
1656–1658
Register 106; vol. 50, p. 505; judgment 524; original text 76a-5
5
4.00
Rumeli Chief Judge Office Court
1704
Register 161; vol. 59, p. 472; judgment 491; original text 119b-1
6
3.84
Galata Court
1596–1599
Register 20; vol. 35, p. 396; judgment 527; original text 78b-3 (Arabic)
Appendix
Table 10: Highest-Scoring Honor Records Examined Qualitatively