Quantifying Cultural-Psychological Constructs in Four Centuries of Ottoman Court Registers
Organizations: Department of Psychology, University of Michigan · Department of Digital Humanities, University of Helsinki · Department of Psychological and Brain Sciences, University of Massachusetts Amherst
Abstract
Populations vary along cultural dimensions such as individualism, collectivism, and honor. But how can these dimensions be studied in historical societies for which conventional psychological data are unavailable? Here, we introduce a computational text-analytic framework for quantifying psychological constructs in 54,670 Ottoman Turkish court records from 21 Istanbul courts spanning 1513-1911 CE. We develop sentence-embedding models adapted to Ottoman legal language and extend a theory-driven, text-analytic pipeline to capture cultural variation along the dimensions of individualism, collectivism, and honor in this historical corpus. We evaluate our models through multiple methods: document retrieval, semantic textual similarity, questionnaire-item classification, and validation against expert annotations. Our adapted sentence transformer (OTA-BERTurk) achieved a Recall@1 of .843 on the test set for document retrieval, a Mean Reciprocal Rank (MRR) of .806 for semantic textual similarity (meaning the best answer usually appeared as a top result), and classified questionnaire items with 89.58% accuracy. Document-level performance varied across constructs and methods. Qualitative analysis further showed that high scores reflected historically specific legal and institutional patterns. We publicly release the models, construct representations, scoring pipeline, as well as an interactive web interface to support transparent, reproducible historical-psychological analyses of Ottoman Turkish court records spanning approximately four centuries. These computational tools enable the study of psychological variation across centuries of cultural and institutional change in a diverse empire.
Figures & tables
| Model | Stage | NDCG@10 | MRR@10 | R@1 | R@5 | R@10 |
|---|---|---|---|---|---|---|
| OTA XLM-RoBERTa | Base | 0.003 | 0.002 | 0.002 | 0.003 | 0.004 |
| BERTurk | Base | 0.011 | 0.009 | 0.006 | 0.013 | 0.019 |
| EmbeddingGemma-2 | Base | 0.566 | 0.528 | 0.449 | 0.627 | 0.692 |
| SimCSE (XLM-RoBERTa) | Stage 1 | 0.005 | 0.003 | 0.001 | 0.005 | 0.012 |
| SimCSE (BERTurk) | Stage 1 | 0.024 | 0.018 | 0.011 | 0.029 | 0.041 |
| OTA-XLM-RoBERTa | Stage 2 | 0.889 | 0.873 | 0.831 | 0.920 | 0.939 |
| Model | Accuracy (%) | F1-Macro (%) | F1-Weighted (%) |
|---|---|---|---|
| Fill-Mask / MLM Encoders | |||
| XLM-RoBERTa-base | 47.17 14.42 | 24.49 5.86 | 33.21 12.42 |
| OTA XLM-RoBERTa | 47.17 13.80 | 24.38 4.47 | 33.12 12.20 |
| BERTurk | 85.08 7.74 | 77.74 12.37 | 83.96 8.51 |
| SerBERT | 85.25 8.35 | 78.10 12.74 | 84.06 9.20 |
| Domain-adapted Models (ours) | |||
| Model | MRR [95% CI] | R@1 | Cross-reg. AUC [95% CI] | AUC (all) | PR-AUC |
|---|---|---|---|---|---|
| MLM encoders | |||||
| XLM-RoBERTa-base | 0.139 [0.105, 0.177] | 0.045 | 0.623 [0.603, 0.648] | 0.486 | 0.005 |
| OTA XLM-RoBERTa | 0.351 [0.290, 0.408] | 0.240 | 0.771 [0.733, 0.809] | 0.679 | 0.016 |
| BERTurk | 0.331 [0.276, 0.389] | 0.235 | 0.683 [0.657, 0.713] | 0.590 | 0.009 |
| SerBERT | 0.566 [0.502, 0.632] | 0.465 | 0.870 [0.841, 0.900] | 0.820 | 0.075 |
| Domain-adapted models | |||||
| Construct | Gwet’s AC1 | Krippendorff’s | Fleiss’ | PABAK | % Agreement |
|---|---|---|---|---|---|
| Honor | 0.89 [0.86, 0.92] | 0.37 [0.27, 0.47] | 0.40 | 0.81 | 86% |
| Individualism | 0.89 [0.85, 0.91] | 0.71 [0.63, 0.78] | 0.71 | 0.83 | 88% |
| Collectivism | 0.53 [0.46, 0.59] | 0.17 [0.11, 0.23] | 0.17 | 0.40 | 55% |
| Model | Honor | Individualism | Collectivism |
|---|---|---|---|
| MLM encoders (ROC-AUC) | |||
| XLM-RoBERTa-base | 0.572 [0.451, 0.682] | 0.519 [0.427, 0.606] | 0.350 [0.288, 0.417] |
| OTA XLM-RoBERTa | 0.697 [0.599, 0.790] | 0.280 [0.212, 0.356] | 0.606 [0.542, 0.672] |
| BERTurk | 0.349 [0.252, 0.460] | 0.712 [0.639, 0.783] | 0.627 [0.558, 0.694] |
| SerBERT | 0.424 [0.312, 0.536] | 0.705 [0.634, 0.778] | 0.500 [0.419, 0.578] |
| Domain-adapted models (ours; ROC-AUC) | |||
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Identifier or base model | Params | Max. length | Adaptation |
|---|---|---|---|---|
| MLM encoders | ||||
| XLM-RoBERTa-base | FacebookAI/xlm-roberta-base | 278M | 256 | None |
| OTA XLM-RoBERTa | enesyila/ota-roberta-base-kadisicilleri | 278M | 256 | MLM on corpus |
| BERTurk | dbmdz/bert-base-turkish-cased | 111M | 256 | None |
| SerBERT | sumeyyeakca/serbert-ottoman-turkish-bert-model | 111M | 256 | None |
| Off-the-shelf embedding models | ||||
| Construct | Representation | Illustrative text |
|---|---|---|
| Honor | Original item | "… your family had a bad reputation?" |
| Ottoman Turkish adaptation | "… aileniz kötü bir şöhrete sahip olsaydı?" | |
| Aligned proxy | Hânedanımızın nâm-ı nişanına halel gelmesi, cümlemiz için ölümden beter bir züldür. | |
| Opposed proxy | Ailenin mazideki hataları, ferdin istikbaline mani teşkil etmemelidir. | |
| Expert-definition excerpt | Şeref; şahsiyet, etvâr, hissiyât ve saâdet-i şahsiyyedeki ihtilâfâtı îzâh içün istimâl olunan dîğer bir temel mefhûmdur. | |
| Individualism | Original item | "I am unique, different from everyone else." |
| Rank | SD above mean | Court | Date | Source locator |
|---|---|---|---|---|
| 1 | 3.56 | Galata Court | 1575–1576 | Register 5; vol. 32, p. 51; judgment 32; original text 17-3 |
| 2 | 3.48 | Istanbul Court | 1661–1663 | Register 10; vol. 51, p. 438; judgment 473; original text 59a-2 |
| 3 | 3.48 | Balat Court | 1563 | Register 2; vol. 11, p. 263; judgment 412; original text 73a-2 |
| 4 | 3.44 | Galata Court | 1596–1599 | Register 20; vol. 35, p. 393; judgment 521; original text 78a-2 |
| 5 | 3.41 | Istanbul Court | 1591–1617 | Register 191; vol. 44, p. 180; judgment 167; original text 34b-4 |
| 6 | 3.40 | Rumeli Chief Judge Office Court | 1633 | Register 56; vol. 14, p. 63; judgment 24; original text 7b-1 |
| Rank | SD above mean | Court | Date | Source locator |
|---|---|---|---|---|
| 1 | 4.98 | Rumeli Chief Judge Office Court | 1656–1658 | Register 106; vol. 50, p. 599; judgment 626; original text 95b-1 |
| 2 | 4.97 | Rumeli Chief Judge Office Court | 1623–1624 | Register 40; vol. 46, p. 120; judgment 94; original text 17a-1 |
| 3 | 4.76 | Evkaf-ı Hümâyûn Inspectorate Court | 1608–1626 | Register 1; vol. 45, p. 358; judgment 311; original text 96a-1 |
| 4 | 4.63 | Eyüp Court | 1585–1587 | Register 3; vol. 22, p. 181; judgment 282; original text 30b-1 (Arabic) |
| 5 | 4.58 | Rumeli Chief Judge Office Court | 1633 | Register 56; vol. 14, p. 39; judgment 1; original text 1b-1 |
| 6 | 4.58 | Eyüp Court | 1644 | Register 49; vol. 26, p. 114; judgment 103; original text 19a-1 |
| Rank | SD above mean | Court | Date | Source locator |
|---|---|---|---|---|
| 1 | 4.13 | Bab Court | 1670–1671 | Register 11; vol. 53, p. 354; judgment 425; original text 80a-2 |
| 2 | 4.08 | Bab Court | 1670–1671 | Register 11; vol. 53, p. 176; judgment 160; original text 33a-4 |
| 3 | 4.08 | Bab Court | 1670–1671 | Register 11; vol. 53, p. 176; judgment 158; original text 33a-2 |
| 4 | 4.01 | Rumeli Chief Judge Office Court | 1656–1658 | Register 106; vol. 50, p. 505; judgment 524; original text 76a-5 |
| 5 | 4.00 | Rumeli Chief Judge Office Court | 1704 | Register 161; vol. 59, p. 472; judgment 491; original text 119b-1 |
| 6 | 3.84 | Galata Court | 1596–1599 | Register 20; vol. 35, p. 396; judgment 527; original text 78b-3 (Arabic) |
Computational Social Science
Computational social science uses data and computational models to study social behavior, institutions, communication, and collective outcomes.
See all 174 papers