cs.CLNov 20, 2025

Algorithmic Reflectivity in Large Language Models: Methodological Study of Linguistic Structure in Mentalization

Authors: Stefano Epifani, Giuliano Castigliego, Giuliano Razzicchia, Elisabeth Seiwald-Sonderegger, Laura Kecskeméti

Organizations: Digital Transformation Institute, Rome, Italy · Department of Communication and Social Research, Sapienza University of Rome, Italy · Psychoanalytic Academy of Italian-Speaking Switzerland, Chur, Switzerland · Psychiatric Services of the Canton of Grisons, Chur, Switzerland

Abstract

Mentalization is a core human capacity involving the interpretation of one's own and others' behavior in terms of underlying mental states. Large language models (LLMs) can generate linguistically reflective discourse, raising the question of whether the formal linguistic structure of mentalization can be reproduced without experiential and affective processes. This study assessed whether an LLM can generate mentalization profiles structurally coherent with the dimensions of mentalization-based treatment (MBT) and recognized as formally mentalizing by expert clinicians. Fifty human-model dialogues were generated with GPT-4.1 under standardized conditions; at the end of each, the model produced a narrative mentalization profile. Five MBT-trained psychiatrists independently rated each profile on 4 canonical MBT axes and 2 higher-order dimensions (regulatory functions, synthetic profile) for evaluative and argumentative coherence. Interrater reliability was assessed with ICC(3,k). Mean scores ranged from 3.63 to 3.98, indicating moderate-to-high structural coherence; ICC(3,k) in the evaluative domain ranged from 0.60 to 0.84. Evaluative and argumentative coherence were strongly associated (Pearson r 0.826-0.967). A negative control with mismatched dialogue-profile pairs yielded minimal ratings. Clinicians described outputs as linguistically organized and clinically interpretable, but with limited affective variability and contextual integration. The findings suggest that LLMs can reproduce the formal linguistic structure of reflective discourse - a phenomenon we term algorithmic reflectivity - which appears more consistent with structural language regularities than with experiential or intentional processes. LLMs may serve as research and training tools for studying the linguistic aspects of mentalization.

Explore similar work

CardsList
  1. Assessing mentalization in humans and large language models

    Aug 26, 2026Aamir Sohail, Xintong Zhong, Arkady Konovalov +2Social ReasoningTheory of Mind

  2. LLMs and their Limited Theory of Mind: Evaluating Mental State Annotations in Situated Dialogue

    Sep 2, 2025Katharine Kowalyshyn, Matthias ScheutzLLM EvaluationTheory of Mind

  3. Measuring Behavioural Signatures of Large Language Models through Psychometric Profiling

    Sep 19, 2026Yu Sha, Junqi Tao, Dixin Zhou +10Personality Modeling in Language ModelsMultilingual Language Model Evaluation