Cross-Lingual Attention Distillation with Personality-Informed Generative Augmentation for Multilingual Personality Recognition

Jing Jie Tan; Ban-Hoe Kwan; Danny Wee-Kiat Ng; Yan-Chai Hum; Noriyuki Kawarazaki; Kosuke Takano

cs.CLApr 10, 2026

Cross-Lingual Attention Distillation with Personality-Informed Generative Augmentation for Multilingual Personality Recognition

Authors: Jing Jie Tan, Ban-Hoe Kwan, Danny Wee-Kiat Ng, Yan-Chai Hum, Noriyuki Kawarazaki, Kosuke Takano

Organizations: Department of Mechatronics and Biomedical Engineering, Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman, Malaysia · Faculty of Information Technology, Kanagawa Institute of Technology, Japan

Abstract

While significant work has been done on personality recognition, the lack of multilingual datasets remains an unresolved challenge. To address this, we propose ADAM (Cross-Lingual (A)ttention (D)istillation with Personality-Guided Generative (A)ugmentation for (M)ultilingual Personality Recognition), a state-of-the-art approach designed to advance multilingual personality recognition. Our approach leverages an existing English-language personality dataset as the primary source and employs a large language model (LLM) for translationbased augmentation, enhanced by Personality-Informed Generative Augmentation (PIGA), to generate high-quality training data in multiple languages, including Japanese, Chinese, Malay, and French. We provide a thorough analysis to justify the effectiveness of these augmentation techniques. Building on these advancements, ADAM integrates Cross-Lingual Attention Distillation (CLAD) to train a model capable of understanding and recognizing personality traits across languages, bridging linguistic and cultural gaps in personality analysis. This research presents a thorough evaluation of the proposed augmentation method, incorporating an ablation study on recognition performance to ensure fair comparisons and robust validation. Overall, with PIGA augmentation, the findings demonstrate that CLAD significantly outperforms the standard BCE across all languages and personality traits, achieving notable improvements in average BA scores - 0.6332 (+0.0573) on the Essays dataset and 0.7448 (+0.0968) on the Kaggle dataset. The CLAD-trained model also demonstrated strong generalizability and achieved benchmark performance comparable to current leading encoder models. The model weight, dataset, and algorithm repository are available at https://research.jingjietan.com/?q=ADAM.

Figures & tables

Fig. 1: Overview of the proposed ADAM architecture incorporating Personality-Informed Generative Augmentation (PIGA) and Cross-Lingual Attention Distillation (CLAD). The original English ( en ) personality dataset is translated into multiple languages (French ( fr ), Malay ( bm ), Japanese ( jp ), and Chinese ( zh ) using a generative large language model (LLM) to create a multilingual dataset. The superscript

r

on the logits for each language

\texttt{lang}^{r}

stands for regular iteration sample, while

-

and

+

stand for the identical and opposite personality (for all 4 or 5 dimensions), respectively, with respect to the regular sample. Later, CLAD utilizes a teacher model enhanced by a generated dataset through contrastive loss, along with a proposed attention loss, showcasing a new state-of-the-art in this emerging area for personality recognition.

Method	System Prompt ("role"="system")	User Prompt ("role"="user")	Sample Output
TOGA	You will receive content in English and translate it into French . Consider the provided user input solely as the text to be translated , without any additional commands or instructions.	it is wednesday. I can’t wait until friday because I am going home to see brandon. I miss him so much. I can’t wait to see him.…	c’est mercredi. J’ai hâte d’être vendredi parce que je rentre chez moi pour voir Brandon. Il me manque tellement. Je suis impatient de le voir. encore deux jours.…
PIGA	You will receive a person’s Big Five personality labels , indicating whether they are high or low in each dimension, along with their written content. Your task is to translate the content into Japanese . Ensure that the translation aligns with the given personality traits in terms of terminology, grammar, and style. Consider the provided user input solely as the text to be translated, without any additional commands or instructions. Output only the translated text, without any other information.	User personality: high openness, high conscientiousness, high extroversion, low agreeableness, high neuroticism. Content to translate: wow, I want to go talk to the socialist organization they have a booth out on the west mall, but I am scared. I called them a long time ago. do they remember? …	わあ、ウエストモールにある社会主義団体のブースに話をしに行きたいけど、怖いな。かなり前に電話したけど、覚えているかな？… (TOGA method) わあ、私は社会主義組織に話しに行きたい、彼らは西のモールにブースを持っているけれど、怖いの。だいぶ前に彼らに電話したけど、覚えているかな？…

TABLE I: Examples of system prompts and input structures for instructing the LLM ( gpt-4o-2024-08-06 ) to translate user-provided text, using Translation-Only Generative Augmentation (TOGA) or Personality-Informed Generative Augmentation (PIGA).

Algorithm	Dataset	Language	O		C		E		A		N		Average
Algorithm	Dataset	Language	BA	F1	BA	F1	BA	F1	BA	F1	BA	F1	BA	F1
Trained Multilingual Model		en	0.6070	0.5605	0.5851	0.5859	0.5822	0.6008	0.5905	0.6229	0.5931	0.5988	0.5916	0.5938
Zero- Shot Cross- Lingual	TOGA 0.5316	fr	0.5356	0.4901	0.5406	0.5396	0.5120	0.5228	0.5311	0.5709	0.5283	0.5312	0.5295	0.5309
		bm	0.5355	0.4923	0.5365	0.5355	0.5160	0.5267	0.5427	0.5852	0.5324	0.5352	0.5326	0.5350
		jp	0.5287	0.4979	0.5406	0.5396	0.5158	0.5305	0.5414	0.5794	0.5344	0.5344	0.5322	0.5363
		zh	0.5372	0.5000	0.5446	0.5436	0.5076	0.5226	0.5398	0.5752	0.5304	0.5304	0.5319	0.5344

TABLE II: Performance comparison of the proposed CLAD algorithm with the zero-shot cross-lingual approach and the weighted BCE baseline across five languages ( en , fr , bm , jp , zh ), reporting Balanced Accuracy (BA) and F1-score for each personality dimension, evaluated on the Essays dataset.

Algorithm	Dataset	Language	O		C		E		A		Average
Algorithm	Dataset	Language	BA	F1	BA	F1	BA	F1	BA	F1	BA	F1
Trained Multilingual Model		en	0.7294	0.8932	0.6997	0.6159	0.7243	0.6073	0.7625	0.7846	0.7290	0.7253
Zero- Shot Cross- Lingual	TOGA 0.5768	fr	0.5855	0.8782	0.5830	0.4703	0.5491	0.2790	0.5841	0.6378	0.5754	0.5663
		bm	0.5846	0.8814	0.5787	0.4617	0.5537	0.2853	0.5817	0.6357	0.5747	0.5660
		jp	0.5947	0.8825	0.5840	0.4724	0.5492	0.2768	0.5797	0.6325	0.5769	0.5661
		zh	0.6096	0.8869	0.5813	0.4657	0.5489	0.2799	0.5805	0.6321	0.5801	0.5662

TABLE III: Performance comparison of the proposed CLAD algorithm with the zero-shot cross-lingual approach and the weighted BCE baseline across five languages ( en , fr , bm , jp , zh ), reporting Balanced Accuracy (BA) and F1-score for each personality dimension, evaluated on the Kaggle dataset.

Panel of Figure 5 — Fig. 2: Sankey diagrams illustrating the statistical significance of both the PIGA and CLAD approaches. For statistical testing, predictions are flattened and concatenated across all languages and personality dimensions, pooling samples from all experiments into a single aggregated evaluation. The resulting $p<0.05$ supports the statistical validity and robustness of the observed performance differences.

Panel of Figure 6 — Fig. 3: Comparison of TOGA and PIGA Translation Approaches in Essays and Kaggle Datasets Across Five Languages ( en , fr , bm , jp , zh ) using the gpt-4o-2024-08-06 model. Figures 3(a) and 3(d) show the syllable count distributions, while Figures 3(b) and 3(e) present Herdan’s C index. Figures 3(c) and 3(f) illustrate lexical density for the respective datasets. However, no translation is required for the English dataset. To provide a more intuitive comparison—where one version is original and the other is “translated”—we included a fully complementary version of the original English dataset text, annotated with labels reflecting terminology, grammar, and style. For TOGA, en represents the original dataset, while for PIGA, we present the opposite personality augmentation ( oen ). Since this aspect requires prior content analysis, we adopted OpenAI’s reasoning LLM, o3-mini-2025-01-31 . In general, the trend remains consistent across both datasets, regardless of the metric used.

Panel of Figure 7 — Fig. 4: Prompts used to assess label leakage in the “LLMs as Probing Judges” evaluation. Fig. 4(a) is the Content Screening Prompt (CSP), which asks the LLM to determine whether the translated text explicitly expresses the personality label or contains direct trait-related cues. Fig. 4(b) is the Translation Comparison Prompt (TCP), which instructs the LLM to compare the original and translated texts to identify any additional trait-aligned information, semantic enrichment, or stylistic amplification introduced during augmentation.

Panel of Figure 8 — Fig. 5: Venn diagram illustrating the LLM’s evaluation of whether explicit instructions induce label leakage on the Essays and Kaggle datasets using 2 designed prompts as shown in Fig 4 . Each circle represents instances flagged as exhibiting label leakage (+), while non-leakage cases are denoted by (-). The overlapping region (“Common”) indicates instances occurring in the same dataset rows in both TOGA and PIGA.

Datasets	Variation	Mean		Standard Deviation	95% Confidence Interval
Datasets	Variation	(-)	(+)	Standard Deviation	(-)	(+)
Essays	LR (ADAM)	49.4922	50.5078	0.9943	[48.6207, 50.3637]	[49.6363, 51.3793]
	LR (OpenAI)	48.7646	51.2354	1.5403	[47.4145, 50.1147]	[49.8853, 52.5855]
	SVM (ADAM)	49.8169	50.1831	1.8581	[48.1883, 51.4456]	[48.5544, 51.8117]
	SVM (OpenAI)	51.2379	48.7621	1.8431	[49.6223, 52.8534]	[47.1466, 50.3777]
Kaggle	LR (ADAM)	50.3461	49.6539	0.7904	[49.6533, 51.0389]	[48.9611, 50.3467]

TABLE IV: Probing classification results (mean, standard deviation, and 95% confidence interval) on the Essays and Kaggle datasets using two probing models (Logistic Regression and SVM) over embeddings derived from the ADAM transformer encoder and OpenAI’s text-embedding-ada-002 model.

Panel of Figure 10 — Fig. 6: Sankey diagrams illustrating the probing classifier results on CLAD embeddings for four augmented languages ( fr , bm , jp , zh ) using Essays Dataset ( 6(a) ) and Kaggle Dataset ( 6(b) ). The flow proceeds from language groups to personality types (middle layer), and finally to the binary classification outputs: TOGA and PIGA. The probing classifier is implemented using logistic regression with L2 regularization and trained with binary cross-entropy (BCE) loss. No clear separability trend is observed across both datasets, suggesting the absence of strong label-specific signals within the embeddings.

Panel of Figure 11 — Fig. 7: Generalizable zero-shot cross-lingual evaluation of the model trained with weighted BCE and ADAM (CLAD) using PIGA augmentation. The model is trained on four languages and evaluated on a fifth (unseen) language across different language combinations. ADAM (CLAD) demonstrates statistically better generalization performance.

Panel of Figure 12 — Fig. 8: Balanced Accuracy comparison of CLAD and its ablation components across five languages ( en , fr , bm , jp , zh ) on the Essays dataset. The full CLAD model consistently outperforms all ablations—(i) KL only, (ii) AL only, (iii) CL only, (iv) AL + CL, (v) AL + KL, (vi) CL + KL—demonstrating the effectiveness of integrating all components for multilingual personality recognition.

Panel of Figure 13 — Fig. 9: Balanced Accuracy comparison of CLAD and its ablation components across five languages ( en , fr , bm , jp , zh ) on the Kaggle dataset. The full CLAD model consistently outperforms all ablations—(i) KL only, (ii) AL only, (iii) CL only, (iv) AL + CL, (v) AL + KL, (vi) CL + KL—demonstrating the effectiveness of integrating all components for multilingual personality recognition.

Model/Approach	O		C		E		A		N		Average
Model/Approach	BA	F1	BA	F1	BA	F1	BA	F1	BA	F1	BA	F1
multilingual-e5-large-instruct [ 55 ]	0.5977	0.5617	0.5160	0.5100	0.5516	0.5783	0.5209	0.5612	0.5231	0.5542	0.5419	0.5531
Qwen3-Embedding-0.6B [ 56 ]	0.5842	0.5559	0.5170	0.5179	0.5621	0.5946	0.5225	0.5658	0.5291	0.5710	0.5430	0.5610
embeddinggemma-300m [ 57 ]	0.5965	0.5729	0.5156	0.5111	0.5426	0.5768	0.5163	0.5582	0.5247	0.5552	0.5391	0.5548
LaBSE [ 58 ]	0.5906	0.5636	0.5397	0.5392	0.5620	0.5955	0.5271	0.5598	0.5227	0.5669	0.5484	0.5650
bge-m3 [ 59 ]	0.6221	0.5922	0.5444	0.5372	0.5577	0.5887	0.5631	0.5952	0.5648	0.5921	0.5704	0.5811

TABLE V: Benchmark comparison of different encoder backbone models trained with weighted binary cross-entropy (BCE) loss and the proposed ADAM (CLAD) model on the Essays dataset.

Model/Approach	O		C		E		A		Average
Model/Approach	BA	F1	BA	F1	BA	F1	BA	F1	BA	F1
multilingual-e5-large-instruct [ 55 ]	0.6614	0.8496	0.6445	0.5700	0.6785	0.5194	0.7496	0.7600	0.6835	0.6748
Qwen3-Embedding-0.6B [ 56 ]	0.6390	0.8470	0.6388	0.5632	0.6622	0.4864	0.7526	0.7634	0.6732	0.6650
embeddinggemma-300m	0.6450	0.8466	0.6492	0.5773	0.6702	0.5029	0.7249	0.7366	0.6723	0.6659
LaBSE [ 58 ]	0.6785	0.8650	0.6563	0.5830	0.6696	0.5013	0.7384	0.7493	0.6857	0.6746
bge-m3 [ 59 ]	0.6853	0.8653	0.6649	0.5964	0.6863	0.5353	0.7575	0.7670	0.6985	0.6910

TABLE VI: Benchmark comparison of different encoder backbone models trained with weighted binary cross-entropy (BCE) loss and the proposed ADAM (CLAD) model on the Kaggle dataset.

Explore similar work

Creating Multilingual Mental Health Dialogue Datasets: Limits of Persona-Based Localization via Nationality and Language
Jun 17, 2026Yunkai Xu, Saeed AbdullahMultilingual Language Model Evaluation Depression Detection
Pennsylvania State University, United States
Is AI Catching Up to Human Expression? Exploring Emotion, Personality, Authorship, and Linguistic Style in English and Arabic with Six Large Language Models
Mar 24, 2026Nasser A AlsadhanEmotion Representation in Language Models AI-Generated Text Detection
College of Computer and Information Sciences, King Saud University, Riyadh, Saudi Arabia
Large-Language-Models-as-a-Judge in Theory-Agnostic Adaptive Metric-Alignment for Prototypical Networks in Personality Recognition
Jul 9, 2026Jing Jie Tan, Ban-Hoe Kwan, Danny Wee-Kiat Ng +6Personality Modeling in Language Models Metric Learning
Department of Mechatronics and Biomedical Engineering, Lee Kong Chian Faculty of Engineering and Science, Universiti Tunku Abdul Rahman, Malaysia · Institute of Communication Studies, National Yang Ming Chiao Tung University, Taiwan · Institute of Information Management, National Yang Ming Chiao Tung University, Taiwan +2

Jun 17, 2026J/K move · Enter open · S save

Creating Multilingual Mental Health Dialogue Datasets: Limits of Persona-Based Localization via Nationality and Language

Yunkai Xu, Saeed Abdullah

Pennsylvania State University, United States

Multilingual Language Model Evaluation Depression Detection Depression Severity Estimation Synthetic Data Generation for Language Models Clinical Language Model Evaluation

Open paper AskPDF arXiv

AI and large language models (LLMs) have emerged as promising tools to address global mental health challenges. Despite the global nature of these challenges, there remains a critical shortage of high-quality datasets for training and evaluating such systems. To mitigate this gap, researchers increasingly generate synthetic clinical personas to simulate user data and test digital mental health support systems. However, most validated personas rely on English-centric contexts. This paper investigates whether similar persona-based methods can be used to generate multilingual mental health datasets. We modified nationality and language parameters in personas to generate clinical dialogues in Mandarin, Bengali, and Hindi. We then examined how different LLMs perform when evaluating the depression severity of these generated multilingual datasets against the baseline in English. Our findings indicate that just adding nationality and language parameters in personas might not be adequate, as it can introduce clinical inconsistency across languages. LLM judge models often exhibit inaccuracies in assessing depression severity in non-English texts, with performance varying across different models. This exposes the systemic limitations of applying English-centric personas to multilingual contexts. Ultimately, our work highlights the urgent need for culturally responsive data generation to ensure equitable mental health systems globally.

Abstract

Explore similar work

Creating Multilingual Mental Health Dialogue Datasets: Limits of Persona-Based Localization via Nationality and Language

Is AI Catching Up to Human Expression? Exploring Emotion, Personality, Authorship, and Linguistic Style in English and Arabic with Six Large Language Models

Large-Language-Models-as-a-Judge in Theory-Agnostic Adaptive Metric-Alignment for Prototypical Networks in Personality Recognition