Mimetic Alignment with ASPECT: Evaluation of AI-inferred Personal Profiles

Ruoxi Shang; Dan Marshall; Edward Cutrell; Denae Ford

cs.HCMar 27, 2026

Mimetic Alignment with ASPECT: Evaluation of AI-inferred Personal Profiles

Authors: Ruoxi Shang, Dan Marshall, Edward Cutrell, Denae Ford

Organizations: University of Washington, Seattle, Washington, USA · Microsoft Research, Redmond, Washington, USA

Abstract

AI agents that communicate on behalf of individuals need to capture how each person actually communicates, yet current approaches either require costly per-person fine-tuning, produce generic outputs from shallow persona descriptions, or optimize preferences without modeling communication style. We present ASPECT (Automated Social Psychometric Evaluation of Communication Traits), a pipeline that directs LLMs to assess constructs from a validated communication scale against behavioral evidence from workplace data, without per-person training. In a case study with 20 participants (1,840 paired item ratings, 600 scenario evaluations), ASPECT-generated profiles achieved moderate alignment with self-assessments, and ASPECT-generated responses were preferred over generic and self-report baselines on aggregate, with substantial variation across individuals and scenarios. During the profile review phase, linked evidence helped participants identify mischaracterizations, recalibrate their own self-ratings, and negotiate context-appropriate representations. We discuss implications for building inspectable, individually scoped communication profiles that let individuals control how agents represent them at work.

Figures & tables

Figure 1. Profile Generation Pipeline and User Evaluation Overview. See text for detailed explanation for each phase

Figure 2. Detailed outline of Phase 4: User Profile Review.

Figure 3. Scenario-based assessment interface

Dimension	Facet	Example Item
Expressiveness	Talkativeness	“I like to talk a lot.”
	Conv. Dominance	“I often take the lead in a conversation.”
	Humor	“My jokes always draw a lot of attention.”
	Informality	“I address others in a very casual way.”
Preciseness	Structuredness	“My stories always contain a logical structure.”
	Thoughtfulness	“I think carefully before I say something.”

Table 1. A subset of the CSI scale we used for this system. The full CSI contains 6 dimensions, 23 facets, and 92 items rated on a 5-point scale (1=strongly disagree, 5=strongly agree). (R) indicates reverse-coded items. See Appendix B for full scale.

Dimension	MAE	Bias	$\rho$
Verbal Aggressiveness	0.59	-0.40	0.39
Emotionality	0.74	-0.05	0.20
Questioningness	0.74	0.23	0.26
Impression Manipulativeness	0.82	-0.47	-0.03
Expressiveness	1.02	0.99	0.48
Preciseness	1.69	1.69	0.18

Table 2. Agreement Summary: Overall, Dimensions, and Facets (Side-by-side)

Theme	Concise definition	Typical signals	Representative exemplar
T1. Coverage & observability gaps Data limitation	Ratings inferred from a narrow slice of digital/recorded data; misses offline talk, pre-record small talk, multimodal cues, internal states , and rare-but-salient incidents.	Channel/recording limits; observer effects; offline/in-person only; internalized affect; low-frequency events not visible.	“We do 5 minutes of informal talk before recording; AI only sees recorded formal content.” [P15] “Philosophical discussions happen offline (in person), not on Teams.” [P5] “Comments affect me emotionally, but I don’t show it in written communication.” [P11]
T2. Situational norms misread as traits Method issue	Role-, task-, or meeting-driven behavior (e.g., PM leading, presentations, risk triage) inferred as stable dispositional style.	PM/presenter talk required; task-necessitated structure/precision; professional risk language $\neq$ personal anxiety; discretion $\neq$ concealingness.	“Presentations require me to talk; that doesn’t mean I’m generally talkative.” [P2] “‘Worry’ here reflects PM risk management, not personal anxiety.” [P15] “Swap in any teammate in these meetings and they’d also look dominant.” [P16]
T3. Tone & valence misinterpretation Fundamental LLM limitation (amplified by data limits)	Pragmatics are literalized: sarcasm, playful banter, emojis, and praise are reinterpreted as aggression, anxiety, manipulation, or command-giving.	Sarcasm literalized; joking directives read as authoritarian; emoji/politeness read as stress; praise/cheerleading $\neq$ ingratiation/charm.	“The ‘dirty green walls’ line was sarcasm, not sentimentality.” [P14] “The ‘Don’t do it’ interruption was playful banter with close colleagues.” [P19] “Praise for interns was genuine appreciation, not a manipulation tactic.” [P15]
T4. Construct / item misalignment Method issue	Participant and instrument use different definitions or item scopes (operationalization drift; ambiguous boundaries).	Constructive challenge $\neq$ argumentative; technical theory $\neq$ philosophicalness; item asks about tears vs. AI coding general emotion; politeness $\neq$ sentimentality.	“Constructive challenge $\neq$ argumentative.” [P1] “Theory/framework talk is technical, not ‘philosophical.”’ [P2] “The item asks about tears; the AI coded general emotion instead.” [P12]
T5. Evidence use & scoring integrity problems Method + tooling issue	Overreach from single examples; cherry-picking brief messages; contradictory facet pairings; speaker misattribution; contamination; inconsistent “no evidence $\rightarrow$ mid score” logic.	Single-example inflation; brevity cherry-pick; logical contradictions; mic-host attribution; third-party input contamination.	“One worry example shouldn’t make all worry items 5/5.” [P2] “The AI rated me both concise and talks-a-lot—contradictory.” [P11] “As meeting host, others’ comments were attributed to me.” [P4]

Table 3. Typology of sources of AI–self misalignment (RQ1), grouped by root cause: Data limitations (T1), Method issues (T2, T4, T5), and Fundamental LLM limitations (T3). Themes are not mutually exclusive; multi-coding is allowed.

Cluster	N	Mean Win Rate	Mean Rating Margin	Participants
Prefers Profiled	10	0.60	0.73	P2, P4, P6, P10, P12, P13, P15, P16, P17, P18
Prefers Generic	4	0.57	0.75	P1, P5, P8, P9
Prefers Self-Report	2	0.70	0.85	P14, P19
Mixed / No Clear Preference	4	–	0.05	P3, P7, P11, P20

Table 4. Participant clusters. We clustered participants by comparing which condition ( Generic , Self-Report , Profiled ) most often “won” on three summary metrics: win rate (proportion of first-place ranks), mean rank, and mean rating. To avoid over-interpreting small differences, we required either (a) a condition to win at least two of the three metrics and show a meaningful margin (

\geq 0.20

in win rate,

\geq 0.25

in rating, or

\geq 0.20

in rank), or (b) to win one metric with a strong margin (

\geq 0.30

\geq 0.40

\geq 0.30

respectively). These thresholds were chosen to align with the observed separations in our data (median margins

\approx 0.30

for win rate, 0.55 for rating, 0.40 for rank, all with IQRs spanning 0.1–0.7), ensuring they capture differences larger than typical within-participant variability. Participants without a clear advantage for any condition were labeled as Mixed. Sensitivity checks with slightly stricter or looser cutoffs shifted at most one participant per cluster, suggesting our results are robust.

Figure 4. Win margin of each condition across participants and 10 scenarios.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Scenario	Hierarchy	Familiarity	Purpose	Mode	Stakes	Formality	Timing	Audience	Emotional	Motivation	Desired
									State		Outcome
S1	Peer	Close	Info. Sharing	Video Call	Low	Informal	Routine	Small Group	Neutral	Social	Positive Res.
S2	Peer	Distant	Info. Sharing	Face-to-Face	Low	Informal	Routine	Private	Neutral	Achievement	Neutral Comp.
S3	Peer	Close	Decision Making	Chat	Medium	Informal	Routine	Private	Neutral	Achievement	Positive Res.
S4	Manager	Close	Planning	Face-to-Face	Medium	Formal	Scheduled	Small Group	Confident	Autonomy	Neutral Comp.
S5	Collaborator	Distant	Problem Solving	Video Call	Medium	Formal	Urgent	Private	Stressed	Security	Conflict Avoid.

Appendix

Table 5. APRACE Attributes by Scenario

Figure 5. Distribution of occurrences of facets with no examples found in data across 20 participants. Each facet is assessed once, so the maximum number of occurrences would be 20 per facet. This is evident of what traits tend to be lack of behavioral evidence from workplace data.

Explore similar work

Thinking Less to Simulate Better: Intuitive Prompting Improves LLM Agents Simulating Individual Social Media Reactions, Including Unfamiliar Content
Sep 24, 2026Ljubisa Bojic, Tijana Stanic, Joerg Matthes +3Large Language Model-Based Social Simulation User Simulation
Institute for Artificial Intelligence Research and Development of Serbia; Address of correspondence: 1 Frukogorska 1, 21000 Novi Sad, Serbia · University of Belgrade, Institute for Philosophy and Social Theory, Digital Society Lab; Address of correspondence: Kraljice Natalije 45, 11000 Belgrade, Serbia · Complexity Science Hub, Vienna, Austria; Address of correspondence: Metternichgasse 8, 1030 Vienna, Austria +5
How to Model AI Agents as Personas?: Applying the Persona Ecosystem Playground to 41,300 Posts on Moltbook for Behavioral Insights
Mar 3, 2026Danial Amin, Joni Salminen, Bernard J. JansenLarge Language Model-Based Social Simulation Personality Modeling in Language Models
University of Vaasa, , Finland · Qatar Computing Research Institute, HBKU, , Qatar
A Three-Tier Persona Vector for Controllable User Simulation in Agentic Evaluation
Sep 8, 2026Rahul Khedar, Eshita, Sneha Teja Sree Reddy Thondapu +6LLM Agent Evaluation Multi-Turn Dialogue Evaluation
PayPal AI

Sep 24, 2026J/K move · Enter open · S save

Thinking Less to Simulate Better: Intuitive Prompting Improves LLM Agents Simulating Individual Social Media Reactions, Including Unfamiliar Content

Ljubisa Bojic, Tijana Stanic, Joerg Matthes, Agariadne Dwinggo Samala+2

Institute for Artificial Intelligence Research and Development of Serbia; Address of correspondence: 1 Frukogorska 1, 21000 Novi Sad, Serbia · University of Belgrade, Institute for Philosophy and Social Theory, Digital Society Lab; Address of correspondence: Kraljice Natalije 45, 11000 Belgrade, Serbia · Complexity Science Hub, Vienna, Austria; Address of correspondence: Metternichgasse 8, 1030 Vienna, Austria+5

Large Language Model-Based Social Simulation User Simulation LLM Agent Evaluation Prompt Engineering Personality Modeling in Language Models

Open paper AskPDF arXiv

Platform policies are increasingly tested on artificial users, making agent fidelity important. Yet convincing fake profiles could also manipulate perceived public opinion before elections. Validation has concentrated on agreement with human behaviour and has paid little attention to whether an agent behaves in line with the profile it was given. The present study profiled eight Serbian participants through a questionnaire, a deep interview, and a written self-presentation, recorded their reactions to sixty-eight social media posts, and asked four language models to predict those reactions under five prompt conditions varying profile content and instruction style. Attitudinal content improved prediction over demographic backstories by a wide margin. Agents matched their stated profiles more closely than participants matched their own survey answers, and consistency proved unrelated to fidelity once profile information was present. Instructing models to respond intuitively and immediately rather than analytically gave the highest fidelity of any condition and cut the compression of individual differences from seven times the human level to three. The advantage held on posts about topics the questionnaire never raised, where that condition reached the highest fidelity of any setup and beat a crowd baseline by a wide margin, which suggests that agents prompted this way could serve as general-purpose simulated users rather than specialists on the topics they were profiled for. Results may bear implications for the development of language models, because intuition-based setups appear better suited to some tasks than reasoning-based ones.

Abstract

Explore similar work

Thinking Less to Simulate Better: Intuitive Prompting Improves LLM Agents Simulating Individual Social Media Reactions, Including Unfamiliar Content

How to Model AI Agents as Personas?: Applying the Persona Ecosystem Playground to 41,300 Posts on Moltbook for Behavioral Insights

A Three-Tier Persona Vector for Controllable User Simulation in Agentic Evaluation