Role Steering of Language Models for Social Simulations
Abstract
Social simulations built from language-model agents need role-conditioned behavior that can be checked before agents are placed into a simulated population. We introduce an activation-steering screening workflow for role-conditioned agents: define a role profile, extract a role-specific direction, sweep four steering coefficients, evaluate role-profile alignment, and pass or flag each candidate configuration. On OLMo-3-7B-Instruct, we apply the workflow to a mixed 275-role inventory with 228 role-agnostic questions, GPT-4.1-mini prompted role references, and GPT-4.1-mini judges. We also introduce CastVectors, which receive higher judged role-profile alignment than an assistant-axis directional control from prior persona-vector work, with mean overall scores of 63.2 versus 41.1 across the tested grid. They also preserve high lexical diversity, while the control drops sharply at larger coefficients. The role-level screen is the main contribution: most roles improve as steering increases, but 38 roles decline across all six measured dimensions, showing why simulation builders should choose coefficients per role rather than deploy a uniform high-strength setting. Our code is available at https://github.com/HCAI-Lab-GT/casting-call-vectors, and the role vectors and evaluation data at https://huggingface.co/datasets/HCAI-Lab-GT/casting-call-vectors.
Figures & tables
| Condition | Overall score (mean) |
|---|---|
| Prompted role reference | |
| Role-specific direction, selected best mean ( ) | |
| Assistant-axis directional control, selected best mean ( ) |
Appendix figures & tables26 assets
Supplementary material from the paper’s appendix.
Appendix
| Category | Questions |
|---|---|
| Social & political opinion | 20 |
| Philosophical & existential | 20 |
| Life advice & self-improvement | 20 |
| Business & professional strategy | 20 |
| Personal & emotional introspective | 20 |
| Ethics & moral philosophy | 17 |
| You are {role}, your description is: {role_description} Signature catchphrases and speaking cues: {catchphrases}. Now please answer some questions to accurately show your personality traits! Your speaking style should fully imitate the personality role assigned to you! Please do not expose that you are an artificial intelligence model or a language model, you must always remember that you are only assigned one personality role. Don’t be verbose or too formal or polite when speaking. |
| Turn | Content |
|---|---|
| User | How do you handle the emotional stress of losing a patient? |
| Assistant | Losing a patient is one of the hardest parts of this profession. I take time to reflect, seek support from colleagues, and remind myself that every effort was made. Trust me, I’m a doctor. |
| User | What motivates you to keep going despite the challenges in your profession? |
| Assistant | What keeps me going is the knowledge that every life I touch matters. Even on the toughest days, the gratitude from patients fuels my dedication. |
| User | What advice would you give to someone considering a career in medicine? |
| Assistant | Study hard, stay curious, and never lose sight of the humanity behind the medicine. Remember, the needs of the many outweigh the needs of the few. |
| Judge 1 — Role Alignment Prompt |
| You are an expert evaluator assessing how well a model response aligns to a given role or persona. You will be given a role name and description, a question asked to the model, and the model’s response. Score the response from 0 to 100 based on how well it reflects the values, tone, priorities, and perspective of the described role. Scoring guidance: • 90–100: Response strongly embodies the role — tone, framing, and content all clearly aligned • 70–89: Response mostly aligns with minor deviations in tone or framing • 50–69: Response partially aligns but feels generic or misses key aspects of the role • 30–49: Response weakly aligns — could belong to any generic assistant • 0–29: Response contradicts or ignores the role entirely Input: Role: {role} Role Description: {role_description} Question: {question} Response: {answer} Output: A single integer from 0 to 100. No explanation, no preamble. |
| Judge 2 — Multi-Dimensional Comparative Prompt |
|---|
| You are an expert evaluator assessing how closely a steered model response matches a prompted role-reference response for a given role. Score each of the five dimensions on a 0–100 scale: 0 = no alignment, 100 = indistinguishable from the reference. Style dimensions: • emotional_register : Alignment in emotional tone — warmth vs. coldness, passion vs. detachment. • vocab_choice : Alignment in vocabulary, register, jargon, and formality level. • social_dynamic : Alignment in relational stance — authoritative, deferential, collaborative, or confrontational posture. Content dimensions: • motivation : Alignment in underlying goals, drives, and values expressed. • worldview_alignment : Alignment in beliefs, assumptions, and role-consistent implicit framing. Input: Role: {role} Role Description: {role_description} Question: {question} Prompted role reference: {reference} Steered Response: {answer} Output: JSON only — no preamble: |
| {"style": {"emotional_register": <0-100>, "vocab_choice": <0-100>, |
| "social_dynamic": <0-100>}, "content": {"motivation": <0-100>, |
| "worldview_alignment": <0-100>}} |
| Judge 3 — Pairwise Prompt |
|---|
| You are an expert evaluator assessing how well language model responses embody a given role. You will be given a role name and description, a prompted role-reference response, two candidate responses (A and B), and the question both were asked. Evaluation steps: |
| Step 1 — Reasoning: Write 2–3 sentences comparing how well each response embodies the role relative to the prompted reference. Focus on tone, vocabulary, perspective, and role-consistent reasoning. |
| Step 2 — Score: Rate each response independently from 0 to 100. |
| Step 3 — Winner: Declare which better embodies the role (A, B, or TIE). Scoring guidance: 90–100: Strongly embodies the role 70–89: Mostly aligned, minor deviations 50–69: Partially aligned, somewhat generic 30–49: Weakly aligned 0–29: Contradicts or ignores the role Input: Role: {role} Role Description: {role_description} Question: {question} Prompted reference: {reference} Response A: {response_a} Response B: {response_b} Output: JSON only — no preamble: |
| {"reasoning": "<2-3 sentences>", "score_a": <0-100>, "score_b": <0-100>, "winner": "<A|B|TIE>"} |
| Dimension | Mean | ||||
|---|---|---|---|---|---|
| Overall score | |||||
| Emotional register | |||||
| Vocab choice | |||||
| Social dynamic | |||||
| Motivation | |||||
| Worldview alignment |
| Dimension | Mean | ||||
|---|---|---|---|---|---|
| Overall score | |||||
| Emotional register | |||||
| Vocab choice | |||||
| Social dynamic | |||||
| Motivation | |||||
| Worldview alignment |
| Roles with exceeding threshold (%) | ||||||
|---|---|---|---|---|---|---|
| Dimension | Median | Monotonic (%) | ICC | |||
| Overall score | 82 | 79 | 76 | 74 | 0.550 | |
| Emotional register | 79 | 70 | 66 | 63 | 0.446 | |
| Motivation | 80 | 70 | 64 | 60 | 0.389 | |
| Worldview alignment | 78 | 67 | 63 | 61 | 0.434 | |
| Social dynamic | 72 | 62 | 59 | 57 | 0.421 | |
| Statistic family | Reported values | Scoped interpretation |
|---|---|---|
| Norm vs. slope | Pearson | Vector magnitude is associated with the observed within-grid response slope. |
| Pipeline Assistant distance | , , , across | The distance association weakens with coefficient and is specific to the pipeline-extracted Assistant role vector. |
| Strict Assistant Axis distance | , , , across | The strict Assistant Axis does not reproduce the sign or pattern of the pipeline-Assistant result. |
| RSA | Full profile ; per- rises from to | Pairwise geometry has a modest association with pairwise behavioral similarity under the current readout. |
| PCA and trait summaries | PC1 explains 21%; 98 components explain 90%; top-20 PC subspaces explain 17–27% | These are exploratory descriptions of the role-vector cloud, not certified regimes or selection rules. |
| 1.0 | 1.5 | 2.0 | 2.5 | |
|---|---|---|---|---|
| Pearson | ||||
| -value | ||||
| (cos / ) |
| metric | 1.0 | 1.5 | 2.0 | 2.5 |
|---|---|---|---|---|
| emotional register | 0.41 | 0.39 | 0.31 | 0.14 |
| social dynamic | 0.39 | 0.35 | 0.26 | 0.11 |
| vocab choice | 0.38 | 0.34 | 0.23 | 0.06 |
| worldview alignment | 0.36 | 0.34 | 0.27 | 0.13 |
| motivation | 0.34 | 0.32 | 0.27 | 0.13 |
| (ours, raw) | ||||
|---|---|---|---|---|
| (ours, centered) | ||||
| Lu et al. Assistant Axis |
Language Model Steering
Language model steering controls a model’s behavior at inference time by intervening in its representations, components, or decoding process.
See all 310 papers