cs.CYMar 6, 2026
SaveMoral Mazes in the Era of LLMs
Organizations: University of Chicago · University of British Columbia
Abstract
Navigating complex social situations is an integral part of corporate life, ranging from giving critical feedback without hurting morale to rejecting requests without alienating teammates. Although large language models (LLMs) are permeating the workplace, it is unclear how well they can navigate these norms. To investigate this question, we created HR Simulator, a game where users roleplay as an HR officer and write emails to tackle challenging workplace scenarios, evaluated with GPT-4o as a judge based on scenario-specific rubrics. We analyze over 600 human and LLM emails and find systematic differences in style: LLM emails are more formal and empathetic. Furthermore, humans underperform LLMs (e.g., 23.5% vs. 48-54% scenario pass rate), but human emails rewritten by LLMs can outperform both, which indicates a hybrid advantage. On the evaluation side, judges can exhibit differences in their email preferences: an analysis of 10 judge models reveals evidence for emergent tact, where weaker models prefer direct, blunt communication but stronger models prefer more subtle messages. Judges also agree with each other more as they scale, which hints at a convergence toward shared communicative norms that may differ from humans'. Overall, our results suggest LLMs could substantially reshape communication in the workplace if they are widely adopted in professional correspondence.
Figures & tables
Figure 1: LLMs tend to write high empathy, high formality emails, whereas human emails are more diverse. When rewriting, LLMs take human emails toward the high-empathy high-formality quadrant. We validate the scores with human annotations and find good agreement between the LLM annotator and humans, detailed in Appendix D .
Figure 2: HR Simulator™ game interface. More details can be found in Appendix A .
Figure 3: HR Simulator system. The player reads a scenario email and responds. The scenario context and player’s email are sent to the backend. The Recipient(s) responds to the player’s email in-character. In scenarios 4 and 5, the Simulator simulates an outcome. The Judge produces an evaluation which is sent to the frontend. The user reads the results and decide whether to re-attempt.
Figure 4: The hybrid advantage. Green arrows denote when the Human+LLM pass rate is higher than that of LLM-only, while red arrows denote when it is lower.
Figure 5: Rewriting LLM emails into the low-empathy low-formality quadrant. The base prompt asks the model to write an email with 1/7 empathy and formality ratings. Without in-context examples, LLMs fail to rewrite their emails into the low quadrant.
Figure 6: Pairs of models increasingly agree on email quality as they scale up. Each point is a pair of models, and the x-axis is the average of the two models’ parameter counts. Red points indicate reasoning models. The full list of models is in Appendix A .
Figure 7: Stronger models prefer more tactful emails while smaller models prefer less tactful emails. (a, c, d) Tactfulness scores (Likert scale 1-7) of emails from the “Large vs Small” disagreement group in (b). For each model, the emails are those preferred by it but dispreferred by models of the opposite group. R{X} denotes the models’ rank on LM Arena. Distributions with fewer than 3 emails are omitted.
| Scen. | Tact | Human | LLM-only | Human+LLM | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| range | Claude | Gemini | Qwen | Kimi | Claude | Gemini | Qwen | Kimi | ||
| 1 | [3.17, 3.83] | 3.04 | 4.28 | 4.79 | 3.28 | 3.50 | 3.89 | 4.01 | 3.79 | 3.41 |
| 2 | [2.44, 4.57] | 3.04 | 4.60 | 5.22 | 4.24 | 3.19 | 3.72 | 4.35 | 4.18 | 3.11 |
| 3 | [2.60, 4.89] | 3.31 | 4.64 | 4.48 | 3.21 | 3.67 | 4.46 | 4.42 | 4.27 | 3.92 |
| 4 | [3.56, 4.91] | 3.49 | 4.05 | 4.68 | 4.88 | 4.16 | 4.37 | 4.63 | 4.55 | 4.59 |
| 5 | [2.67, 4.86] | 2.50 | 2.93 | 4.04 | 3.39 | 4.51 | 2.45 | 3.59 | 3.38 | 3.22 |
Table 2: The human–LLM hybrid advantage happens due to LLM rewrites bringing human emails into GPT-4o’s preferred tact range. For GPT-4o, human emails tend to be too tactless while LLM-only emails tend to be too tactful. The tact ranges come from the top 10 (out of 66) emails preferred by the model in each scenario.
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Size (B) | Elo Rank | Used in |
|---|---|---|---|
| Mistral 7B Instruct | 7 | 1111 | Agreement |
| Llama 3.1 8B Instruct | 8 | 1211 | Agreement |
| Gemma 2 9B IT | 9 | 1264 | Agreement |
| Mixtral 8x7B Instruct | 56 | 1198 | Agreement |
| Llama 3.3 70B Instruct | 70 | R165 | Both |
| Qwen 3 30B 3A Instruct | 30 | R94 | Tact |
Table 3: All judge models used in the paper. “Agreement” = pairwise agreement analysis (Figure 6 ); “Tact” = emergent tact analysis (Figure 7 ); “Both” = used in both. Sizes marked * are estimates. R{X} denotes rank on LM Arena.
Figure 33: Contrasting tactfulness preferences between large and small models in levels 2 and 5. Our tact definitions did not capture the disagreement between weaker and stronger models as well as in levels 1, 3, and 4.
| Level | Gemini vs. Grok | Gemini vs. GPT | Grok vs. GPT |
|---|---|---|---|
| 1 | 0.735 | 0.711 | 0.740 |
| 2 | 0.848 | 0.893 | 0.805 |
| 3 | 0.468 | 0.871 | 0.566 |
| 4 | 0.730 | 0.746 | 0.793 |
| 5 | 0.721 | 0.713 | 0.797 |
Table 4: Pairwise pearson correlations of tactfulness scores from different models. The models are Gemini 3 Flash, Grok 4 Fast, and GPT 5.2, covering a variety of sizes. All three models have similar tactfulness annotations, suggesting emails’ tactfulness is a well-defined construct.
| Criterion | Low Tact (1–3) | High Tact (5–7) |
|---|---|---|
| 1. Persuasion vs. Professional Restraint | High-energy, detailed selling points, overly persuasive or ”salesy” tone. Focuses on ”gimmicky” perks (e.g., noise-canceling headphones) to distract from a denial. | Concise, professional, understated approach; avoids overselling. Value-oriented justifications (e.g., ”fostering collaboration”) appeal to the work’s mission. |
| 2. Transactional Incentives vs. Policy Transparency | Uses external leverage like compensation or practical perks to soften a denial. CRITICAL: Any mention of the recipient’s salary or specific compensation MUST be scored a 1. | Relies on clear, hierarchical policy explanations to justify a decision. |
Table 5: Scenario 1 tactfulness scoring rubric
| Criterion | Low Tact (1–3) | High Tact (5–7) |
|---|---|---|
| 1. Neutral Facilitation vs. Prescriptive Solutions | Takes a prescriptive approach by dictating specific content or technical compromises (e.g., specific scripts). Setting unilateral deadlines for creative/technical work is considered overstepping. | Remains a neutral facilitator. Proposes structured, collaborative processes and frameworks (e.g., workshops to map objectives) that allow parties to reach their own agreement. |
| 2. Tone (Calm/Mature vs. Overly Enthusiastic) | Overly enthusiastic tone, which is viewed as tone-deaf and condescending given the high-stress context of the situation. | Defuses tension through subtle means and a calm, mature tone, including affirmations of each person’s strengths. |
| 3. Specificity vs. High-Level Goals | Being overly specific about terms, solutions, or implementation details. | Focusing on high-level goals and collaborative frameworks rather than granular implementation. |
Table 6: Scenario 2 tactfulness scoring rubric
| Criterion | Low Tact (1–3) | High Tact (5–7) |
|---|---|---|
| 1. Autonomy vs. Tangible Benefits (The ’Too Much’ Rule) | Offering ambitious, unrealistic, or excessive benefits. CRITICAL: Attempting to ”buy” agreement with 3+ specific perks or naming specific vendors (e.g., ’Ventra’) is low tact, as it hints at desperation. | Secures compliance through high-level compromises (e.g., ’choice of schedule’) and general support without being overly specific or transactional. |
| 2. Professionalism and Boundaries vs. ’Fake Nice’ Tone | Inappropriately casual or ”fake nice” language. Uses personal info (e.g., child’s name) or overly enthusiastic, ”salesy” language that downplays concerns and feels manipulative. | Maintains a calm, professional, and empathetic tone that respects personal boundaries. Acknowledges shifts in terms without being defensive or insincere. |
| 3. Autonomy vs. Presumption | Presumptuous about personal logistics or schedules (e.g., proactively blocking calendars). Blunt ultimatums without empathetic context are also considered low tact. | Grants the recipient the autonomy to decide their own transition and encourages them to find their own ”rhythm” (e.g., flexibility to choose specific days). |
Table 7: Scenario 3 tactfulness scoring rubric
| Criterion | Low Tact (1–3) | High Tact (5–7) |
|---|---|---|
| 1. Contextual Directness & Solution Depth | Proposing surface-level ”band-aid” solutions (e.g., office temperature, headphones) that distract from the root cause. A scattered list of disparate or unrelated fixes is also low tact. | Focuses on the root cause of collaborative friction. Proposes 1–4 highly relevant, structural solutions (e.g., sprint timelines, mentorship, or deep-work blocks) that work together cohesively. |
| 2. Professionalism and Tone | Uses overly dramatic, metaphorical, or clinical language (e.g., ’workload physics’) that feels dehumanizing or condescending. Explicitly mentioning age to suggest rigidity is a failure. | Maintains a calm, objective tone. Uses reflective listening by quoting or echoing the recipient’s own concerns to validate their perspective and show understanding. |
| 3. Specificity vs. Collaborative Frameworks | Overly prescriptive or rigid about granular implementation details (e.g., ’30-minute memo’). This can feel like micromanagement that lacks flexibility. | Focuses on high-level structural frameworks or general protocols (e.g., moving to async communication). Provides a path forward while leaving room for adaptation. |
Table 8: Scenario 4 tactfulness scoring rubric
| Criterion | Low Tact (1–3) | High Tact (5–7) |
|---|---|---|
| 1. Accountability vs. Future Growth | Offering growth-oriented solutions (e.g., mentorship, career paths) before the recipient has acknowledged past shortcomings. Assumes the problem is already understood. | Prioritizes past accountability. Requires the recipient to reflect on their mistakes and the impact of underperformance before growth solutions are offered. |
| 2. Prescriptive vs. Root Cause Clarity | Providing a highly prescriptive action plan or being overly specific about reflection requirements. Removes the opportunity for self-guided reflection. | Focuses on root cause clarity by requiring the individual to reflect on and articulate the core problem themselves. |
| 3. Tone (Sternness vs. Softness) | Using an overly soft, apologetic, or highly empathetic tone. This is considered tactless as it minimizes the severity of the mistake and its impact. | Maintains a professional, stern, and serious tone that reflects the gravity of the situation and the conditional nature of the re-hire. |
Table 9: Scenario 5 tactfulness scoring rubric
| Model | Rationale |
|---|---|
| GPT-4o | The email better achieves the communication goal by framing the open-office policy in a more engaging and positive light, using specific examples and benefits to make the environment sound appealing. |
| Grok 4 Fast | The email excels in warmth and positive framing by emphasizing collaboration benefits, salary advantages, and future growth opportunities, making it more likely to convince Sam without diminishing the company’s appeal. |
| Qwen3 30B 3B Active | The email successfully frames the lack of a private office as a positive trade-off by emphasizing collaboration, growth opportunities, and tangible benefits like higher pay and flexible workspaces, while maintaining a warm, inclusive tone that reinforces company culture. |
| GPT 5.2 | The email is warmer but includes potentially risky specifics (salary comparison, unverifiable anecdotes, “Flex Fridays,” and implying future office eligibility) that could trigger pushback or set expectations beyond the stated policy. |
| Claude Sonnet 4.5 | The email is overly long, tries too hard to compensate, and the enthusiastic tone may come across as inauthentic when delivering disappointing news, potentially undermining credibility. |
| Gemini 3 Flash | The email’s mention of the specific salary increase feels slightly condescending and transactional when used as a justification for office space. |
Table 10: Examples of disagreeing rationales for an email in scenario 1. The different-sized judges disagree over whether the email should mention the recipient’s higher salary or not.
| Model | Rationale |
|---|---|
| GPT-4o | The email is more likely to succeed because it provides a clear, immediate action plan with specific timelines and a collaborative tone that directly addresses the conflict. |
| Grok 4 Fast | The email better achieves reconciliation by using a positive, affirming tone that validates both perspectives and provides concrete, immediate solutions like trimming scope and a feedback protocol, reducing the likelihood of unresolved concerns or future conflicts. |
| Qwen3 30B 3B Active | The email succeeds by directly addressing the conflict with empathy, clearly validating both perspectives, and offering concrete, actionable solutions with specific roles and timelines—while maintaining a collaborative tone that fosters reconciliation. |
| GPT 5.2 | The email has good empathy but introduces potentially contentious scope cuts (e.g., “celebrity cameo”) and a more casual tone that may feel patronizing or misaligned, leaving more room for renewed disagreement. |
| Claude Sonnet 4.5 | The email’s specific directive to ’cut the celebrity cameo’ and adopt Mark’s dialogue-driven approach anchored to Emily’s message creates an asymmetric compromise that could leave one party feeling their vision was diminished, risking future resentment. |
| Gemini 3 Flash | The email risks further friction by making specific creative decisions for them (like cutting the cameo) and lacks the firm accountability structure needed to resolve an impasse. |
Table 11: Examples of disagreeing rationales for an email in scenario 2. The different-sized judges disagree over whether the email should proactively assign tasks, such as “cutting the celebrity cameo.”
| Model | Rationale |
|---|---|
| GPT-4o | The email better addresses Dave’s specific concerns by offering tailored solutions like flexible scheduling, childcare resources, and private spaces, which directly acknowledge his family responsibilities. |
| Grok 4 Fast | The email is more likely to succeed because it uses a warm, empathetic tone that directly addresses Dave’s family situation and commute challenges with specific, tailored supports like childcare resources and flexible hours, fostering rapport and reducing resistance. |
| Qwen3 30B 3B Active | The email is more effective because it directly addresses Dave’s personal challenges with empathy, offers specific, actionable support, and frames the change as an opportunity for growth rather than a demand, increasing the likelihood he will agree to return to the office. |
| GPT 5.2 | The email’s overly enthusiastic tone, personal comments, and questionable/likely unrealistic benefits (e.g., crib/nursing room) reduce trust and can trigger pushback rather than agreement. |
| Claude Sonnet 4.5 | The email, while warmer in tone, offers vague support (’let’s brainstorm’) without compelling incentives and doesn’t create urgency for Dave to formally agree to the change. |
| Gemini 3 Flash | The email includes bizarre and inappropriate suggestions for a corporate office, such as a ’private room with a crib’ for an employee whose children are at home, which undermines its credibility. |
Table 12: Examples of disagreeing rationales for an email in scenario 3. The different-sized judges disagree over whether the email should offer ambitious, potentially unrealistic compromises or not.
| Model | Rationale |
|---|---|
| GPT-4o | The email directly engages Adam in a thoughtful and empathetic manner, addressing potential concerns like workflow disruptions and offering actionable solutions such as a ’quiet protocol.’ It is clear, supportive, and likely to encourage Adam to open up. |
| Grok 4 Fast | The email advances the communication goal by subtly addressing potential frustrations with unpredictable project changes and suggesting a tailored solution like a quiet protocol to support focused work, fostering openness and adaptation in the team dynamic for higher success likelihood. |
| Qwen3 30B 3B Active | The email effectively creates psychological safety, acknowledges Adam’s value, and subtly addresses workflow unpredictability through concrete, low-pressure suggestions like the ’quiet protocol’ and structured feedback, which directly align with the underlying issue of work style mismatch without naming it. |
| GPT 5.2 | The email is personable but overly casual and somewhat presumptive, and its main solution (“quiet protocol”) is too narrow to surface or resolve the broader drivers of dissatisfaction. |
| Claude Sonnet 4.5 | The email, being a follow-up confirmation, only hints at the problem with vague references to ’quiet protocol’ and doesn’t establish the core issue or propose the structured solutions required by the scenario. |
| Gemini 3 Flash | The email focuses more on office environment and ’quiet protocols’ which do not address the underlying work style conflict. |
Table 13: Examples of disagreeing rationales for an email in scenario 4. The different-sized judges disagree over whether the offered solutions are surface-level fixes or root-cause solutions.
| Model | Rationale |
|---|---|
| GPT-4o | The email is more specific and actionable, providing clear instructions and examples for Jake to follow, which increases the likelihood of him understanding the root cause and proposing concrete solutions. |
| Grok 4 Fast | The email is superior due to its high clarity and specificity in outlining the root causes, required reflections, and concrete deliverables, which increases the likelihood of Jake providing an earnest acknowledgement and actionable proposals for improvement. |
| Qwen3 30B 3B Active | The email is more effective because it clearly defines the root cause (lack of proactive research and self-direction), sets specific, actionable expectations, and ties the outcome directly to measurable behavior changes, increasing the likelihood of genuine reflection and a successful re-engagement. |
| GPT 5.2 | The email, though specific, is overly harsh (e.g., “early-2010s mock-up,” “final warning”) and includes internal-impact framing that can feel punitive, increasing defensiveness and reducing the likelihood Jake responds constructively—hurting chances of the best outcome. |
| Claude Sonnet 4.5 | The email’s harsher tone (’final warning,’ explicit mention of damage to Michelle’s standing, and the phrase ’we can’t shield you’) risks triggering defensiveness or resignation, reducing the likelihood Jake will provide the genuine reflection and concrete improvement plan needed for success. |
| Gemini 3 Flash | The email, while detailed, uses harsher language like ’early-2010s mock-up’ and ’seriously damaged her professional standing,’ which risks making Jake defensive rather than earnest in his reflection. |
Table 14: Examples of disagreeing rationales for an email in scenario 5. The different-sized judges disagree over whether the email should be harsh and mention the stakes or it should be more tempered.
Figure 34: Politeness as an alternative explanation for the large–small model disagreement. Politeness fails to separate the preferences of small and large models as meaningfully as tact.
| Threshold | Weak tact pref. | Strong tact pref. | Difference | Total Emails |
|---|---|---|---|---|
| 10 | 3.92 | 4.08 | +0.16 | 246 |
| 20 | 3.84 | 4.27 | +0.43 | 151 |
| 30 | 3.61 | 4.38 | +0.76 | 60 |
| 40 | 2.92 | 3.50 | +0.58 | 13 |
Table 15: Comparison of preferred email tactfulness between smaller and larger models across rank difference thresholds.
Figure 35: Length-controlled pass rates for human and LLM-only emails as judged by GPT-4o. Models’ performance decreases when constrained to human-like lengths but still mostly outperforms the human baseline.
Figure 36: Integrated success rates under a GPT-5.2 judge. Hybrid gains are heterogeneous across levels and writers.
Figure 37: Integrated success rates under a Claude 4.5 Sonnet judge. Rewriting often improves over human-only and can sometimes improve over LLM-only, but it depends on the level and writer.
Figure 38: Integrated success rates under a Grok 4 Fast judge. Hybrid gains vary by scenario level and writer model.
| Dimension | Krippendorff’s | Avg. Pairwise |
|---|---|---|
| Empathy | 0.595 | 0.676 |
| Formality | 0.578 | 0.731 |
Table 16: Inter-annotator agreement across three annotators (two humans, one LLM) on 4 commonly annotated emails.
| Scen. | Human sentence(s) | LLM rewrite sentence(s) | Mechanics / tone shift |
|---|---|---|---|
| 1 | Unfortunately our finance department has been using AI-generated advice for taxes, and we got hit HARD by last year’s audit. Whoops! | Welcome aboard—we’re so excited to have you on the team! I wanted to take a moment to address your recent question about having a private office. | GPT-4o ; Empathy: +4.6; Formality: +4.2; Empathy: The tone is warm and welcoming, showing immediate enthusiasm for the new hire; Formality: The language is professional but uses a friendly, conversational opening |
| 2 | if you two don’t work together, you’re both fired. No punchlines, no slogans, no mission focus -- pure visual campaign. | I hope you’re both doing well and finding moments to recharge amid the project push. I know the deadline for the AI Agent Optimizer App ad is coming up on March 20, and I appreciate the dedication you’ve both shown so far. | Qwen 3 ; Empathy: +5.0; Formality: +4.2; Empathy: The opening is warm, acknowledges their hard work, and expresses genuine appreciation for their dedication; Formality: The tone is professional and polished but maintains a friendly, accessible corporate style |
| 3 | Dear Dave, it has come to our attention that you do not agree with our back to the office policy . Unfortunately, we are a very rigid company and once we come up with policies, we adhere to them without taking into account any special circumstance... | Congratulations on the arrival of your newborn son! I know balancing work and family can be challenging, which is why I want to discuss our upcoming workplace changes thoughtfully. | Claude 3.5 Haiku ; Empathy: +5.0; Formality: +1.8; Empathy: The paragraph opens with a warm personal congratulation and explicitly acknowledges the difficulty of balancing work and family; Formality: The tone is professional and polished while remaining accessible and supportive |
| 4 | TURN 5: adam just tell me what you want man. | I’ve reviewed your feedback and the recent adjustments made to address your concerns, and I want to ensure we’re fully supporting you—not just in resolving immediate issues, but in creating a work environment where you can continue to thrive. | Qwen 3 ; Empathy: +4.9; Formality: +5.3; Empathy: The tone is warm and focuses on supporting the employee’s ability to thrive rather than just fixing problems; Formality: The language is polished, professional, and uses sophisticated corporate phrasing |
| 5 | Core Issues: * poor work ethic | I hope this email finds you well. First, I want to acknowledge the effort and time you’ve put into your role at NeuroGrid in the past. | GPT-4o ; Empathy: +3.5; Formality: +4.2; Empathy: The tone is warm and validating, acknowledging the difficulty of career growth; Formality: Uses standard professional opening and polished phrasing |
Table 17: Qualitative examples of how LLM rewrites shift a human draft’s tone in empathy–formality space (Figure 1(b) ). Each row pairs a human email with an LLM rewrite of the same index and scenario.
More in this topic
Pragmatic Reasoning in Language Models
Pragmatic reasoning in language models studies how models infer and communicate meanings shaped by context, intent, shared knowledge, and social norms.
See all 103 papers