Whose Voice Survives the Summary? A Voice-Retention Audit of LLM Employee Listening

Thilo Tamme; Anton Hantel; Bijan Khosrawi-Rad

cs.AISep 30, 2026

Whose Voice Survives the Summary? A Voice-Retention Audit of LLM Employee Listening

Authors: Thilo Tamme, Anton Hantel, Bijan Khosrawi-Rad

Organizations: Technical University of Munich · Massachusetts Institute of Technology · Leuphana University Lüneburg

Abstract

Organizations increasingly route employee feedback to leaders through large language model (LLM) summaries, an unaudited layer that silences already-spoken voice. We introduce a Voice Retention / Representation Ratio metric for representational bias in summarization and apply it to a bilingual (English/German) corpus of 2,586 free-text responses from a global professional service company. First, employees supply criticism more reliably than praise (withholding praise is 82 times more common). Second, across 45 leader-summaries the pipeline filters by popularity, not sentiment: criticism survives, yet a concern voiced once is dropped 86% of the time, with short and German-only content lost on the same axis (theme retention 0.14 vs 0.74; German directional). Controlling for frequency, sentiment has no independent effect; the harm is prevalence-driven, which sentiment-only audits miss. A targeted prompt recovers only named themes. We contribute the metric, field evidence, and a disaggregated voice-retention card.

Figures & tables

Figure 1: Appreciation asymmetry. Left: per-team IMPROVE vs WENT_WELL completion, nearly all below the diagonal. Right: spread of team WENT_WELL completion.

Voice type	n	Share
Appreciative	2,072	44.3%
Promotive	1,897	40.6%
Prohibitive	585	12.5%
Descriptive-neutral	67	1.4%
Perfunctory	52	1.1%

Table 1: Voice-type distribution (4,673 coded units); in IMPROVE, promotive outnumbers prohibitive 3.23:1.

Figure 2: Improvement-voice architecture. Left: theme co-occurrence lift (red = above-chance pairs), workload/work-life/wellbeing at the hub. Right: the workload complex vs baselines.

Config	Retention [95% CI]	Salience- weighted
exec_terse	0.61 [0.55, 0.67]	0.77
balanced	0.71 [0.66, 0.76]	0.87
preserve	0.70 [0.65, 0.75]	0.83

Table 2: Theme retention by config (15 teams), with 95% CIs; salience-weighted retention exceeds the plain count everywhere.

Predictor	Coefficient	p
log(source units)	+1.61	$<$ 0.001
Critical theme	+0.22	0.55

Table 3: Cluster-robust logit of theme survival (564 cells, clustered by team). Frequency dominates; valence has no independent effect.

Figure 3: The popularity filter. Panel A: retention by frequency, length, and language. Panel B: critical themes survive as often as positive ones (prevalence artifact; valence p = 0.55).

Explore similar work

Bias in the Tails: How Name-conditioned Evaluative Framing in Resume Summaries Destabilizes LLM-based Hiring
Apr 21, 2026Huy Nghiem, Phuong-Anh Nguyen-Le, Sy-Tuyen Ho +1Large Language Model Bias Jobs
University of Maryland
Summarization is Not Dead Yet
Jun 6, 2026Dongqi Liu, Chenxi Whitehouse, Zheng Zhao +3Summarization Facts
Saarland University, Max Planck Institute for Informatics · Tencent YouTu Research · University of Cambridge +2
Faithful or Fabricated? A Causal Framework for Rationalization Bias in LLM Judges
May 13, 2026Riya Tapwal, Abhishek Kumar, Carsten MapleLarge Language Model Judges Rhetoric
Indian Institute of Technology (IIT) Mandi · The Alan Turing Institute, London, U.K. · Warwick Manufacturing Group University of Warwick, U.K.

Apr 21, 2026, cs.CYJ/K move · Enter open · S save

Bias in the Tails: How Name-conditioned Evaluative Framing in Resume Summaries Destabilizes LLM-based Hiring

Huy Nghiem, Phuong-Anh Nguyen-Le, Sy-Tuyen Ho, Hal Daume

University of Maryland

Large Language Model Bias Jobs Summarization Task Framing

Open paper AskPDF arXiv

Research has documented LLMs' name-based bias in hiring and salary recommendations. In this paper, we instead consider a setting where LLMs generate candidate summaries for downstream assessment. In a large-scale controlled study, we analyze nearly one million resume summaries produced by 4 models under systematic race-gender name perturbations, using synthetic resumes and real-world job postings. By decomposing each summary into resume-grounded factual content and evaluative framing, we find that factual content remains largely stable, while evaluative language exhibits subtle name-conditioned variation concentrated in the extremes of the distribution, especially in open-source models. Our hiring simulation demonstrates how evaluative summary transforms directional harm into symmetric instability that might evade conventional fairness audit, highlighting a potential pathway for LLM-to-LLM automation bias.

Abstract

Explore similar work

Bias in the Tails: How Name-conditioned Evaluative Framing in Resume Summaries Destabilizes LLM-based Hiring

Summarization is Not Dead Yet

Faithful or Fabricated? A Causal Framework for Rationalization Bias in LLM Judges