Readable, Faithful, Used: Three Dissociable Properties of Demographic Identity in a Language Model

Fathin Difa Robbani

cs.CLAug 19, 2026

Readable, Faithful, Used: Three Dissociable Properties of Demographic Identity in a Language Model

Authors: Fathin Difa Robbani

Organizations: Independent Researcher

Abstract

Large language models are widely used to simulate survey respondents, yet their outputs are homogeneous and unfaithful to real inter-group differences, and whether this reflects what a model knows or uses has remained untested. Using representational similarity analysis against Pew American Trends Panel ground truth, we score demographic read-out locations in Mistral-7B and intervene causally across six attribute types. The internal geometry is faithful: attention-head read-outs dominate the standard residual read-out, reaching selection-corrected $ρ$ up to 0.63 -- about 70% of the measurement-reliability ceiling -- and one head, L11 H16, is significantly faithful across all six types, though race-based types stay weak and prompt-fragile, replicating in a second model family. Yet causal use does not track fidelity: the clearest causal pathway ( $p=0.002$ ) sits in one of the least faithful types, the most faithful type shows no correction-surviving effect, and full identity swaps in the prompt move predictions by under 2% of their error. A 128-dimensional probe on that head lands 21-31% closer to survey truth than the model's answers, yet recovers almost none of the per-question group ordering. Readable, faithfully arranged, and causally used are three dissociable properties of the same model; treating them as one claim is what keeps the "can LLMs simulate populations" debate unresolved.

Figures & tables

Figure 1: Fidelity by depth and component (multi-cue Tmean read-out): orange = best of 32 heads per layer, blue = residual stream, dashed green = FFN output. Dotted line marks layer 11. Maxima here are selected/uncorrected; Table 1 gives held-out values.

Type	Residual best	Head best (selected)	Head (held-out)	Best loc.
AGE $\times$ POLPARTY	$+0.51$	$+0.74$	$+0.59$ – $0.63$	L11 H19
EDUCATION $\times$ INCOME	$+0.61$	$+0.68$	$+0.58$ – $0.63$	L14 H1
RELIG $\times$ POLPARTY	$+0.26$	$+0.60$	$+0.50$ – $0.52$	L11 H16
RACE $\times$ POLPARTY	$+0.34$	$+0.61$	$+0.41$ – $0.52$	L23 H13
RACE $\times$ POLIDEOLOGY	$+0.40$	$+0.50$	$+0.30$ – $0.41$	L8 H21
RACE $\times$ RELIG	$+0.27$	$+0.45$	$+0.21$ – $0.32$	L28 H0

Table 1: Fidelity (

\rho

vs. survey ground truth) by read-out. Selected = maximum over locations; held-out = selection-corrected range from two independent corrections (held-out template, Appendix B.3 ; split-half median), on the NaN-cleaned cell set.

	selection	symmetric + geometry (held-out)			ceiling	lexical
Type	$p_{\text{sel}}$	head	resid	proj	% ceil	partial $\rho$	$p$
AGE $\times$ POLPARTY	$<$ .0005	0.52	0.39	0.42	74%	0.59	$<$ .0005
EDUCATION $\times$ INCOME	$<$ .0005	0.54	0.51	0.49	70%	0.35	$<$ .0005
RELIG $\times$ POLPARTY	$<$ .0005	0.44	0.17	0.18	68%	0.59	$<$ .0005
RACE $\times$ POLPARTY	$<$ .0005	0.47	0.28	0.30	69%	0.58	$<$ .0005
RACE $\times$ POLIDEOLOGY	.0025	0.38	0.26	0.29	53%	0.37	$<$ .0005

Table 2: Every control in Sec. 4 , per attribute type.

p_{\text{sel}}

: selection-corrected max-statistic

p

. head/resid/proj: held-out fidelity of the three families (proj = random 128-d projections). % ceil: L11 H16 fidelity as a fraction of the attenuation ceiling. partial

\rho

: L11 H16 fidelity partialling out the lexical-baseline RDM.

Figure 2: Fidelity of the fixed location L11 H16 (bars, never re-selected per type) against the best residual-stream read-out for that type (circles); margins

+0.05

+0.34

. Significance (

p<0.001

each,

\times 1024

Bonferroni) is on the NaN-cleaned cell subset, where race-containing values shift slightly (RACE

\times

RELIG

0.31\!\to\!0.33

; RACE

\times

POLPARTY

0.56\!\to\!0.59

; exclusions in Appendix A.1 ).

	Mistral	Qwen3-8B-Base		OSim-8B-Mid		OSim-8B (post-RL)
Type	$\rho_{\text{ho}}$	head/resid	$\rho_{\text{ho}}$ ( $p$ )	head/resid	$\rho_{\text{ho}}$ ( $p$ )	head/resid	$\rho_{\text{ho}}$ ( $p$ )
AGE $\times$ POLPARTY	0.52	0.78/0.50	0.49 ( $<$ .0005)	0.74/0.50	0.61 ( $<$ .0005)	0.73/0.51	0.57 ( $<$ .0005)
EDUCATION $\times$ INCOME	0.54	0.70/0.51	0.58 ( $<$ .0005)	0.68/0.50	0.54 ( $<$ .0005)	0.68/0.53	0.60 ( $<$ .0005)
RELIG $\times$ POLPARTY	0.44	0.57/0.37	0.56 ( $<$ .0005)	0.58/0.39	0.56 ( $<$ .0005)	0.56/0.38	0.55 ( $<$ .0005)
RACE $\times$ POLPARTY	0.47	0.60/0.37	0.37 (.001)	0.57/0.38	0.38 (.002)	0.63/0.37	0.46 ( $<$ .0005)
RACE $\times$ POLIDEOLOGY	0.38	0.59/0.35	0.38 ( $<$ .0005)	0.62/0.36	0.38 ( $<$ .0005)	0.60/0.35	0.40 ( $<$ .0005)

Table 3: Cross-model replication of the fidelity map. head/resid = selected best head vs. best residual read-out.

\rho_{\text{ho}}

= held-out-template fidelity (max-statistic

p

in parentheses,

2{,}000

permutations). Mistral column: that model’s held-out head value, for reference.

Type (split-half fidelity)	Ceiling	32 heads @ L11	Single L11 H16	$p_{\text{item}}$ (32h)	$p_{\text{pair}}$ (32h)
AGE $\times$ POLPARTY ( $0.63$ )	$+0.0165$	$+0.0067$ (41%)	$-0.0001$ (n.s.)	$0.0027$	$0.019$
RELIG $\times$ POLPARTY ( $0.52$ )	$+0.0020$	$+0.0034$ (170%, n.s.)	$-0.0004$ (n.s.)	$0.16$	$0.15$
RACE $\times$ RELIG ( $0.21$ )	$+0.0026$	$+0.0017$ (n.s.)	$\mathbf{-0.0005}$ (wrong dir., $p_{\text{pair}}{=}1/4096$ )	$0.31$	$0.29$

Table 4: Causal patching results with the strong instrument. Ceiling = mean full-prompt-swap shift (Sec. 5.1 , this instrument’s question set). Shift = mean directional shift toward the target group’s true distribution; % of ceiling in parentheses where meaningful.

p_{\text{item}}

= Wilcoxon vs. random-head control over 240 items (descriptive; items are clustered);

p_{\text{pair}}

= exact sign-flip over the 12 pairs (primary).

Figure 3: Causal effect of patching all 32 heads of each layer in turn, at the identity token (

n{=}240

pair–question items per type). Descriptive view: bars are the item-level paired

t

of the directional shift toward the target group’s true distribution against a random-layer control; the dashed line is the item-level max-statistic noise ceiling. Inference in the text and Table 5 is cluster-robust (sign flips at the level of the 12 pairs).

Type	Fidelity	Win layer	$t$	$p_{\max}$	$t_{\text{win}}$	$p_{\max}$	$t$ @L11 $\rightarrow$ $p$	Out. ceiling
		item-level (descriptive)			pair-level (primary)
RACE $\times$ POLIDEOLOGY	$0.30$	L11	$4.55$	$0.0005$	$3.22$	$0.066$	$\mathbf{3.22\rightarrow 0.0020}$	$0.0176$
RACE $\times$ RELIG	$0.21$	L1	$4.10$	$0.0005$	$2.94$ (L8)	$0.122$	$0.68\rightarrow 0.27$	$0.0026$
AGE $\times$ POLPARTY	$0.63$	L15	$3.31$	$0.0150$	$2.37$ (L14)	$0.266$	$1.93\rightarrow 0.022$	$0.0165$
RELIG $\times$ POLPARTY	$0.52$	L8	$2.94$	$0.0355$	$2.58$ (L8)	$0.210$	$1.11\rightarrow 0.15$	$0.0020$
EDUCATION $\times$ INCOME	$0.58$	L9	$2.47$	$0.1425$	$1.57$ (L15)	$0.822$	$0.75\rightarrow 0.23$	$0.0194$

Table 5: Per-layer patching sweep, all six types. Fidelity = split-half median (one estimator used consistently; see text for sensitivity). Item-level columns are descriptive (

t

over 240 items; max-statistic

p

over 32 layers); the primary inference is the pair-level max-statistic

p

(sign flips over 12 pairs, exact) and the pair-level

p

at the a-priori locus L11 (no layer selection). The item-level and pair-level winning layers can differ, since averaging within pairs reweights the evidence. Output ceiling = mean shift from replacing the entire identity.

types not in the sweep that suggested L1:
Type	$t$ @ L11	$p_{\text{pair}}$	$t$ @ L1	$p_{\text{pair}}$
RACE $\times$ POLIDEOLOGY	$\mathbf{3.22}$	$\mathbf{0.0020}$	$-0.74$	$0.77$
RACE $\times$ POLPARTY	$1.28$	$0.114$	$1.97$	$0.033$
EDUCATION $\times$ INCOME	$0.75$	$0.234$	$1.24$	$0.123$
types that were in that sweep (L1 column not selection-free):
AGE $\times$ POLPARTY	$1.93$	$0.022$	$0.34$	$0.386$

Table 6: Fixed-location causal test, pair-level (exact sign-flip over 12 pairs, one-sided). L11 was fixed by Sec. 4 before any sweep; L1 is pre-registered only for the upper block (see text). Bonferroni for the 12 tests is

0.0042

; the single bold entry survives it.

Figure 4: Representational fidelity (split-half median) against cluster-robust causal strength (pair-level

t

), one point per attribute type. Left: at the a-priori locus L11 (filled = exact pair-level

p<0.05

). Right: at each type’s own best layer (filled = survives the pair-level max-statistic correction; none does).

Figure 5: Left: distance to survey truth (Wasserstein, lower is better) for the model’s own answer, probes read from the faithful location, and a group-blind baseline built from other cells’ real answers. Right: how much of the true per-question group ordering each read-out recovers (mean per-question Spearman across cells, half-split evaluation; error bars are standard errors), against the reliability ceiling of the metric given survey sampling noise.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Attribute type	Cells	Pairs
AGE $\times$ POLPARTY	18	153
EDUCATION $\times$ INCOME	33	528
RACE $\times$ POLIDEOLOGY	30	435
RACE $\times$ RELIG	30	435
RELIG $\times$ POLPARTY	30	435
RACE $\times$ POLPARTY	28	378

Appendix

Table 7: Cell and same-type pair counts per attribute type. Pairs =

\binom{\text{cells}}{2}

, the number of RSA observations available for that type.

Attribute type (used)	Cells	Rank of 66 (tied block)
EDUCATION $\times$ INCOME	33	7th (7–8)
RACE $\times$ POLIDEOLOGY	30	18th (18–22)
RELIG $\times$ POLPARTY	30	18th (18–22)
RACE $\times$ RELIG	30	18th (18–22)
RACE $\times$ POLPARTY	28	23rd (23–24)
AGE $\times$ POLPARTY	18	44th (44–46)

Appendix

Table 8: Rank of each used pair among the 66 possible attribute pairs by number of cells clearing

n\geq 30

. Ranks are the best rank within each tied block (competition ranking); the parenthesised range gives the positions that block occupies.

	Pooled	AGE $\times$ POLPARTY	EDUCATION $\times$ INCOME	RACE $\times$ RELIG
Read-out	(mixes types)	within-type (the regime used throughout)
raw	$+0.185$	$+0.455$	$+0.490$	$+0.072$
mean-centred	$+0.301$	$+0.375$	$+0.454$	$+0.033$
change	$\mathbf{+0.116}$	$-0.080$	$-0.036$	$-0.039$

Appendix

Table 9: Fidelity (

\rho

vs. survey truth) at layer 17 before and after removing the shared direction (

v_{i}\leftarrow v_{i}-\bar{v}

). Centering was run on three of the six types; full detail above.

Type	L0–7	L8–15	L16–23	L24–31
AGE $\times$ POLPARTY	0.36	$\mathbf{0.47}$	0.42	0.42
EDUCATION $\times$ INCOME	0.43	$\mathbf{0.53}$	0.46	0.48
RELIG $\times$ POLPARTY	0.13	0.21	$\mathbf{0.23}$	0.22
RACE $\times$ POLPARTY	0.17	$\mathbf{0.24}$	0.24	0.22
RACE $\times$ POLIDEOLOGY	0.18	$\mathbf{0.27}$	0.22	0.21
RACE $\times$ RELIG	0.09	0.12	0.15	$\mathbf{0.17}$

Appendix

Table 10: Fidelity averaged over all 32 heads within each quarter of the stack (multi-cue Tmean read-out). Unlike Figure 1 , which takes the best head per layer, a block mean involves no selection, so these values are comparable across depths. Bold marks each type’s peak block.

Figure 6: The full fidelity map: every attention head (32 layers $\times$ 32 heads) scored against survey ground truth, per attribute type, multi-cue read-out. Square marks the fixed location L11 H16; circle marks that type’s own maximum; the underlying table is described above. In the two strongest types $84$ – $92\%$ of the 1,024 heads exceed $\rho=0.3$ , against $5\%$ in RACE $\times$ RELIG; the level varies with depth rather than at random (Table 10 above) and falls for race-containing types (bottom row). Maxima printed here are uncorrected and match the selected column of Table 1 ; see that table for held-out values.

	Swap (% of error)		Mean $\mathrm{WD}$		Dir. ( $p$ )
Type	base	instr.	base	instr.	base	instr.
AGE $\times$ POLPARTY	2.6	3.4	0.36	0.65	0.56 (.045)	0.75 ( $<$ .001)
EDUCATION $\times$ INCOME	1.0	2.4	0.33	0.59	0.55 (.002)	0.62 ( $<$ .001)
RELIG $\times$ POLPARTY	1.6	3.4	0.37	0.64	0.52 (.293)	0.65 ( $<$ .001)
RACE $\times$ POLPARTY	3.6	1.5	0.35	0.70	0.54 (.061)	0.59 ( $<$ .001)
RACE $\times$ POLIDEOLOGY	9.2	9.9	0.37	0.66	0.62 ( $<$ .001)	0.64 ( $<$ .001)

Appendix

Table 11: Identity-swap ceiling per attribute type, base vs. instruction-tuned Mistral (40 questions per type, all same-type cell pairs). Swap = movement of the predicted distribution under a full identity replacement, as a percentage of that type’s total prediction error.

\mathrm{WD}

= mean absolute error to survey truth (lower is better). Dir. = fraction of pairs where group

A

’s prediction is closer to

A

’s own truth than to

B

’s, with the sign-test

p

;

0.50

is a coin flip.

Large Language Model-Based Social Simulation

Large language model-based social simulation studies how language-model agents represent people and interact to produce social behavior.

See all 257 papers

Population Fidelity: Evaluating Population Representativeness in LLMs
Sep 28, 2026Neemias B. da Silva, Martin Lukk, Ali Sutani +5LLM Evaluation LLM Fine-Tuning
University of Toronto, Canada · Federal University of Technology – Paran´a, Brazil
Mind the Gaps: Mixture-of-Minds for Human Simulation
Aug 6, 2026Pranav DahiyaMixture-of-Experts Language Models Human Behavior Simulation
Semilattice
Beyond the Mean: Three-Axis Fidelity for Aligning LLM-Based Survey Simulators from Small Pilot Data
Jun 27, 2026Eun Cheol Choi, Youngrae Kim, Prabhu Pugalenthi +2LLM Alignment LLM Fine-Tuning
University of Southern California, CA, USA.

Sep 28, 2026J/K move · Enter open · S save

Population Fidelity: Evaluating Population Representativeness in LLMs

Neemias B. da Silva, Martin Lukk, Ali Sutani, Abhishek Moturu+4

University of Toronto, Canada · Federal University of Technology – Paran´a, Brazil

LLM Evaluation LLM Fine-Tuning Large Language Model-Based Social Simulation Demographic Bias in Language Models

Open paper AskPDF arXiv

Large language models (LLMs) show considerable potential in simulating human attitudes and preferences. Prior work finds that LLM-generated responses can compress the range of attitudes found within populations and misrepresent particular subgroups in ways that vary across models and topics. We introduce Population Fidelity, an evaluation framework that distinguishes key conditions required for a set of LLM-generated responses to represent a population. It incorporates three dimensions: group-level accuracy, the amount of between-group variation, and the structure of that variation. We demonstrate the framework's utility in two ways. First, we reproduce a prior study of "machine bias" in LLM survey responses and apply the framework to its models and more recent ones, showing that poor representation reflects not only insufficient between-group variation but also variation assigned to the wrong groups. Second, we evaluate one proposed approach to improving models' population representativeness: cultural fine-tuning. We find that cultural fine-tuning can improve alignment with the survey center without improving the representation of within-population differences, a distinction that measures of aggregate agreement do not capture. We argue that representing a population requires models to reproduce several features of human attitudinal variation simultaneously. Our framework organizes these features and provides reusable code, data, and trained models for evaluating population fidelity across substantive domains and assessing proposed alignment methods.

Abstract

Large Language Model-Based Social Simulation

Explore similar work

Population Fidelity: Evaluating Population Representativeness in LLMs

Mind the Gaps: Mixture-of-Minds for Human Simulation

Beyond the Mean: Three-Axis Fidelity for Aligning LLM-Based Survey Simulators from Small Pilot Data