VISTA: Steering LLMs with Latent User Values Inferred from a Few Survey Answers

Yinuo Xu; Jadie Adams; Brian Hu; Shivani Kumar; David Jurgens; Arslan Basharat

cs.CLOct 16, 2026

VISTA: Steering LLMs with Latent User Values Inferred from a Few Survey Answers

Authors: Yinuo Xu, Jadie Adams, Brian Hu, Shivani Kumar, David Jurgens, Arslan Basharat

Organizations: University of Michigan · Kitware Inc.

Abstract

Personalizing a language model requires knowing what a person values, which stays largely consistent across tasks. Yet most methods either learn task-specific preferences or place the user's profile in the prompt, where models struggle to act on it. We introduce VISTA (Value-Informed Steering and Targeted Acquisition), which infers a latent profile from users' (full or partial) survey answers, and maps the profile through a hypernetwork to a user-specific steering intervention in a frozen LLM to select answer choices or generate free-text responses. Given the same evidence, VISTA matches or beats in-context profiles on synthetic preferences (Food2D), the World Values Survey (WVS), and PrimeX, without adding prompt tokens. To reduce the cognitive burden of completing full surveys, VISTA also chooses which questions to ask and when to stop: on WVS it stops after 6% of the questionnaire while retaining 94% of the full-history gain, where standard adaptive testing retains at most 42%. The dominant steering direction is an interpretable value axis on which VISTA places unseen users accurately. Our results point toward pluralistic alignment that adapts to individuals at deployable cost.

Figures & tables

Figure 1: Overview of VISTA. User responses $D_{u}$ come from full or adaptive surveys. ① Value inference: a shared encoder (grey) maps $D_{u}$ to a posterior $q_{\phi}(z\mid D_{u})$ over a 32-d latent with mean $\hat{z}_{u}$ . ② Value steering: hypernetworks $h_{\psi}$ map $\hat{z}_{u}$ to a layer-10 steering vector $\Delta_{u}$ for forced choice or 16 soft-prefix tokens $P_{u}$ for free text in a frozen LLM. ③ Adaptive elicitation (forced choice only): select the unasked question with highest expected predictive information gain (EPIG) until its score falls below $\varepsilon$ times the initial maximum.

Figure 2: The elicited value latent is a continuous, interpretable value space, and the steered model places each held-out respondent on it. (WVS; $\Delta=\mathrm{hypernet}(z)$ is the steering vector from a respondent’s latent.) (a) Respondents in the top 2 answer principal axes, colored by $\Delta$ -PC1: a smooth gradient with no clusters. (b) Spearman correlation of $\Delta$ -PC1 with each topic block’s mean answer (six largest of fourteen), where the axis spans religiosity, traditional morality, and immigration restriction, and tracks the unseen moral-foundations binding factor ( $r=0.49$ ). (c) The same binding projection applied to a respondent’s held-out answers ( $x$ ) and to the steered model’s predictions on them ( $y$ ), relative to the unsteered model ( $y=0$ ).

Figure 3: Only VISTA significantly personalizes free-text generation on PrimeX. Results cover 686 unseen users, evaluated by an LLM judge. Gain is relative accuracy to each system’s user-agnostic control. (a) Accuracy versus personalization gain with 95% respondent-level CIs; only VISTA’s gain is significant (+1.2 points). (b) Gains on minority-view (39%) and majority-view (61%) rows. Unlike persona prompting, VISTA achieves a positive net gain (+1.2).

Figure 4: Adaptive elicitation over the value latent, and when to stop. (a,b) Accuracy on the held-out answer vs questions asked, for nine selection methods, across WVS (848 respondents, 6784 scored answers) and 858 respondents, 12141 answers). 95% person-clustered intervals are ±1.0 points on WVS and ±0.95 on PRIMEX for every arm. (c) The same two datasets as % of the full questionnaire’s personalization gain recovered, paired over respondents against a run in which every item is observed. Bands are 95 % respondent-clustered bootstrap intervals. Stars mark where each method’s own stopping rule stops.

$K$	Trials	Converged (NLL $<0.1$ )	Best	Median
16	6	2/6	0.0296	0.5061
32	3	3/3	0.0398	0.0455
64	6	4/6	0.0460	0.0507

Table 1: Latent size, aggregated over the trials of Table 2 .

K=32

is the only latent size at which every trial converged. The single best trial is

K=16

, but its advantage depends on two of six trials while the other four collapse; so we choose

K=32

as the robust choice. Trial counts are small and were not balanced by design (TPE allocates adaptively), so

K

=32 is a safe choice.

$K$	$\beta_{\text{target}}$	$\beta_{\sigma}\,(\times 10^{-3})$	Free bits	$\eta\,(\times 10^{-4})$	Val. NLL
16	0.1066	2.6	0.362	9.7	0.0296
16	0.0317	1.3	0.333	7.8	0.0360
32	0.2143	0.97	0.862	3.6	0.0398
32	0.0929	4.4	0.190	8.0	0.0455
64	0.1881	7.6	0.158	12	0.0460
64	0.0819	0.11	0.595	8.2	0.0491

Table 2: Value-model search. Optuna (TPE), 15 trials, objective: minimize validation NLL. The study ran as distributed workers that number trials independently, so rows are ordered by the objective rather than by worker-local trial id. Runs that fail to converge land near

0.5

(a collapsed posterior), not at a marginally worse fit.

\beta_{\sigma}

and

\eta

are shown in units of

10^{-3}

and

10^{-4}

, respectively.

Layer	Variant	Blocks	$\\|\Delta\\|$	$\eta$	Val. user-specific [95% CI]
10	no_context	8	1.235	$\mathbf{2.4\times 10^{-4}}$	$\mathbf{+0.0513}$ $[+0.0416,+0.0607]$
10	no_context	4	1.476	$6.2\times 10^{-5}$	$+0.0479$ $[+0.0386,+0.0575]$
10	no_context	4	1.282	$1.9\times 10^{-4}$	$+0.0478$ $[+0.0382,+0.0575]$
10	no_context	2	0.831	$7.2\times 10^{-5}$	$+0.0441$ $[+0.0354,+0.0531]$
6	no_context	8	0.692	$4.1\times 10^{-5}$	$+0.0441$ $[+0.0343,+0.0534]$
10	no_context	2	0.940	$1.8\times 10^{-4}$	$+0.0438$ $[+0.0349,+0.0522]$

Table 3: Steering-architecture search. Optuna (TPE), 13 trials, scored on the validation split; the test split was never used for selection. Search space: layer

\in\{6,10,14,20,26\}

, variant

\in

{no_context, cross_attn}, blocks

\in\{2,4,8\}

\|\Delta\|\sim\mathcal{U}(0.3,3.0)

\eta\sim\log\mathcal{U}(3\!\times\!10^{-5},3\!\times\!10^{-4})

. The objective is the validation user-specific effect (steered

-

shuffled). Trials ran at reduced epochs; the selected configuration was retrained at full length and scored once on test.

		Person-cond. ( $z$ )		Population		Shuffled		Gain vs pop.		Gain vs shuf.
Data	Method	All	Cont.	All	Cont.	All	Cont.	All	Cont.	All	Cont.
Food2D	VISTA-VI	0.625	0.624	0.497	0.494	0.511	0.513	+12.8*	+13.0*	+11.4*	+11.1*
								[+11.0, +14.5]	[+11.1, +14.8]	[+9.1, +13.8]	[+8.8, +13.5]
	PAL	0.600	0.599	0.497	0.494	0.508	0.507	+10.4*	+10.5*	+9.3*	+9.2*
								[+8.4, +12.2]	[+8.5, +12.5]	[+7.2, +11.4]	[+7.0, +11.4]
	LoRe	0.591	0.591	0.497	0.494	0.507	0.506	+9.4*	+9.7*	+8.4*	+8.5*

Table 4: Value inference only: how much of each model’s accuracy is about the person. Held-out answer accuracy for the three value models, two controls that strip person information, and the paired gains over each; bold marks the best person-conditioned accuracy and gain per dataset. Population knows no person (

\bar{z}

over training users on Food2D, the per-item training-fold average on PRIMEX,

z=0

on WVS). Shuffled deranges the test users’ latents so nobody gets their own, which also removes the marginal fit of the latent distribution, making gain vs shuffled the user-specific lift and usually the stricter quantity. Gains are accuracy points, paired per person on identical rows, with 95% person-clustered bootstrap intervals (4,000 resamples); ∗ marks an interval clear of zero. No steering arms appear here. Contested uses each dataset’s own threshold — train-label entropy

>0.60

nats on Food2D (

4{,}380/4{,}800

) and WVS (

33{,}932/73{,}288

), leave-fold-out answer entropy

>0.65

on PRIMEX (

8{,}615/12{,}141

).So it is exact within a dataset but not comparable across them. Two readings: (i) gains are larger on contested items, as expected, since where the population already agrees there is nothing for a person model to add; (ii) VISTA-VI has the largest user-specific lift on Food2D (

+11.4

vs PAL’s

+9.3

) and WVS (

+8.7

+6.6

), PAL on PRIMEX AB (

+3.5

+2.9

). Every cell is paired: WVS’s PAL and LoRe population arms were added by re-scoring the published checkpoints, which reproduce their published accuracies to

2\times 10^{-4}

. One exception to the usual ordering: LoRe’s WVS population arm (

0.549

contested) falls below its shuffled arm (

0.570

), so there population is the weaker null. WVS’s contested mask was built separately for VISTA-VI (

33{,}932

rows) and PAL/LoRe (

33{,}632

); the All column is the same

73{,}288

rows for every arm.

Figure 5: Personalized text generation on PrimeX.

		All pairs					Contested items
		Accuracy (%)			Gain (pts)		Accuracy (%)			Gain (pts)
Data	System	Steer.	Pop.	Shuf.	vs pop.	vs shuf.	Steer.	Pop.	Shuf.	vs pop.	vs shuf.
WVS7 UK+US 848 resp. $n=73{,}288$ cont. $33{,}932$	VISTA	64.87	63.31	62.58	$+1.56$ [+1.21, +1.91]	+2.28 [+1.84, +2.72]	55.48	51.93	51.89	$+3.55$ [+3.09, +4.03]	$+3.60$ [+3.03, +4.16]
WVS7 UK+US 848 resp. $n=73{,}288$ cont. $33{,}932$	In-context profile	62.25	–	60.44	–	$+1.81$ [+1.45, +2.20]	55.16	–	50.56	–	+4.60 [+4.08, +5.11]
PrimeX ctx $B$ 858 people 5 folds $n=12{,}141$ cont. $8{,}615$	VISTA	63.26	60.85	58.00	$+2.41$ [+1.70, +3.10]	+5.26 [+4.15, +6.35]	59.99	56.61	53.80	$+3.38$ [+2.36, +4.34]	+6.19 [+4.79, +7.56]
	In-context profile	62.85	–	60.21	–	$+2.64$ [+1.88, +3.39]	59.36	–	55.68	–	$+3.68$ [+2.63, +4.75]

Table 5: Steering accuracy and personalization gain, three datasets. All arms within a dataset are scored on identical rows; 95% CIs from a person-clustered bootstrap. Bold marks the better of the two arms wherever both are defined. Population conditions on the mean training representation, shuffled on another respondent’s. VISTA is the hypernetwork conditioned on VPL’s

z

. The in-context baseline puts the person’s own answers in the prompt instead; it has no population condition—its arms are the user’s own profile, another user’s, and none—and it receives the same evidence the encoder does, every observed answer, so the comparison is about how the person is represented and not about how much is known. Contested items have train-label entropy

>0.6

nats (WVS, Food2D); PrimeX contested items exceed

0.65

nats on the other folds, so the split never sees the scored fold. Food2D is the simulated UserAlign domain: 300 held-out users

\times

16 held-out pairs, and accuracy is the steered model’s forced-choice letter against the simulated user’s own answer, so the metric is the same quantity as on the two real datasets. Its VISTA arm is the

z

-prefix hypernetwork (

k=16

soft tokens); its in-context arm reads the same

z

export, so the two rows are scored on byte-identical pairs, and paired on them VISTA leads by

+9.57

points

[+7.83,+11.33]

. Food2D items are near-balanced by construction, so

91\%

of rows (

4{,}380

4{,}800

) pass the

0.6

-nat rule and the contested subset is nearly the whole sample—which is why its two halves agree, unlike WVS, where the contested half is the part of the bank that separates methods at all.

System	Accuracy	Gain	Change	Net gain
	Minority-view rows (38%)		Majority-view rows (62%)
In-context LLM	18.1	+15.4	$-$ 7.8	+1.1
Text-pooled + HS	15.6	+10.6	$-$ 5.0	+1.0
PAL + HS	19.1	+13.0	$-$ 6.6	+1.0
VPL + HS (ours)	24.4	+11.2	$-$ 3.7	+2.0

Table 6: Where the personalization gain comes from (PrimeX, setting B: 18 Primals answers as context; 858 unseen users, 12,141 rows). A row is minority-view when the user’s answer differs from the most common answer among users in the other folds. Gain and change are in accuracy points over the same system’s user-agnostic version (population representation; no profile for the in-context LLM). Net gain is the share-weighted sum of the two columns. All gains and changes have 95% person-bootstrap CIs excluding 0.

System	Accuracy	Gain	Change	Net gain
	Minority-view rows (27%)		Majority-view rows (73%)
Text-pooled + HS	32.1	$+$ 2.8	$-$ 0.2 †	$+$ 0.6
PAL + HS	43.0	$+$ 9.4	$-$ 1.5	$+$ 1.5
VPL + HS (ours)	40.7	$+$ 6.4	$-$ 0.3 †	$+$ 1.6
In-context LLM ‡	16.3	$+$ 2.0	$-$ 5.6	$-$ 3.5

Table 7: Where the personalization gain comes from (WVS7 UK+US; 848 unseen respondents, 73,288 rows). A row is minority-view when the respondent’s answer differs from the most common answer to that item among training respondents. Gain and change are accuracy points over the same system’s user-agnostic version (the population representation). Net gain is the share-weighted sum. 95% person-bootstrap CIs; † marks the two cells whose CI includes zero, all others exclude it. ‡ The in-context LLM has no population condition, so it is referenced against another respondent’s profile rather than a user-agnostic one and is not directly comparable; its no-profile reference gains on both subsets (

+

8.9 /

+

8.0) because a bare prompt lacks the generic shift the steered population condition already has.

Q: What’s your view on the federal government’s role in providing retirement income?
User’s answer: Adequate retirement income is not the federal government’s responsibility to provide (minority view)
Real explanation	It is something that is more the individual responsibility with a little help from government but people need to be responsible for themselves
Unsteered (no side)	A timely and thought-provoking question! The role of the federal government in providing retirement income is a complex and debated topic. Here’s my neutral AI perspective: Pros of government involvement: …
Persona prompting (other side)	I think the federal government should take a more active role in providing retirement income. It seems to me that what happens in life is not always meant to happen, and it’s only natural …
Ours (user’s side)	They have no business being involved in retirement income.
Q: What’s your view on modern medical treatments and whether they are worth their costs?

Table 8: Illustrative VISTA generated texts on PrimeX for two unseen users holding the minority view; parentheses give the side a judge reads in each text. Drawn at random from rows where our text takes the user’s side only under their own profile (8.1% of minority-view rows, vs. 5.7% flipping away).

Figure 6: Personalized text generation on Food2d

Figure 7: Interpretability results on PrimeX

Figure 8: Elicitation results on food2d

Personalized Language Model Alignment

Personalized language model alignment adapts model behavior to individual preferences while preserving safety, accuracy, and other shared constraints.

See all 61 papers

Model vs. User: Whose Values Prevail? Inference-Time Value Alignment in Multi-Turn Conversations
Oct 14, 2026Enrico Liscio, Anne Arzberger, Aditya Shankar +1LLM Alignment Inference-Time Language Model Alignment
Delft University of Technology, the Netherlands
Persona Following Is Not Selective Control: The Neutrality Gap in LLM User Simulation
Sep 28, 2026Jiashen Ren, Wenlin Zhang, Bohan Zhang +5Persona-Conditioned Generation User Simulation
City University of Hong Kong · Stanford University
FleetPilot: Fine-Grained Steering Empowers Real-World LLM Personalization
Oct 10, 2026Kounianhua Du, Jianxing Liu, Kangning Zhang +7Personalized Language Models Language Model Steering
Shanghai Jiao Tong University · Xiaohongshu Inc., Shanghai, China

Oct 14, 2026J/K move · Enter open · S save

Model vs. User: Whose Values Prevail? Inference-Time Value Alignment in Multi-Turn Conversations

Enrico Liscio, Anne Arzberger, Aditya Shankar, Pradeep K. Murukannaiah

Delft University of Technology, the Netherlands

LLM Alignment Inference-Time Language Model Alignment Language Model Steering Multi-Objective Language Model Alignment Multi-Turn Dialogue Evaluation

Open paper AskPDF arXiv

Human values shape how we expect language models to behave. Dominant alignment approaches treat values as global objectives, although the appropriate value orientation is personal and contextual. Training-free alignment can steer models toward target values at inference time, yet has been largely evaluated in single-turn settings and with a single steering direction (e.g., toward safety). We evaluate four prompt- and activation-based steering methods in multi-turn conversations, where user and assistant models push toward the same or opposing, equally desirable values. We track alignment with a linear probe across three models and two value trade-offs. When assistant and user pull in opposite directions, the side matching the assistant's baseline bias often prevails, and effects that look successful early often fade or reverse after a few turns. Steering should therefore be evaluated over full conversations and in both directions of a value trade-off.

Abstract

Personalized Language Model Alignment

Explore similar work

Model vs. User: Whose Values Prevail? Inference-Time Value Alignment in Multi-Turn Conversations

Persona Following Is Not Selective Control: The Neutrality Gap in LLM User Simulation

FleetPilot: Fine-Grained Steering Empowers Real-World LLM Personalization