SpecSteer: Synergizing Local Context and Global Reasoning for Efficient Personalized Generation

Hang Lv; Sheng Liang; Hao Wang; Yongyue Zhang; Hongchao Gu; Wei Guo; Defu Lian; Yong Liu

cs.CLMar 17, 2026

SpecSteer: Synergizing Local Context and Global Reasoning for Efficient Personalized Generation

Authors: Hang Lv, Sheng Liang, Hao Wang, Yongyue Zhang, Hongchao Gu, Wei Guo, Defu Lian, Yong Liu,

Organizations: University of Science and Technology of China · Huawei Technologies Co., Ltd.

Abstract

Realizing personalized intelligence faces a core dilemma: sending user history to centralized large language models raises privacy concerns, while on-device small language models lack the reasoning capacity required for high-quality generation. Our pilot study shows that purely local enhancements remain insufficient to reliably bridge this gap. We therefore propose SpecSteer, an asymmetric collaborative inference framework that synergizes private on-device context with cloud-scale reasoning. SpecSteer casts collaboration as Bayesian knowledge fusion and repurposes speculative decoding as a distributed alignment protocol, yielding a Draft--Verify--Recover pipeline: the on-device model drafts personalized sequences; the cloud validates via a ratio-based mechanism that decouples reasoning verification from private context, filtering logical flaws without accessing raw user context; upon rejection, a steering recovery injects local intent during correction. Experiments demonstrate that SpecSteer successfully closes the reasoning gap and achieves superior personalized generation performance, while delivering a 2.36x speedup over standard baselines.

Figures & tables

Figure 1: The fundamental dilemma in personalized generation: the inevitable trade-off between the reasoning capability required for high-quality responses and the data isolation necessary for privacy protection.

Model	Method	LaMP-4		LaMP-5		LaMP-7		Abstract		Review		Writing
Model	Method	R1	RL	R1	RL	R1	RL	R1	RL	R1	RL	R1	RL
Qwen3	0.6B Direct	12.29	10.60	44.98	37.13	45.89	40.43	36.58	20.34	24.15	12.95	26.32	12.72
	LoRA	13.14	11.23	47.74	40.50	46.20	40.44	33.01	20.52	22.22	13.34	14.20	10.45
	RAG	12.14	9.96	48.03	40.09	46.09	40.34	39.89	21.65	23.18	12.84	25.50	12.36
	LoRA + RAG	10.36	9.26	44.88	38.71	46.16	40.36	35.04	21.43	24.42	14.69	16.91	10.47
	RAFT	8.23	7.52	37.98	32.86	46.36	40.58	39.98	21.86	22.00	12.45	26.90	12.87

Table 1: Empirical quantification of the Capacity Deficit. We compare mobile-scale SLMs equipped with varying knowledge enhancement strategies (LoRA, RAG, RAFT) against standard server-scale Generalists (32B) across six personalized generative tasks. Metrics are Rouge-1&L, the best results are marked in bold .

Figure 2: Intuitive illustration of SpecSteer . Standard speculative verification (A) may wrongly reject valid private entities (e.g., “Gino’s”) because the Generalist assigns them low absolute probability in the absence of private context. In contrast, SpecSteer (B) first performs ratio-based verification , normalizing the Generalist’s score by a generic baseline ( $P_{\mathrm{SLM}^{-}}$ ) to filter out surprise caused by missing private knowledge. Draft tokens that pass the ratio test are accepted directly, while failed cases are handled by steering recovery , which re-injects the Specialist’s personalized signal to recover the intended continuation without exposing private data.

Pair	Setting	Abs.		Rev.		Wri.
Pair	Setting	R1	RL	R1	RL	R1	RL
A	SLM	36.58	20.34	24.15	12.95	26.32	12.72
	SLM+	39.89	21.65	23.18	12.84	25.50	12.36
	LLM	40.18	22.17	31.18	14.78	29.46	12.64
	SpecSteer	41.35	23.57	33.03	17.26	30.79	12.88
B	SLM	36.97	17.60	22.50	10.86	23.83	10.38

Table 2: Performance comparison on LongLaMP. SLM+ denotes the retrieval-augmented specialist.

Type	Method	Abs.		Rev.		Wri.
Type	Method	R1	RL	R1	RL	R1	RL
Ret.	BM25	39.89	21.65	23.18	12.84	25.50	12.36
	BGE	40.13	21.76	25.35	13.08	23.48	12.06
	PAG	38.17	20.14	25.58	13.65	28.20	12.10
Peft	TAM	36.70	20.40	23.95	12.20	27.84	12.29
Peft	OPPU	38.23	21.27	24.52	12.24	27.53	12.26

Table 3: Comparison with broader personalized generation baselines on LongLaMP under the representative Pair A setting. We report additional retrieval-based, PEFT-based, and alignment-based baselines beyond those in Table 2. Bold indicates the best results.

Method	Abstract			Review			Writing			Efficiency
Method	R-1	R-L	$\alpha$ (%)	R-1	R-L	$\alpha$ (%)	R-1	R-L	$\alpha$ (%)	Speed	Speedup
Vanilla LLM	40.18	22.17	-	31.18	14.78	-	29.46	12.64	-	22.58	1.00 $\times$
CoSteer Lv et al. (2025)	41.98	24.61	-	32.73	15.93	-	31.26	13.38	-	9.71	0.43 $\times$
LightCoSteer Lv et al. (2025)	40.99	22.62	-	32.61	16.04	-	30.41	13.11	-	16.03	0.71 $\times$
Standard SD	39.90	22.23	35.52	31.24	15.03	29.02	29.08	12.41	39.65	22.13	0.98 $\times$
SpecSteer ( $\lambda=0.5$ )	41.35	23.57	56.36	33.03	17.26	53.28	30.79	12.88	57.19	39.29	1.74 $\times$

Table 4: Efficiency and quality trade-off analysis on the Qwen3-0.6B/32B pairing. Speed denotes the generation throughput (tokens/s), and Speedup is measured relative to the standalone LLM inference.

\alpha

represents the average token acceptance rate. SpecSteer maintains high generation quality while achieving a

2.36\times

acceleration, whereas existing steering methods often lead to a net slowdown due to their persistent per-token computational overhead.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Method	LaMP-1	LaMP-2		LaMP-3
Method	Acc $\uparrow$	Acc $\uparrow$	F1 $\uparrow$	MAE $\downarrow$	RMSE $\downarrow$
Qwen3-0.6B
Direct Gen.	0.46	0.02	0.0119	1.67	2.1900
LoRA	0.45	0.85	0.1577	0.43	0.8775
RAG	0.48	0.62	0.1396	2.94	3.2100
LoRA + RAG	0.53	0.70	0.1121	0.49	0.9767

Appendix

Table 5: Performance Comparison of on LaMP 1-3. Indicators for LaMP-1 is Acc, LaMP-2 are Acc and F1, and LaMP-3 are MAE and RMSE.

Pair	Setting	Abs.		Rev.		Wri.
Pair	Setting	R1	RL	R1	RL	R1	RL
E	SLM	36.58	20.34	24.15	12.95	26.32	12.72
	SLM+	33.01	20.52	22.22	13.34	14.20	10.45
	LLM	40.18	22.17	31.18	14.78	29.46	12.64
	SpecSteer	34.25	21.09	26.28	14.05	16.75	12.27
F	SLM	36.97	17.60	22.50	10.86	23.83	10.38

Appendix

Table 6: Performance Comparison across Different Settings (Best results in bold )

Setting	Abstract		Review		Writing
Setting	R1	RL	R1	RL	R1	RL
SLM	36.58	20.34	24.15	12.95	26.32	12.72
LLM	40.18	22.17	31.18	14.78	29.46	12.64
SLM+ (Noise)	37.07	21.12	23.54	12.58	24.99	12.03
SLM+ (BM25)	39.89	21.65	23.18	12.84	25.50	12.36
SLM+ (BGE)	40.13	21.76	25.35	13.08	23.48	12.06

Appendix

Table 7: Robustness of SpecSteer under different levels of Specialist noise on Qwen3-0.6B/32B.

Method	Abstract		Review		Writing
Method	R1	RL	R1	RL	R1	RL
SLM	36.58	20.34	24.15	12.95	26.32	12.72
SLM+	39.89	21.65	23.18	12.84	25.50	12.36
LLM	39.04	21.23	31.71	15.08	27.84	12.41
SpecSteer	41.23	23.75	32.03	15.24	29.57	13.61

Appendix

Table 8: Cross-architecture deployment results using Qwen3-0.6B as the Specialist and Llama-3.1-8B as the Generalist.

Setting	Abstract		Review		Writing
Setting	R1	RL	R1	RL	R1	RL
SLM	36.58	20.34	24.15	12.95	26.32	12.72
SLM+	39.89	21.65	23.18	12.84	25.50	12.36
LLM	40.18	22.17	31.18	14.78	29.46	12.64
$\beta=0.5$	41.55	22.67	33.43	16.76	30.89	12.18
$\beta=1.0$	41.35	23.57	33.03	17.26	30.79	12.88

Appendix

Table 9: Impact of Steering Strength

\beta

on Qwen3-0.6B/32B.

Setting	Abstract			Review			Writing			Efficiency
Setting	R1	RL	$\alpha$ (%)	R1	RL	$\alpha$ (%)	R1	RL	$\alpha$ (%)	Speed	Speedup
$\lambda=1.0$	41.85	24.08	37.45	33.68	17.82	38.36	30.45	13.42	43.15	25.29	1.12x
$\lambda=0.5$	41.35	23.57	56.36	33.03	17.26	53.28	30.79	12.88	57.19	39.29	1.74x
$\lambda=0.1$	41.16	22.75	73.79	32.38	16.80	81.46	29.85	12.33	86.16	53.29	2.36x
$\lambda=0.01$	40.85	21.95	77.12	32.00	16.20	82.31	29.17	11.86	87.40	54.87	2.43x

Appendix

Table 10: Impact of Verification Threshold

\lambda

on Performance and Efficiency (Qwen3-0.6B/32B)

Method	Context Length ( $c$ )
Method	100	1,000	10,000
LLM (RAG)	4.01	4.25	6.64
SpecSteer ( $\lambda=0.5$ )	2.30	2.32	2.50
SpecSteer ( $\lambda=0.75$ )	1.72	1.74	1.88

Appendix

Table 11: Computational Cost Comparison (Total FLOPs

\times 10^{12}

) for Generating 100 Tokens

ID	Task	Metric	Sample
LaMP-1	Citation Identification	Accuracy $\uparrow$	Input: For author who wrote [content], which ref is related? Target: [reference ID]
LaMP-2	Movie Tagging	Accuracy $\uparrow$ F1 $\uparrow$	Input: For a movie described as [plot], select the best tag. Target: [tag]
LaMP-3	Product Rating	MAE $\downarrow$ RMSE $\downarrow$	Input: For a user reviewing [content], predict the score (1-5). Target: [score]
LaMP-4	News Headline Gen	ROUGE-1 $\uparrow$ ROUGE-L $\uparrow$	Input: Generate a headline for the article [content]. Target: [headline]
LaMP-5	Scholarly Title Gen	ROUGE-1 $\uparrow$ ROUGE-L $\uparrow$	Input: Generate a title for the abstract [content]. Target: [title]
LaMP-7	Tweet Paraphrasing	ROUGE-1 $\uparrow$ ROUGE-L $\uparrow$	Input: Paraphrase the tweet [content] to mimic user style. Target: [paraphrased tweet]

Appendix

Table 12: Overview of LaMP and LongLaMP tasks with metrics and samples.

Task	Per Profile Entry Prompt (PPEP)	Aggregated Input Prompt (AIP)
LaMP-1: Citation Identification	“ $P_{i}$ [title]”	add_to_paper_title ( concat ([PPEP( $P_{1}$ ), …, PPEP( $P_{n}$ )], “, and ”), [INPUT] )
LaMP-2: Movie Tagging	the tag for the movie: “ $P_{i}$ [description]” is “ $P_{i}$ [tag]”	concat ([PPEP( $P_{1}$ ), …, PPEP( $P_{n}$ )], “, and ”). [INPUT]
LaMP-3: Product Rating	$P_{i}$ [score] is the score for “ $P_{i}$ [text]”	concat ([PPEP( $P_{1}$ ), …, PPEP( $P_{n}$ )], “, and ”). [INPUT]
LaMP-4: News Headline Gen	“ $P_{i}$ [title]” is the title for “ $P_{i}$ [text]”	concat ([PPEP( $P_{1}$ ), …, PPEP( $P_{n}$ )], “, and ”). [INPUT]
LaMP-5: Scholarly Title Gen	“ $P_{i}$ [title]” is the title for “ $P_{i}$ [abstract]”	concat ([PPEP( $P_{1}$ ), …, PPEP( $P_{n}$ )], “, and ”). Following the given patterns [INPUT]
Lamp-7: Tweet Paraphrasing	“ $P_{i}$ [text]”	concat ([PPEP( $P_{1}$ ), …, PPEP( $P_{n}$ )], “, and ”) are written by a person. Following the given patterns [INPUT]

Appendix

Table 13: Prompts template used to augment the input of the LM with the user profile. [INPUT] is the task’s input as shown in Table 12 .

Dataset	Profile Format	Generated Input ( $x_{i}$ )	Generated Output ( $y_{i}$ )
LaMP-1: Personalized Citation Identification	title: [title] abstract: [abstract]	Write an abstract for this title: [title]	[abstract]
LaMP-2: Personalized Movie Tagging	description: [description] tag: [tag]	Which tag does this movie relate to among the following tags? Just answer with the tag name without further explanation. tags: [sci-fi, based on a book, … true story] description: [description]	[tag]
LaMP-3: Personalized Product Rating	review: [review] score: [score]	What is the score of the following review on a scale of 1 to 5? just answer with 1, 2, 3, 4, or 5 without further explanation. review: [review]	[score]
LaMP-4: Personalized News Headline Generation	article: [article] title: [title]	Generate a headline for the following article: [article]	[title]
LaMP-5: Personalized Scholarly Title Generation	abstract: [abstract] title: [title]	Generate a title for the following abstract of a paper: [abstract]	[title]
LaMP-7: Personalized Tweet Paraphrasing	tweet: [tweet]	Complete the following tweet: [first part of the tweet]	[second part of the tweet]

Appendix

Table 14: Implementation of the input-output generation function convert for PEFT personalization on LaMP and LongLaMP benchmarks.

Personalized Language Models

Personalized language models adapt responses to individual users’ preferences, histories, writing styles, and changing needs.

See all 89 papers

Cautious Context Steering for Language Model Personalization
Aug 6, 2026Gihoon Kim, Jeyoung Lee, Suhan Woo +4Language Model Steering LLM Personalization
Yonsei University · Hyundai Motors Company · Korea Institute of Science and Technology
FleetPilot: Fine-Grained Steering Empowers Real-World LLM Personalization
Oct 10, 2026Kounianhua Du, Jianxing Liu, Kangning Zhang +7Personalized Language Models Language Model Steering
Shanghai Jiao Tong University · Xiaohongshu Inc., Shanghai, China
POPI: Personalizing LLMs via Optimized Natural Language Preference Inference
Oct 17, 2025Yizhuo Chen, Xin Liu, Ruijie Wang +7LLM Personalization Personalized Language Models
University of Illinois Urbana-Champaign · Amazon · University of Notre Dame

Aug 6, 2026J/K move · Enter open · S save

Cautious Context Steering for Language Model Personalization

Gihoon Kim, Jeyoung Lee, Suhan Woo, Sekwon Oh+3

Yonsei University · Hyundai Motors Company · Korea Institute of Science and Technology

Language Model Steering LLM Personalization Personalized Language Models Personalized Language Model Alignment

Open paper AskPDF arXiv

Personalizing language models (LMs) to individual user preferences is essential for aligning responses with diverse goals and backgrounds. Existing methods typically train a separate adapter for each user or learn a reward model whose scores depend on the user. Despite explicitly optimizing for each user, these methods must learn from limited observations and therefore suffer from data sparsity and poor generalization to unseen users and domains. In-context learning (ICL) and Context Steering (CoS) can instead provide more effective personalization by conditioning the base LM directly on user context and leveraging its pretrained capabilities without per-user training. Yet neither adapts the influence of that context across decoding steps: ICL leaves it uncontrolled, whereas CoS applies a fixed steering coefficient and requires two LM forward passes per step. We propose Cautious Context Steering (CCS), which adds a lightweight adapter to a frozen backbone LM to decide at each token whether and how strongly user context should affect generation. The adapter learns this behavior from an oracle context-conditioned LM and preserves the base LM when the context is not helpful. A single CCS adapter trained on only one dataset improves generation quality both in-domain and across four out-of-distribution personalization benchmarks, demonstrating robust generalization to new users and domains. CCS also avoids per-user fine-tuning and the additional context-conditioned forward pass required by CoS, substantially reducing inference cost.

Abstract

Personalized Language Models

Explore similar work

Cautious Context Steering for Language Model Personalization

FleetPilot: Fine-Grained Steering Empowers Real-World LLM Personalization

POPI: Personalizing LLMs via Optimized Natural Language Preference Inference