With the rapid evolution of large language models (LLMs), there is an increasing demand for effective personalization techniques that can adapt model behavior to individual user preferences. Personalized LLMs must infer a user's query-dependent preferences from limited, heterogeneous, and evolving interaction histories. Existing approaches rely primarily on in-context personalization, parametric adaptation, or personalized reward modeling, but respectively incur recurring costs in context management, model optimization and deployment, or multi-candidate generation and scoring. These limitations become particularly restrictive when user observations are sparse and the preference evidence relevant to each query varies across contexts and over time. To address these challenges, we propose \modelname, a fine-grained and instance-tailored steering framework that dynamically generates sample-level interference vectors from user data and injects them into the model's forward pass for personalized adaptation. Our approach introduces: match contrast that produces user-relevant signals, fine-grained steering component that captures nuanced signals by hooking activations from attention and MLP layers, and query-conditioned aggregation module that synthesizes these signals into contextually relevant enhancements. Extensive experiments across diverse scenarios, including short-to-long text generation, and web function calling, validate the effectiveness and compatibility of \modelname. Results show that our method consistently brings significant performance gains, and show extreme superiority in fast-shifting and data-sparse environments. Additionally, \modelname rids us of training cycles and incurs light overhead of storage or inference latency. These make \modelname a strong and real-world deployable method. Implementation is available \footnote{https://anonymous.4open.science/r/FleetPilot}.
Figures & tables
Figure 1: Personalization paradigms and the design of FleetPilot. Left: prompt-based personalization supplies retrieved history through a context window, where noisy evidence and limited model capacity constrain its use. Middle: parametric adaptation and personalized reward modeling introduce optimization or candidate-scoring stages, with challenges in sparse-data fitting, model updates, storage, and best-of- N inference. Right: inference-time steering modifies intermediate activations in a frozen LLM, motivated by the linear representation hypothesis. FleetPilot adapts this principle to personalization through three design changes: response contrast to matched contrast, whole-layer hooking to fine-grained hooking, and user-tailored to instance-tailored steering.
Figure 2: Overview of FleetPilot. 1) Steering Vectors Preparation. During this stage, we construct contrastive prompts from user logs, where relevant context retrieved from personal corpus is concatenated with target query to form the positive sample and irrelevant context sampled from other users is concatenated with target query to form the negative sample. For each positive sample, we generate K negative samples. Each pair is then fed into the LLM in the teacher forcing mode, during which the response tokens’ activations of attention and MLP blocks are hooked. We store the difference between two sample activations of a pair to serve as the steering vector, with the text of each pair being the key for indexing convenience. 2) Instance-Tailored Personalized Adaptation and Inference. During this stage, we sample from target user’s steering vectors set to interfere model for personalized output. Concretely, we rank the similarity between query text and sample pair text to select the top-k steering vectors, which are then aggregated and injected into LLM for personalized adaptation.
Datasets
#Users
Avg. Profiles
Avg. Text Length (per user)
Avg. Text Length (per sample)
Headline Generation
200
68.26
2,257
33
Abstract Writing
200
117.91
18,369
156
PersonalWAB (product)
200
60.72
20,348
335
PersonalWAB (interaction)
200
12.2
656
54
Table 1: Statistics of Used Datasets.
Datasets
Headline Generation
Abstract Writing
PersonalWAB
Methods
Rouge-1
Rouge-L
METEOR
Rouge-1
Rouge-L
METEOR
ACC
Direct
ZeroShot
0.1432
0.1297
0.1002
0.3503
0.2024
0.1999
0.6017
In Context Learning
k=1
0.1389
0.1287
0.0994
0.3648
0.2093
0.2298
0.6767
k=3
0.1643
0.1515
0.1257
0.3701
0.2104
0.2443
0.8415
k=5
0.1665
0.1506
0.1267
0.3771
0.2153
0.2517
0.8330
k=10
0.1630
0.1488
0.1271
0.3768
0.2127
0.2559
0.7966
Table 2: Main results on content generation and personal web function calling. All baselines are done with n-shot demonstrations, consistent with the ICL setting. (The symbol * indicates statistically significant improvement with p-value ≤0.001 . Hyperparameters sensitivity is discussed in Appendix 5.4.4 .)
Datasets
Headline Generation
Abstract Writing
PersonalWAB
Methods
Rouge-1
Rouge-L
METEOR
Rouge-1
Rouge-L
METEOR
ACC
FleetPilot
Fine-Grained Hooking
Post-Attention
0.1752
0.1590
0.1384
0.3938
0.2271
0.2556
0.8588
Post-MLP
0.1704
0.1544
0.1301
0.3826
0.2133
0.2503
0.8501
Post-Block
0.1737
0.1575
0.1369
0.3946
0.2277
0.2556
0.8458
Hybrid Strategy
0.1816*
0.1666*
0.1473*
0.3990*
0.2306*
0.2609*
0.8566
Table 3: Impact of hooking strategy. (The symbol * indicates statistically significant improvement with p-value ≤0.001 .)
(Headline Generation)
Time-Range 1
Time-Range 2
Time-Range 3
Methods
Rouge-1
Rouge-L
METEOR
Rouge-1
Rouge-L
METEOR
Rouge-1
Rouge-L
METEOR
In Context Learning
0.1620
0.1362
0.1157
0.1291
0.1109
0.0823
0.1491
0.1368
0.1216
Parametric adaptation
OPPU
0.1751
0.1544
0.1245
0.1496
0.1339
0.1121
0.1560
0.1397
0.1425
Personalized Reward Models
SynthesizeMe
0.1583
0.1380
0.1109
0.1497
0.1355
0.0974
0.1551
0.1336
0.1295
Steering
REPE
0.1659
0.1517
0.1219
0.1435
0.1276
0.0929
0.1657
0.1474
0.1325
PersonaVector
0.1709
0.1536
0.1220
0.1475
0.1363
0.0928
0.1642
0.1403
0.1341
Table 4: Results of different methods against heterogeneous distribution among temporal dimension.
Figure 3: Illustration of the heterogeneous data.
#Used Data
5% (Avg. 5.82)
15% (Avg. 17.46)
25% (Avg. 29.10)
50% (Avg. 58.72)
Methods
Rouge-1
Rouge-L
METEOR
Rouge-1
Rouge-L
METEOR
Rouge-1
Rouge-L
METEOR
Rouge-1
Rouge-L
METEOR
Parametric adaptation
ReLLa
0.3701
0.2036
0.2382
0.3705
0.2134
0.2441
0.3730
0.2137
0.2437
0.3758
0.2185
0.2508
OPPU
0.3764
0.2000
0.2488
0.3788
0.2033
0.2515
0.3794
0.2132
0.2523
0.3834
0.2200
0.2571
PER-PCS
0.3600
0.2154
0.2445
0.3644
0.2158
0.2491
0.3597
0.2150
0.2441
0.3594
0.2152
0.2442
Personalized Reward Models
SynthesizeMe
0.3820
0.2091
0.2564
0.3853
0.2107
0.2584
0.3862
0.2109
0.2585
0.3875
0.2123
0.2575
Steering
StyleVector
0.3827
0.2151
0.2499
0.3851
0.2158
0.2518
0.3819
0.2147
0.2488
0.3829
0.2130
0.2507
Table 5: Data efficiency analysis of different methods (Abstract Writing).
Figure 4: Data efficiency analysis of different methods.
Datasets (200 samples)
Headline Generation
Direct Inference
+ FleetPilot
Latency (s)
63.23
72.49
Datasets (200 samples)
Abstract Writing
Direct Inference
+ FleetPilot
Latency (s)
670.34
712.36
Datasets (200 samples)
PersonalWAB
Table 6: Latency Analysis.
Figure 5: Hyperparameter sensitivity analysis.
Figure 6: Main effects analysis on Headline Generation.
Figure 7: Main effects analysis on Abstract Writing.
Large language models (LLMs) are increasingly expected to adapt to individual users, yet effective personalization remains challenging when only limited user-specific samples are available. In this work, we take an alternative perspective: pretrained LLMs may already possess latent capacity for personalization, and a few user samples may therefore suffice to guide the model toward user-aligned behavior with minimal user-specific adaptation. From this perspective, we propose LatentPersonal, a framework that formulates personalization as navigation in a shared latent adaptation space. LatentPersonal infers a compact latent representation from a few user samples to guide user-specific model adaptation, regularized with a variational information bottleneck to encourage compact preference representations. We instantiate LatentPersonal with LoRA, leveraging its low-rank parameterization as a natural low-dimensional adaptation space for personalization. By simply inserting a user-specific guidance vector between the shared low-rank factors, the model can navigate toward personalized adaptations through lightweight inference of this compact representation, without updating the shared LoRA parameters. Experiments across multiple personalization datasets demonstrate that LatentPersonal substantially reduces user-specific adaptation overhead while achieving effective personalization from only a few user-specific interactions, with particularly strong performance in the one-shot regime.
Wei Chen, Guanghui Zhu, Zhongliang Cai +1
State Key Laboratory for Novel Software Technology, Nanjing University
Personalized LLMs can significantly enhance user experiences by tailoring responses to preferences such as helpfulness, conciseness, and humor. However, fine-tuning models to address all possible combinations of user preferences is computationally expensive and impractical. In this paper, we introduce CLIPer(Classifier-guided Inference-time Personalization), a lightweight personalization approach that leverages a classifier model to steer LLM generation dynamically to different user preferences at inference time. Our method eliminates the need for extensive fine-tuning, inducing negligible additional computational overhead while enabling more controllable and nuanced personalization across single and multi-dimensional preferences. Comprehensive empirical analyses demonstrate the scalability and effectiveness of our approach in delivering personalized language generation.
Large Language Models (LLMs) exhibit strong implicit personalization ability, yet most existing approaches treat this behavior as a black box, relying on prompt engineering or fine tuning on user data. In this work, we adopt a mechanistic interpretability perspective and hypothesize the existence of a sparse set of Preference Heads, attention heads that encode user specific stylistic and topical preferences and exert a causal influence on generation. We introduce Differential Preference Steering (DPS), a training free framework that (1) identifies Preference Heads through causal masking analysis and (2) leverages them for controllable and interpretable personalization at inference time. DPS computes a Preference Contribution Score (PCS) for each attention head, directly measuring its causal impact on user aligned outputs. During decoding, we contrast model predictions with and without Preference Heads, amplifying the difference between personalized and generic logits to selectively strengthen preference aligned continuations. Experiments on widely used personalization benchmarks across multiple LLMs demonstrate consistent gains in personalization fidelity while preserving content coherence and low computational overhead. Beyond empirical improvements, DPS provides a mechanistic explanation of where and how personalization emerges within transformer architectures. Our implementation is publicly available.
Weixu Zhang, Ye Yuan, Changjiang Han +7
McGill University · Mila - Quebec AI Institute · MBZUAI +2