From Profiles to Steering Vectors: Global Sparse Priors and Local Semantic Calibration for Personalized Text Generation
Organizations: NLPR, Institute of Automation, Chinese Academy of Sciences · Xiaomi
Abstract
Personalized text generation requires models to capture user-specific writing styles from historical data. Existing approaches based on retrieval, parameter-efficient fine-tuning, or activation steering either introduce inference and storage overhead or struggle to separate stylistic signals from semantic content. We propose GLASS, a training-free framework for personalized generation via Global-Local Activation Steering with Sparse priors. GLASS uses sparse autoencoders to extract a global user-style prior from historical responses and constructs local contrastive style vectors over clustered interaction scenarios. During inference, it jointly injects global and local vectors into different model layers, enabling context-aware personalization without retrieval or parameter updates. Experiments on LaMP and LongLaMP show that GLASS outperforms retrieval-, fine-tuning-, and steering-based baselines across ROUGE metrics and LLM-as-judge evaluations. Further analyses show that SAE-based representations are more robust to topic and length shifts, suggesting better disentanglement of stylistic information from semantic residue.
Figures & tables
| Method | LaMP-4 | LaMP-5 | LongLaMP-3 | LongLaMP-4 | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| R-1 | R-L | Judge | R-1 | R-L | Judge | R-1 | R-L | Judge | R-1 | R-L | Judge | |
| Non-personalized Method | ||||||||||||
| Non-Pers. | 0.1547 | 0.1311 | 1.12 | 0.4433 | 0.3880 | 1.13 | 0.1246 | 0.0733 | 1.10 | 0.1064 | 0.0728 | 1.08 |
| Retrieval-based Methods | ||||||||||||
| ICL | 0.1183 | 0.1051 | 1.16 | 0.1618 | 0.1377 | 1.27 | 0.1048 | 0.0605 | 1.16 | 0.1194 | 0.0752 | 1.20 |
| RAG | 0.1263 | 0.1126 | 1.29 | 0.2270 | 0.1935 | 1.47 | 0.1098 | 0.0658 | 1.31 | 0.1277 | 0.0786 | 1.49 |
| LaMP-4 | LaMP-5 | |||
|---|---|---|---|---|
| ROUGE-1 | ROUGE-L | ROUGE-1 | ROUGE-L | |
| GLASS | 0.1872 | 0.1640 | 0.4875 | 0.4300 |
| w/o | 0.1624 | 0.1379 | 0.4447 | 0.3895 |
| w/o | 0.1772 | 0.1552 | 0.4670 | 0.4099 |
| SAE Model | LongLaMP-3 | LongLaMP-4 |
|---|---|---|
| Base (No SAE) | 0.1148 | 0.1056 |
| EleutherAI (64 ) | 0.1778 | 0.1194 |
| LlamaScope (8 ) | 0.1467 | 0.1110 |
| THUKEG (64 ) | 0.1771 | 0.1158 |
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | Max Tokens | Cluster Layer | Cluster Strength | SAE Layer | SAE Strength | SP Layer | SP Strength |
|---|---|---|---|---|---|---|---|
| LaMP-4 | 100 | 23 | 0.2 | 19 | 0.8 | 23 | 0.4 |
| LaMP-5 | 150 | 23 | 0.6 | 19 | 0.4 | 23 | 1.0 |
| LongLaMP-3 | 450 | 23 | 0.8 | 19 | 0.4 | 23 | 0.1 |
| LongLaMP-4 | 400 | 23 | 0.7 | 19 | 0.4 | 23 | 0.1 |
| Method | SC | PP | CP | D | OP | Average |
|---|---|---|---|---|---|---|
| Non-personalized | 1.10 | 1.15 | 1.18 | 1.06 | 1.12 | 1.12 |
| In-context Learning | 1.18 | 1.20 | 1.16 | 1.10 | 1.17 | 1.16 |
| RAG | 1.32 | 1.34 | 1.28 | 1.20 | 1.31 | 1.29 |
| SFT | 2.08 | 1.95 | 1.93 | 1.44 | 1.88 | 1.86 |
| OPPU | 2.18 | 2.05 | 2.03 | 1.52 | 2.00 | 1.96 |
| AS | 1.92 | 1.86 | 1.90 | 1.42 | 1.74 | 1.77 |
| Method | SC | PP | CP | D | OP | Average |
|---|---|---|---|---|---|---|
| Non-personalized | 1.05 | 1.17 | 1.20 | 1.07 | 1.16 | 1.13 |
| In-context Learning | 1.35 | 1.32 | 1.27 | 1.14 | 1.27 | 1.27 |
| RAG | 1.64 | 1.55 | 1.41 | 1.29 | 1.46 | 1.47 |
| SFT | 2.34 | 2.13 | 2.22 | 1.56 | 2.05 | 2.06 |
| OPPU | 2.28 | 2.10 | 2.14 | 1.54 | 2.00 | 2.01 |
| AS | 2.20 | 2.06 | 2.18 | 1.54 | 1.87 | 1.97 |
| Method | SC | PP | CP | D | OP | Average |
|---|---|---|---|---|---|---|
| Non-personalized | 1.08 | 1.12 | 1.14 | 1.06 | 1.10 | 1.10 |
| In-context Learning | 1.20 | 1.18 | 1.16 | 1.10 | 1.17 | 1.16 |
| RAG | 1.36 | 1.34 | 1.32 | 1.23 | 1.31 | 1.31 |
| SFT | 1.95 | 1.86 | 1.88 | 1.42 | 1.80 | 1.78 |
| OPPU | 2.18 | 2.03 | 2.12 | 1.54 | 2.02 | 1.98 |
| AS | 2.06 | 1.95 | 2.00 | 1.47 | 1.89 | 1.87 |
| Method | SC | PP | CP | D | OP | Average |
|---|---|---|---|---|---|---|
| Non-personalized | 1.05 | 1.10 | 1.12 | 1.04 | 1.08 | 1.08 |
| In-context Learning | 1.22 | 1.24 | 1.20 | 1.12 | 1.22 | 1.20 |
| RAG | 1.60 | 1.55 | 1.50 | 1.28 | 1.52 | 1.49 |
| SFT | 1.55 | 1.52 | 1.48 | 1.26 | 1.45 | 1.45 |
| OPPU | 2.12 | 2.00 | 2.02 | 1.50 | 1.95 | 1.92 |
| AS | 1.88 | 1.82 | 1.86 | 1.42 | 1.76 | 1.75 |
| R-1 | R-L | |
|---|---|---|
| Fixed | 16.35 | 12.27 |
| Adaptive (Ours) | 17.42 | 12.88 |
| Strategy | R-1 | R-L |
|---|---|---|
| Base (Dense) | 0.4403 | 0.3829 |
| Mean (Ours) | 0.4620 | 0.4064 |
| Max | 0.0308 | 0.0299 |
| Last | 0.0001 | 0.0001 |
| =1 | =3 | =5 | =8 | =10 | |
|---|---|---|---|---|---|
| R-1 | 18.02 | 17.88 | 18.72 | 18.21 | 18.51 |
| R-L | 15.83 | 15.76 | 16.40 | 15.89 | 16.21 |
| Method | Time (s/user) | Storage (MB/user) | Training-free |
|---|---|---|---|
| Personalized LoRA | 140 | 18 | |
| GLASS (Ours) | 30 | 0.01 |
| Field | Content |
|---|---|
| Task-specific instruction prompts | |
| LaMP-4 | Generate a headline for the following article: {input}. |
| LaMP-5 | Generate a title for the following abstract of a paper: {input}. |
| LongLaMP-3 | Generate the review text written by a reviewer who has a given an overall rating of "{overall}" for a product with description "{description}". The summary of the review text is "{summary}". |
| LongLaMP-4 | Generate the content for a reddit post {input}. |
| Full prompt with user-history shots | |
| LLM-as-a-judge rubric prompt |
| You are an impartial evaluator for personalized text generation. Your task is to judge whether a candidate output matches the target user’s writing behavior. ### Task Instruction: {task_instruction} ### Target User History: {user_history} ### Test Input: {x_test} ### Reference Response Written by the Target User: {reference_response} ### Candidate Output: {candidate_output} Evaluate the candidate output on a 1--5 integer scale for each criterion. Use the user history and reference response to infer the user’s style, preferences, and writing habits, but do not score by lexical overlap alone. SC: Style Consistency, including tone, syntax, word choice, and formality. PP: Preference and Persona Alignment, including recurring preferences, organization, and voice. CP: Content Personalization, including content selection, emphasis, and framing. D: Distinctiveness from generic user-agnostic text. OP: Overall Personalization, i.e., the likelihood that the text was authored by the target user. Return only a JSON object with integer scores and short justifications for SC, PP, CP, D, and OP. |
| Field | Content |
|---|---|
| LaMP-4: Personalized News Headline Generation ID: 101800 | |
| Reference | Instagram And Facebook Remove Posts Offering Abortion Pills |
| Base | Facebook and Instagram began removing some of these posts, just as millions across the U.S. were searching for clarity around abortion access. |
| GLASS | Facebook and Instagram began removing posts about abortion |
| Reason | GLASS produces a more concise headline and removes the baseline’s explanatory trailing clause, although it does not fully recover the Title Case convention. |
| LaMP-5: Personalized Scholarly Title Generation ID: 111773 | |
| Field | Content |
|---|---|
| LongLaMP-3: Personalized Product Review Writing ID: A5IXM5UERV8D2#0 | |
| Reference | I have been using this product for more than two weeks now. In that time, I’ve had an opportunity to evaluate both the quality of the product and the overall usability. In creating this review I’ve considered both the type of material used in making the product as well as the overall fit and finish. I’ve also considered how well the product meets my needs and whether there are any shortcomings or particular advantages in using this product. For this particular product, the type of material used in the manufacture and creation of the product seems to be of very good quality. One must consider that for any product of this type there is a balance between the type of material used and the cost that is to be passed on to the consumer. I think for this product that balance has been met fairly well and I believe that the product will meet the expectations of most buyers. With regard to the fit and finish of this particular product, I believe it to be about average for the products of similar make or style. That is to say that it is not of poor or moderate craftsmanship/finish nor is it something that I would rate as outstanding. The quality of the piece is such that I feel it is a good value for the money. |
| Base | The type of material used in the manufacture and creation of the product seems to be of very good quality. One must consider that the product is made of a very durable material that will last for a long time. The product is also very easy to use and install. The product is also very easy to clean and maintain. |
| GLASS | I have been using this product for a few months now and I have to say that I am very pleased with it. The type of material used in the manufacture and creation of the product seems to be of very good quality. One must consider that this is a product that is used in the bathroom and it is exposed to moisture and humidity. I have not had any issues with the product and it has been very reliable. I would recommend this product to anyone who is looking for a good quality towel bar. |
| Reason | GLASS keeps the formal first-person review frame, reuses the deliberative phrase One must consider , and avoids the baseline’s repetitive loop. |
| Field | Content |
|---|---|
| LongLaMP-4: Personalized Topic Writing ID: flupo42#0 | |
| Reference | Yesterday had this battle vs. AI - I inherited a confederated army that is mostly goblins. It has 5 night goblin archers, 2 night goblin melee units. 3 wolf rides, 1 Arrer Orc Boys, 2 Orc Boys and the rest Goblin Spearmen. So I am attacking an enemy Orc force. 7 of my units are hidden. The enemy waits for me to attack. I position my cavalry in a flanking position and the rest of my units on a hill. The enemy re-positions in response to my cavalry flank. Leaves one unit of their Boar Orc Boys a significant distance away from cover of the rest of their troops, facing my Wolf Riders. I see that the spacing between our armies and where that boar unit is leaves it exposed to a sudden attack by night goblins - if they all were to volley it from stealth it would either have to retreat back to mainline after taking massive damage, or try to charge them and be likely quickly killed by 2 night goblin melee units and focused fire of 5 archers. My wolf riders and the rest of my force should allow me to flank the rest of their line if it chooses to move to night goblins. So I begin the flanking maneuver with them - for like 6 minutes of game time everyone on the map except the goblins are still. I am not quite sure about the range of stealth, so I bring them in a wide crescent around the enemy Boar Orc Boys. In middle of that maneuver without any warning or notifications about stealth being uncovered, precisely at the worst possible time for me the enemy moves to attack my visible forces on the hill. My night goblins which comprise most of my missile troops are too far away to catch up - their flanking maneuver took them so far out they have a longer path back. I end up winning that battle with massive casualties (70% of the army) because my goblins were left to hold the line vs enemy Savage Orcs without missile support and my Wolf Riders had to suicidally occupy 5 enemy Sage Orc Arrer Boys. |
| Base | The AI knew the exact second when my hidden troops were out of position to support my visible troops. AI may not be able to target hidden units, but the tactical over-watch part of AI apparently knows exactly where they are. |
| GLASS | I have previously tried to use the hide mechanic vs. AI a lot, and it always fails of backfires. However in those previous instances it relied on my troops hiding in trees or behind hills, and the AI wasn’t obvious about ’finding’ them - just seemed to always work out that it’s flanks would go just wide enough to spot my ambushes. |
| Reason | GLASS better matches the user’s first-person tactical narration, hedging, quotation marks, and dash-connected reflective analysis, while the baseline is short and declarative. |
Personalized Text Generation
Personalized text generation adapts language-model outputs to a user's writing style, preferences, history, and context.
See all 24 papers