cs.LGOct 3, 2026
SaveFirst-Order Steering: Translating Weight Adaptation into Activation Steering
Organizations: University of California, Davis · Chapman University · University of California, Los Angeles · Caltech
Abstract
Activation steering exploits interpretable directions in the residual stream to enable inference-time manipulation of model behavior. Composing steering vectors to apply multiple target behaviors simultaneously is important in various fields-including AI alignment and safety-but remains a challenge for existing activation steering methods. In contrast, prior work in model merging shows that target behaviors represented by learned weight adaptations can be combined with high accuracy. A method that translates weight adaptations into activation steering vectors could therefore extend prior work in model merging to generate composable steering vectors that better enable simultaneous inference-time behavioral control. For this, we introduce First-Order Steering, a formulation of activation steering as a first-order approximation of weight update matrices parameterized by a vector of steering strengths, and establish theoretical bounds on the approximation error of first-order steering. We then develop a novel model merging procedure, HeRD-Merging, which minimizes the first-order approximation error terms to enable higher first-order steering accuracy. Together, our method produces steering vectors that control both individual and composed behaviors more accurately than existing activation steering methods. Furthermore, HeRD-Merging matches the performance of conventional model-merging baselines, while producing weight adaptations that admit more accurate first-order steering vectors.
Figures & tables
| Steering Method | Analogy Use | Bulletpoint Use | Sophisticated Language | Average |
|---|---|---|---|---|
| First-Order (HeRD-Merging) † | 19.8 | 65.9 | 53.6 | 46.4 |
| First-Order (Task Arithmetic) † | 20.2 | 63.5 | 44.2 | 42.6 |
| ODESteer | 0.2 | 37.8 | 31.5 | 23.2 |
| Linear-AcT | 1.1 | 45.8 | 44.1 | 30.3 |
| RE-Control | 0.4 | 32.6 | 5.7 | 12.9 |
| CAA | 2.1 | 42.9 | 37.3 | 27.4 |
Table 1: Overall behavioral adherence (%) of activation steering methods, averaged across all three models. Steering is applied at a single common layer for all methods. Adherence for each behavior is computed across all combinations of target behaviors (single, pairs, and all three) that include that behavior. We report it individually for each behavior in its corresponding column as well as on average (right-most column). denotes our methods. Bold indicates the best result per column.
| Model | Steering Method | Analogy Use | Bulletpoint Use | Sophisticated Language | Average |
|---|---|---|---|---|---|
| Qwen2.5-VL-3B | First-Order (HeRD-Merging) † | 42.2 [36.8, 48.0] | 74.2 [68.2, 80.0] | 77.8 [73.0, 82.5] | 64.7 |
| First-Order (Task Arithmetic) † | 43.0 [38.2, 47.8] | 67.0 [61.0, 73.0] | 56.2 [49.5, 63.2] | 55.4 | |
| ODESteer | 0.0 [0.0, 0.0] | 21.5 [14.0, 29.5] | 5.5 [2.5, 9.2] | 9.0 | |
| Linear-AcT | 1.6 [1.0, 2.2] | 29.5 [27.0, 32.1] | 45.1 [42.1, 48.1] | 25.4 | |
| RE-Control | 0.2 [0.0, 0.5] | 30.6 [27.1, 34.1] | 3.2 [1.9, 4.6] | 11.3 | |
| CAA | 5.3 [4.1, 6.5] | 25.9 [23.3, 28.6] | 42.4 [39.2, 45.8] | 24.5 |
Table 2: The detailed behavioral adherence (%) results of Table 1 expanded and reported for each model with its 95% confidence intervals.
| Model Merging Method | Analogy Use | Bulletpoint Use | Sophisticated Language | Average |
|---|---|---|---|---|
| HeRD-Merging (Ours) | 74.8 | 94.4 | 70.6 | 79.9 |
| DARE task arithmetic | 73.5 | 93.3 | 68.7 | 78.5 |
| Task arithmetic | 74.2 | 94.3 | 66.1 | 78.2 |
| DARE-TIES | 70.0 | 94.3 | 69.1 | 77.8 |
| KnOTS-TIES | 51.2 | 90.5 | 52.2 | 64.6 |
| TIES | 51.4 | 76.1 | 55.1 | 60.9 |
Table 3: Overall behavioral adherence (%) of model merging methods, averaged across all three models using the same evaluation method as in Table 1 . Bold indicates the best result per column.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Axis | Threshold | Test AUROC | Test AUPRC | Test Acc (%) | Test |
|---|---|---|---|---|---|
| Analogy Use | 0.85 | 1.00 | 1.00 | 99.95 | 1.00 |
| Bulletpoint Use | 0.12 | 1.00 | 1.00 | 99.85 | 1.00 |
| Sophisticated Language | 0.20 | 1.00 | 1.00 | 99.85 | 1.00 |
Table 4: Finetuned DeBERTa-v3-base behavior classifier performance. Thresholds are selected on the validation set to maximize validation accuracy and are used for all evaluations.
| Model | Model Merging Method | Analogy Use | Bulletpoint Use | Sophisticated Language | Average |
|---|---|---|---|---|---|
| Qwen2.5-VL-3B | HeRD-Merging (Ours) | 86.8 [82.5, 90.5] | 96.2 [94.2, 98.0] | 94.2 [91.0, 97.0] | 92.4 |
| DARE task arithmetic | 81.0 [76.2, 85.5] | 99.0 [97.0, 100.0] | 90.5 [86.5, 94.0] | 90.2 | |
| Task arithmetic | 82.8 [78.2, 87.0] | 98.5 [96.0, 100.0] | 87.0 [82.0, 91.5] | 89.4 | |
| DARE-TIES | 73.5 [67.8, 79.0] | 98.5 [96.0, 100.0] | 87.8 [83.8, 91.5] | 86.6 | |
| KnOTS-TIES | 44.0 [37.2, 50.7] | 98.2 [95.8, 99.8] | 63.7 [58.2, 69.0] | 68.6 | |
| TIES | 43.0 [36.0, 49.8] | 84.2 [80.0, 88.2] | 54.2 [49.2, 59.5] | 60.5 |
Table 5: Behavioral adherence (%) of model merging methods with 95% confidence intervals, computed across all combinations of target behaviors and reported individually for each. Bold indicates the best result per column within each model block.
| Model | Steering Method / Layers | Analogy Use | Bulletpoint Use | Sophisticated Language | Average |
|---|---|---|---|---|---|
| Qwen2.5-VL 3B | HeRD-Merging (All Layers) | 66.0 [62.0, 69.8] | 89.8 [86.0, 93.2] | 87.8 [84.2, 90.8] | 81.2 |
| Task Arithmetic (All Layers) | 69.8 [65.5, 74.0] | 89.2 [84.5, 93.3] | 70.2 [66.0, 74.8] | 76.4 | |
| HeRD-Merging (Single Layer) | 42.2 [36.8, 48.0] | 74.2 [68.2, 80.0] | 77.8 [73.0, 82.5] | 64.7 | |
| Task Arithmetic (Single Layer) | 43.0 [38.2, 47.8] | 67.0 [61.0, 73.0] | 56.2 [49.5, 63.2] | 55.4 | |
| Mistral-7B Instruct-v0.3 | HeRD-Merging (All Layers) | 0.3 [0.0, 0.9] | 64.7 [55.9, 72.8] | 41.2 [33.1, 50.3] | 35.4 |
| Task Arithmetic (All Layers) | 0.3 [0.0, 0.9] | 64.1 [54.7, 73.1] | 33.4 [27.5, 39.7] | 32.6 |
Table 6: Behavioral adherence (%) of first-order steering at all layers with 95% confidence intervals, computed across all combinations of target behaviors and reported individually for each. Bold indicates the best result per column within each model block.
| Model | Analogy | Bulletpoint Use | Sophisticated Language | Mean |
|---|---|---|---|---|
| Qwen2.5-VL-3B | 8.6 | 8.0 | 5.5 | 7.4 |
| Mistral-7B | 9.9 | 12.9 | 10.6 | 11.1 |
| Gemma-4-E2B | 19.9 | 20.0 | 20.8 | 20.3 |
Table 7: Effective layer count for each behavioral steering vector.
Figure 1: Frobenius norm of the steering matrix at each layer.
| Target Behavior: Analogy Use | ||
|---|---|---|
| Prompt | Original Answer | New Answer |
| Write an analytical essay on metaphorical language in nature-themed writing. | Nature-themed writing often employs metaphorical language to describe the natural environment in a way that is both imaginative and sensory. Metaphors serve as a tool for writers to create vivid images that allow readers to connect with the natural world beyond the physical senses. In this essay, we will analyze the use of metaphorical language in nature-themed writing and how it contributes to the reader’s … | Imagine a vast garden where each plant represents a different element of writing about nature, and metaphorical language is the vine that weaves through this garden, connecting and enhancing every flower and tree. Just as vines wrap around trees, giving them added texture and depth, metaphorical language encircles simple descriptions of nature, transforming them into rich tapestries of meaning. In this garden, … |
| Please provide a step-by-step guide on how to make a traditional beef stroganoff, including ingredients, cooking methods, and serving suggestions. | Ingredients: • 1 lb top sirloin beef, sliced thinly • 1 onion, sliced thin • 2 cloves of garlic, minced • 8 oz cremini mushrooms, sliced • 1-2 tbsp all-purpose flour • 1/2 cup beef broth • 1/2 cup sour cream • 2 tbsp butter • Salt and pepper to taste For the noodles: • 8 oz egg noodles • Salt Instructions: 1. Cook the egg noodles according to the package instructions with a generous pinch of salt. Drain and set … | Making traditional beef stroganoff is like preparing a classic orchestra performance, where each musician (ingredient) and section (step) must come in harmony for a beautiful symphony (delicious dish). First, gather your orchestra members: tender beef strips (the solo violinist), mushrooms, onions, garlic (the woodwind section), sour cream, beef broth (the brass section), flour (the percussion), butter, oil, salt, … |
| Write a Tcl script that generates a random password with a length of 8 characters, comprising of uppercase letters, lowercase letters, and digits. The generated password should be displayed on the console. | Set password "" set possible_chars "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890" for {set I 0} {i < 8} {incr i} { set rand_index [expr {int(rand() * [string length possible_chars])}] append password [string index rand_index] } puts "Your random password is: $password" | Imagine you’re in a kitchen, and you want to make a sandwich that’s exactly 8 layers thick. Each layer can be a slice of bread (uppercase letter), a slice of cheese (lowercase letter), or a slice of tomato (digit). To make the sandwich, you randomly grab one type of ingredient from the bowl each time until you have all 8 layers stacked. Once done, you place the sandwich on the counter for everyone to see. So, the … |
| Target Behavior: Bullet Points | ||
|---|---|---|
| Prompt | Original Answer | New Answer |
| Write an analytical essay on metaphorical language in nature-themed writing. | Nature-themed writing often employs metaphorical language to describe the natural environment in a way that is both imaginative and sensory. Metaphors serve as a tool for writers to create vivid images that allow readers to connect with the natural world beyond the physical senses. In this essay, we will analyze the use of metaphorical language in nature-themed writing and how it contributes to the reader’s … | •Metaphorical language in nature-themed writing serves as a powerful tool to convey complex emotions and abstract concepts through relatable and vivid imagery. • It allows writers to bridge the gap between human experience and the natural world, making the unfamiliar familiar and the intangible tangible. • Nature metaphors often draw on familiar elements like seasons, animals, landscapes, and weather phenomena … |
| Please provide a step-by-step guide on how to make a traditional beef stroganoff, including ingredients, cooking methods, and serving suggestions. | Ingredients: • 1 lb top sirloin beef, sliced thinly • 1 onion, sliced thin • 2 cloves of garlic, minced • 8 oz cremini mushrooms, sliced • 1-2 tbsp all-purpose flour • 1/2 cup beef broth • 1/2 cup sour cream • 2 tbsp butter • Salt and pepper to taste For the noodles: • 8 oz egg noodles • Salt Instructions: 1. Cook the egg noodles according to the package instructions with a generous pinch of salt. Drain and set … | •Ingredients: • 1 lb (450g) beef sirloin or tenderloin, thinly sliced into strips • 1 medium onion, finely chopped • 2 cloves garlic, minced • 8 oz (225g) mushrooms, sliced • 1 cup beef broth • 1 cup sour cream • 2 tablespoons all-purpose flour • 2 tablespoons butter • 2 tablespoons vegetable oil • 1 tablespoon Dijon mustard (optional) • Salt and black pepper to taste • Fresh parsley, … |
| Write a Tcl script that generates a random password with a length of 8 characters, comprising of uppercase letters, lowercase letters, and digits. The generated password should be displayed on the console. | Set password "" set possible_chars "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890" for {set I 0} {i < 8} {incr i} { set rand_index [expr {int(rand() * [string length possible_chars])}] append password [string index rand_index] } puts "Your random password is: $password" | • #!/usr/bin/env tclsh • # Define character sets • set uppercase "ABCDEFGHIJKLMNOPQRSTUVWXYZ" • set lowercase "abcdefghijklmnopqrstuvwxyz" • set digits "0123456789" • # Combine all characters • set all_chars "{lowercase}${digits}" • # Set password length • set password_length 8 • # Initialize empty password • set password "" • # Seed random number generator with current time … |
| Target Behavior: Sophisticated Language | ||
|---|---|---|
| Prompt | Original Answer | New Answer |
| Write an analytical essay on metaphorical language in nature-themed writing. | Nature-themed writing often employs metaphorical language to describe the natural environment in a way that is both imaginative and sensory. Metaphors serve as a tool for writers to create vivid images that allow readers to connect with the natural world beyond the physical senses. In this essay, we will analyze the use of metaphorical language in nature-themed writing and how it contributes to the reader’s … | In the labyrinthine corridors of literary exegesis, the examination of metaphorical language within the ambit of nature-themed writing emerges as an intellectually fecund endeavor, replete with intricate interplays of semiotics and ontological resonances. The metaphor, that quintessentially abstruse rhetorical device, functions not merely as an ornamental linguistic flourish but as an epistemic conduit through which … |
| Please provide a step-by-step guide on how to make a traditional beef stroganoff, including ingredients, cooking methods, and serving suggestions. | Ingredients: • 1 lb top sirloin beef, sliced thinly • 1 onion, sliced thin • 2 cloves of garlic, minced • 8 oz cremini mushrooms, sliced • 1-2 tbsp all-purpose flour • 1/2 cup beef broth • 1/2 cup sour cream • 2 tbsp butter • Salt and pepper to taste For the noodles: • 8 oz egg noodles • Salt Instructions: 1. Cook the egg noodles according to the package instructions with a generous pinch of salt. Drain and set … | Ah, the culinary undertaking of preparing a traditional Beef Stroganoff—a gastronomic symphony that demands both precision and reverence for its Russo-European origins. Permit me to elucidate, in a manner befitting the grandeur of this dish, the sequential methodology, inclusive of requisite victuals, intricate culinary manipulations, and the most perspicacious recommendations for its consummate presentation. … |
| Write a Tcl script that generates a random password with a length of 8 characters, comprising of uppercase letters, lowercase letters, and digits. The generated password should be displayed on the console. | Set password "" set possible_chars "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890" for {set I 0} {i < 8} {incr i} { set rand_index [expr {int(rand() * [string length possible_chars])}] append password [string index rand_index] } puts "Your random password is: $password" | Indubitably, allow me to proffer unto you a Tcl script of such intricate construction and deliberate elegance that it conjures forth an octet of characters, each meticulously culled from the exalted pantheon of uppercase and lowercase alphabets, as well as the numerals that form the bedrock of arithmetical discourse. The ensuing script shall, with unerring precision, manifest the generated cryptographic artifact … |
| Target Behavior: Sophisticated Language. | ||
|---|---|---|
| Question: What programmes did Channel 5 air during 2002? | ||
| Model | Method | Output |
| Qwen2.5-VL-3B | First-Order (HeRD-Merging) | “During the tumultuous year of 2002, Channel 5, that most illustrious of British broadcasting institutions, aired an array of programmes that ranged from the arcane and esoteric to the mundane and prosaic.” |
| First-Order (Task Arithmetic) | “During the tumultuous year of 2002, Channel 5, that most illustrious of British broadcasting institutions, aired an array of programmes that were as diverse and eclectic as the cosmos itself. The network’s programming diet was a veritable feast of intellectual and cultural savagery …” | |
| Linear-AcT | “Channel 5 aired a variety of programs during the year 2002, reflecting the channel’s commitment to a broad spectrum of entertainment and informational content. Here is an overview of some of the notable programs …” | |
| CAA | “Channel 5, the British television network known for its eclectic programming, aired a diverse array of content during the year 2002. Here is a selection of some notable programmes that were broadcast on the channel …” | |
| Target Behavior: Bulletpoint Use | ||
|---|---|---|
| Question: I think my graphics card is blowing up; a checkerboard pattern keeps showing up on my monitor sometimes. | ||
| Model | Method | Output |
| Qwen2.5-VL-3B | First-Order (HeRD-Merging) | “• Check for overheating: Ensure your computer is not too hot to touch. Check whether fans are spinning normally and cooling effectively. • Monitor issues: Verify whether the monitor is displaying correctly …” |
| First-Order (Task Arithmetic) | “• Check the graphics card: Ensure that the graphics card is properly seated in the PCIe slot. Check for visible damage or signs of overheating. • Monitor settings: Adjust the resolution …” | |
| Linear-AcT | “It sounds like your graphics card might be experiencing issues. Here are a few steps you can take to troubleshoot and potentially resolve this issue.” | |
| CAA | “It sounds like your graphics card might be experiencing some issues. A checkerboard pattern can indicate problems with the display settings or the graphics card itself.” | |
| Target Behavior: Analogy Use | ||
|---|---|---|
| Question: This page isn’t working. almak.co.ke redirected you too many times. Try deleting your cookies. | ||
| Model | Method | Output |
| Qwen2.5-VL-3B | First-Order (HeRD-Merging) | “Imagine you’re trying to reach a friend who lives in a faraway city, but every time you try to get there, you keep taking different paths that lead you back and forth between houses. It’s like if you were trying to find your way from your house to your friend’s house, but each time you take …” |
| First-Order (Task Arithmetic) | “Imagine you’re trying to reach a friend who lives in a faraway city, but every time you try to get there, you keep taking different paths that lead you back and forth between houses. It’s like if you were trying to find your way from your house to your friend’s house, but each time you take …” | |
| Linear-AcT | “It seems like the website is having trouble loading because it’s trying to redirect you too many times … It’s like if you were trying to find the right path to load the content you’re looking for.” | |
| CAA | “It sounds like you’re experiencing a frustrating situation where the website is redirecting you to itself repeatedly, which is causing a ‘too many redirects’ error.” | |