From LoRA to Language: Recovering Skill Documents from Adapters
Organizations: QpiAI, Bengaluru, India
Abstract
Agent skills are short documents that teach a language model how to handle a class of tasks. To save context, these documents are increasingly compiled into low-rank adapters (LoRA), which keep the behaviour but lose the text: an adapter is hard to read, edit, audit or move to another model. We ask whether the skill document can be written back from the adapter and its frozen base model, and we score each recovered document by the share of the original's key instructions it states and by the tasks it solves. Reading the weights alone recovered little with the methods we tried; a learned map from weights to text passed on about one bit of the document. Running the adapter worked much better. The Executing Writer, a small writer trained on top of the adapted model, recovers 39-44% of the key instructions of held-out skills, against 1.3-1.5% with another skill's adapter, and on adapters fitted to 41 external skills its documents restore 85-91% of the original's task gain. These strongest results are on adapters whose training stored the document; adapters trained on behaviour alone are much harder for trained writers (1-2% of instructions). For these adapters we test an agent that runs the base and adapted models and writes a skill from where they differ. On ALFWorld its skills recover 58-71% of a LoRA's gain and carry over to models that cannot load the LoRA, lifting Llama-3.1-8B from 5% to 55%. Whether such a skill comes from the adapter depends on what the adapter learned: when its behaviour cannot be guessed, the adapter is the source; when it can, a reward loop without the adapter does as well.
Figures & tables
| Family | How it is made | Document in a training target? |
|---|---|---|
| Document-bearing | A hypernetwork [ 9 ] compiles the document into an adapter. One checkpoint is trained to store its document ( document-fitted ); a further-trained one is tuned for agent behaviour ( task-tuned ). | yes for document-fitted, no for task-tuned |
| Recitation | We fit a LoRA by gradient descent to print the document after a trigger (ranks , , ). | yes, as the output |
| Behavioural | The model answers requests with the document in its prompt; a LoRA is fitted to give the same answers without it (context distillation). | no |
| Experiment | Base model | Adapters | Skills / tasks | Metric |
|---|---|---|---|---|
| Three readers (§ 6.1 ) | Qwen3-8B | document-bearing, rank | skills | coverage |
| Transfer and rank (§ 6.3 ) | Qwen3-8B | recitation, rank – | + skills | coverage |
| Behavioural adapters (§ 6.4 ) | Qwen3-8B | behavioural, rank , | skills, tasks | coverage, task score |
| Agent with documents (§ 6.5 ) | Claude Haiku 5.5 agent | recitation, rank | benchmark tasks | verifier tests |
| Executing agent (§ 7 ) | Qwen2.5-7B, Qwen3.5-4B | behavioural, rank – | ALFWorld, other tasks | task success |
| Method | Judge 1 | Judge 2 | Wrong adapter | 30 skills | Unsupported |
| Retrieval of a similar document | 26.9 | 35.2 | — | 15.3 | 66 |
| Keyword writer (weights only) | 16.1 | 23.5 | 3.3 | 14.6 | — |
| Instruction writer (weights only) | 22.3 | 33.3 | 1.6 | 20.4 | 54 |
| Soft-prompt writer, 4B (weights only) | 22.7 | 33.2 | 22.5 | 18.2 | 55 |
| Soft-prompt writer, 8B (weights only) | 20.9 | 31.3 | 23.1 | 18.2 | 55 |
| EW-T † (adapter and text) | 43.7 | 61.5 | 1.3 | 39.9 | 39 |
| Reader | Loss, correct | Loss, wrong | Gap | Bits per 1k tokens |
|---|---|---|---|---|
| R1 soft prompt, complete adapter | 1.220 | 1.221 | 0.001 | |
| R2 keywords | 1.292 | 1.375 | 0.083 | |
| R2 keywords and instruction lines | 1.271 | 1.398 | 0.127 | |
| R3 Executing Writer, EW-T | 0.289 | 3.934 | 3.645 | |
| R3 Executing Writer, EW-A | 0.323 | 3.936 | 3.613 |
| Coverage of original | Precision | Whole doc. | |||||
| Method | Rank | Prose | Struct. | Ident. | Prose | Struct. | Same |
| Recitation adapters | |||||||
| EW-T | 16 | 84.1 | 85.1 | 85.1 | 85.8 | 65.5 | 83 |
| EW-T | 64 | 93.3 | 94.0 | 94.1 | 94.9 | 72.0 | 90 |
| EW-A | 16 | 79.7 | 80.1 | 78.0 | 84.5 | 63.2 | 73 |
| EW-A | 64 | 90.2 | 90.2 | 91.2 | 94.2 | 70.0 | 86 |
| Condition | Rank | All tasks | Hardest tasks | Gain recovered (%) |
| No skill | – | 55.5 | 23.2 | 0 |
| Original skill in prompt | – | 77.0 | 65.9 | 100 |
| Recovered document in prompt, from recitation adapters | ||||
| EW-T | 16 | 73.6 | 59.2 | 84 |
| EW-T | 64 | 75.1 | 60.8 | 91 |
| EW-A | 16 | 71.5 | 56.0 | 74 |
| Skill in prompt | Intrusion det. | 3D parser | Finance QA | PDDL plan | All runs | Tests (%) |
|---|---|---|---|---|---|---|
| None | 0/8 | 0/8 | 0/8 | 3/8 | 3/32 | 59.7 |
| Original | 5/8 | 8/8 | 2/8 | 2/8 | 17/32 | 85.0 |
| EW-T reconstruction | 5/8 | 8/8 | 1/8 | 1/8 | 15/32 | 82.5 |
| EW-A reconstruction | 0/8 | 8/8 | 1/8 | 4/8 | 13/32 | 83.1 |
| Arm | Sessions | Success | Range | ||
|---|---|---|---|---|---|
| References | |||||
| LoRA | – | 0.813 | 1.00 | ||
| Base model (one session / loop protocol) a | – | 0.254 / 0.261 | 0.00 | ||
| Three in-context expert games a | – | 0.709 / 0.679 | 0.81 / 0.76 | ||
| One session, Haiku writer, no reward | |||||
| Agent reads the adapter | 6 | 0.581 | 0.42–0.72 | 0.58 | |
| Task (control) | What the adapter learned | Real | Control | Difference [ CI] |
|---|---|---|---|---|
| Docker run compose (wrong) | arbitrary output conventions | 0.212 | 0.000 | [ , ] ✓ |
| S4 ticket triage (wrong) | arbitrary rule | 0.873 | 0.645 | [ , ] ✓ |
| S2 fee calculation (wrong) | arithmetic rule | 0.467 | 0.394 | [ , ] |
| S1 house format (wrong) | learnable from inputs | 0.908 | 0.893 | [ , ] |
| ALFWorld, one session (wrong) | expert household play | 0.581 | 0.521 | [ , ] |
| ALFWorld, loop (no adapter) | expert household play | 0.653 | 0.673 | [ , ] |
| Actor | Base | One-session skill | Loop, adapter | Loop, no adapter | Three games |
|---|---|---|---|---|---|
| Qwen2.5-7B (the LoRA’s base) | 0.254 | 0.612 | 0.676 | 0.658 | 0.709 |
| Llama-3.1-8B-Instruct | 0.052 | 0.545 | 0.454 | 0.405 | 0.269 |
| Qwen3.5-4B | 0.022 | 0.664 | 0.385 | 0.564 | 0.396 |
| Claude Haiku 5.5 (18 games) | 0.944 | 1.000 | n.r. | n.r. | n.r. |
| Pick (24) | Examine (18) | Clean (31) | Heat (23) | Cool (21) | Pick two (17) | |
|---|---|---|---|---|---|---|
| Base model | 0.50 | 0.06 | 0.16 | 0.39 | 0.19 | 0.24 |
| LoRA | 1.00 | 0.56 | 1.00 | 1.00 | 1.00 | 0.00 |
| Loop skill, adapter | 0.84 | 0.64 | 0.69 | 0.75 | 0.54 | 0.35 |
| Loop skill, no adapter | 0.83 | 0.61 | 0.70 | 0.73 | 0.65 | 0.43 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Coverage | Wrong adapter | Whole-doc. match | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Writer | |||||||||
| EW-T (no text) | 72.4 | 86.8 | 92.0 | 2.3 | 1.9 | 1.7 | 60.7 | 79.3 | 82.7 |
| EW-A | 79.8 | 87.4 | 92.0 | 1.9 | 1.9 | 2.0 | 52.2 | 69.3 | 76.1 |
| Writer | Rank | Prose cov. | Structure cov. | Identifier recall | Match prob. |
|---|---|---|---|---|---|
| EW-T (no text) | 16 | 84.1 | 85.1 | 85.1 | 74.3 |
| EW-T (no text) | 32 | 92.1 | 93.4 | 94.4 | 79.1 |
| EW-T (no text) | 64 | 93.3 | 94.0 | 94.1 | 81.2 |
| EW-A | 16 | 79.7 | 80.1 | 78.0 | 52.1 |
| EW-A | 32 | 86.8 | 86.9 | 87.7 | 64.8 |
| EW-A | 64 | 90.2 | 90.2 | 91.2 | 73.1 |
| Set | Method | M1, own | M1, wrong | Cov. own | Cov. wrong | |
|---|---|---|---|---|---|---|
| Held-out skills | Generic probes | 8 | ||||
| Vocabulary lens | 8 | |||||
| Amplified answers | 7 | 0.19 | ||||
| Plain request | 8 | |||||
| Best of | 7 | |||||
| Retrieval | 8 |
| Arm | S1 | S2 | S3 | S4 | R1 | R3 |
|---|---|---|---|---|---|---|
| Base model | 0.000 | 0.194 | 0.000 | 0.225 | 0.735 | 0.140 |
| LoRA ( examples) | 0.891 | 0.698 | 1.000 | 1.000 | 0.770 | 0.895 |
| True skill | 0.898 | 0.447 | 0.998 | 0.902 | – | – |
| Skill P (task text) | 0.909 | 0.404 | 0.000 | 0.860 | 0.730 | 0.190 |
| Skill D ( examples) | 0.955 | 0.472 | 1.000 | 0.993 | 0.665 | 0.245 |
| Skill A (adapter only) | 0.927 | 0.454 | 0.970 | 0.935 | – | 0.200 |
| Arm | Exact | Soft |
|---|---|---|
| Base model (Qwen2.5-7B-Instruct) | 0.000 | 0.186 |
| LoRA docker_compose_gen | 0.595 | 0.885 |
| Wrong adapter ( dockerfile_gen ) | 0.000 | 0.079 |
| Skill P (task text) | 0.000 | 0.340 |
| Skill D (training data), 2 sessions | 0.313 | 0.819 |
| Skill A (adapter only), 3 sessions | 0.263 | 0.771 |
| Added to the LoRA | Success | Change |
|---|---|---|
| Nothing | 0.813 | – |
| Short skill ( words), system prompt | 0.858 | |
| Short skill, after the task text | 0.843 | |
| Three in-context games | 0.836 | |
| Three games full skill, system prompt | 0.821 | |
| Reward-loop skills, system prompt (mean of ) | 0.744–0.767 | to |
| Arm | Sessions | Win | Rule A | Rule B | Skill states its rule |
| Base model | – | 0.261 | 0.000 | 0.000 | – |
| LoRA-A | – | 0.896 | 0.866 | 0.000 | – |
| LoRA-B | – | 0.851 | 0.000 | 0.851 | – |
| Agent reading LoRA-A | 6 | 0.604 | 0.005 | 0.000 | 5/6 (never rule B) |
| Agent reading LoRA-B | 6 | 0.565 | 0.000 | 0.053 | 5/6 (never rule A) |
| Agent reading LoRA-A with a divergence list | 6 | 0.636 | 0.001 | 0.000 | 6/6 |
LLM Agent Skill Learning
Large language model agent skill learning studies how agents acquire, represent, reuse, and refine procedures for multi-step tasks.
See all 424 papers