A Cost That Practice Accuracy Hides: Worked Solutions in Self-Generated Training Data and the Compositional Use of New Rules
Organizations: Independent Researcher
Abstract
A worked solution can make self-generated training targets shorter and cheaper, but accuracy on the practiced operation may hide how well a model can reuse newly learned rules. We study this choice in a 2B-parameter reasoning model learning two invented quantitative rules. The model generates targets from a rule sheet alone or from the sheet plus a program-written worked solution; targets are accepted for final-answer correctness, and the support is removed for fine-tuning and evaluation. Training questions request forward use in one segment; composition requires sequential reuse of the same rule across two segments. After six passes, using a program-checked numerical signature, sheet-only targets lead by 24.3 percentage points (95% item-bootstrap CI 14.6-34.0) on composition in a prospective test. On unseen argument values, observed forward accuracy is identical (87.5% each), but composition accuracy is 58.3% versus 23.6%, a 34.7-point gap (25.7-43.8). An independently sampled second target-bank pair repeats the direction (+14.6 points, 2.1-26.0). Ten passes on the shorter worked-solution targets, approximately matching the domain tokens of six sheet-only passes while changing other training quantities, narrow the gap to 16.3 points (6.3-26.7) in the first bank pair and 6.2 points (-6.3 to 18.8) in the second, where the pre-specified test does not hold. At six passes each, the worked-solution policy uses fewer capture tokens and less update time, yet the trained models use 2.6 times as many generated tokens per composition success under the numerical signature (2.1 times in the second bank pair). Choosing a capture context therefore requires testing how the new rules are reused, alongside the inference cost of those uses, rather than relying on forward accuracy alone.
Figures & tables
| Items | Measure | Gen 6 [3] | Inst 6 [3] | Inst 10 [2] | Gen 6 Inst 6 | Gen 6 Inst 10 |
|---|---|---|---|---|---|---|
| Composition (48) | signature | 59.0 | 34.7 | 42.7 | TB-1: [14.6, 34.0] | TB-2: [6.3, 26.7] |
| Composition (48) | official | 56.9 | 31.2 | 38.5 | [16.0, 35.4] | [8.0, 28.8] |
| Inversion (48) | signature | 66.7 | 50.0 | 54.2 | [4.9, 28.5] | [0.3, 25.3] |
| Forward (24) | signature | 86.1 | 83.3 | 85.4 | [-9.7, 15.3] | [-15.3, 17.4] |
| Items | Measure | Gen 6 | Inst 6 | Base model | Gen 6 Inst 6 |
|---|---|---|---|---|---|
| Forward (24) | signature | 87.5 | 87.5 | 0.0 | [-9.7, 9.7] |
| Forward (24) | official | 86.1 | – | 0.0 | – |
| Inversion (48) | signature | 62.5 | 47.9 | 2.1 | [2.8, 26.4] |
| Composition (48) | signature | 58.3 | 23.6 | 2.1 | OS-2: [25.7, 43.8] |
| Composition (48) | official | 54.9 | 20.1 | – | [25.7, 43.8] |
| Bank 1 | Bank 2 | Gen Inst | |||||
| Items | Gen [3] | Inst [3] | Gen [2] | Inst [2] | Bank 1 | Bank 2 | Pooled |
| Forward (24) | 86.1 | 83.3 | 87.5 | 79.2 | [-5.6, 17.4] | ||
| Inversion (48) | 66.7 | 50.0 | 60.4 | 33.3 | [13.9, 30.0] | ||
| Composition (48) | 59.0 | 34.7 | 45.8 | 31.2 | [12.3, 26.2] | ||
| Composition, official | 56.9 | 31.2 | 43.8 | 27.1 | [14.6, 27.6] | ||
| Matched dose: worked-solution policy trained for ten passes [2] | |||||||
| Test | Prediction | Prob. | Outcome |
|---|---|---|---|
| New arguments | forward gap at least pp (argument-bound) | 0.35 | no: [-9.7, 9.7] |
| forward gap within pp (configuration-bound) | 0.55 | yes (observed rates; not equivalence) | |
| composition gap minus forward gap positive | 0.60 | yes: | |
| T2: trained product at least pp above new product (value association) | 0.35 | no: | |
| Likelihood | larger for Inst than for Gen targets, each rule | 0.90 | yes; sign close to guaranteed by construction |
| arm difference larger for T2 than for T1 | 0.65 | yes (two rules; weak) |
| Gen T1 | Inst T1 | Gen T2 | Inst T2 | |
| Targets | 121 | 119 | 128 | 127 |
| Rule written symbolically (%) | 100 | 100 | 100 | 100 |
| Rule evaluated with the question’s numbers (%) | 91.7 | 95.8 | 84.4 | 85.8 |
| Worked solution named (%) | 0 | 80.7 | 0 | 96.1 |
| A value stated to match the solution (%) | 0 | 42.0 | 0 | 46.5 |
| Median length (tokens) | 2,746 | 2,268 | 3,339 | 1,501 |
| Bank pair 1 | Bank pair 2 | |||||
|---|---|---|---|---|---|---|
| Gen 6 | Inst 6 | Inst 10 | Gen 6 | Inst 6 | Inst 10 | |
| Capture: accepted / judged answers | 249/256 | 246/256 | (same) | 242/256 | 247/256 | (same) |
| Capture: generated tokens per accepted target | 3,568 | 2,057 | (same) | 3,533 | 2,023 | (same) |
| Training: supervised domain tokens (M) | 2.67 | 1.49 | 2.49 | 2.57 | 1.50 | 2.49 |
| Training: update time per seed (s) | 1,894 | 1,357 | 2,245 | 1,664 | 1,224 | 2,076 |
| Inference: median tokens per composition answer | 4,744 | 8,837 | 6,526 | 5,349 | 8,700 | 6,710 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| State | Passes | Forward A (48) | Inversion (24) | Composition (24) |
|---|---|---|---|---|
| Base model, no sheet [2 draws] | – | 0.0 | 0.0 | 0.0 |
| Gen [3] | 2 | 8.3 | 2.8 | 2.8 |
| Gen [2] | 4 | 63.5 | 43.8 | 29.2 |
| Gen [2] | 6 | 82.3 | 72.9 | 68.8 |
| Inst [2] | 2 | 4.2 | 0.0 | 0.0 |
| Inst [2] | 4 | 63.5 s | 22.9 s | 14.6 s |
| State | Domain tokens | Replay tokens | Target visits | Updates (per seed) | LR cycles |
|---|---|---|---|---|---|
| Gen 6 passes | 2,667,150 | 469,206 | 768 | 429 / 428 / 432 | 6 |
| Inst 6 passes | 1,494,891 | 469,206 | 762 | 284 / 292 / 281 | 6 |
| Inst 10 passes | 2,491,485 | 782,010 | 1,270 | 476 / 487 | 10 |
| Comparison | Seeds or contrast | Difference |
|---|---|---|
| Six passes, fresh items, bank pair 1 | seeds 1, 2, 3 | , , |
| Six passes, fresh items, bank pair 2 | seeds 1, 2 | , |
| Six passes, new arguments, bank pair 1 | seeds 1, 2, 3 | , , |
| Matched dose, fresh items, bank pair 1 | seeds 1, 2 | , |
| Matched dose, fresh items, bank pair 2 | seeds 1, 2 | , |
| Same seed, two banks: Gen 6, bank 1 minus bank 2 | seed 1 | [-14.6, 18.8] |
| Items (answers) | State | Correct | Familiar law | One current only | Partial | Other |
|---|---|---|---|---|---|---|
| Two-current (72/72/48) | Gen 6 | 38.9 | 40.3 | 2.8 | 0.0 | 18.1 |
| Inst 6 | 8.3 | 41.7 | 6.9 | 0.0 | 43.1 | |
| Inst 10 | 16.7 | 60.4 | 2.1 | 0.0 | 18.8 | |
| Rest-segment (72/72/48) | Gen 6 | 79.2 | 1.4 | – | 4.2 | 15.3 |
| Inst 6 | 61.1 | 2.8 | – | 22.2 | 13.9 | |
| Inst 10 | 68.8 | 2.1 | – | 18.8 | 10.4 |
| Test or prediction | Recorded | Probability | Outcome |
|---|---|---|---|
| TI-1, TI-2 (the capture context changes forward acquisition) | A2 | 0.35, 0.20 | not held |
| QT-1 (second model, forward dose) | A1, A2 | 0.60 | held |
| QT-2 (second model, unpracticed operations, 20 pp margin) | A1, A2 | 0.50 | not held ( ) |
| Kept six passes, forward signature at most 5% (first; second seed) | protocol; A3 | 0.80; 0.85 | held (0%) |
| TB-1 (fresh composition, same passes) | A3 | 0.60 | held |
| TB-2 (fresh composition, about the same domain tokens) | A3 | 0.40 | held |
| Model | Items | Answers | Agree (%) | Sig. T, off. fail | Sig. not T, off. success |
|---|---|---|---|---|---|
| MiniCPM5-2B | sealed A | 479 | 94.8 | 16 | 9 |
| MiniCPM5-2B | sealed B-inv | 239 | 92.5 | 5 | 13 |
| MiniCPM5-2B | sealed B-comp | 240 | 93.8 | 15 | 0 |
| MiniCPM5-2B | fresh B-comp | 383 | 96.9 | 12 | 0 |
| MiniCPM5-2B | new-argument A | 72 | 98.6 | 1 | 0 |
| MiniCPM5-2B | new-argument B-comp | 288 | 93.8 | 14 | 4 |
Self-Training for Language Models
Self-training for language models uses their own generations, feedback, and interaction traces as supervision to improve capabilities with less labeled data.
See all 95 papers