Value-Conditioned Proposal Acquisition for Test-Time Reasoning
Organizations: Shanghai Jiao Tong University Shanghai, China
Abstract
Sampling and selection spend test-time compute inside a candidate pool, but they cannot recover a correct answer that never enters it. We introduce ValueFormation, a test-time policy that decides after each primary-model draw whether to stop, continue sampling, or acquire one proposal from a heterogeneous reasoning specialist. The three actions are compared on one cost-regularized value scale learned from vote-state features; the calibration is frozen across tasks and primary models, and acquired proposals are accepted only through a frozen posterior comparison. On held-out MATH-500, one Llama-fitted calibration improves over the strongest of eight identically evaluated baselines by more than fifteen points for each of three 8B primaries and transfers unchanged to Qwen3-14B. Removing acquisition lowers Llama MATH accuracy by 20.43 points, while same-trace and same-access replays attribute the gain to state-conditioned acquisition rather than specialist access alone. Where primary coverage is high, as on GSM8K, acquisition is rare and gains mainly come from cheaper allocation; on MBPP, a fixed code route attains the top accuracy at the lowest interaction cost. Varying the token price monotonically reduces calls without retraining.
Figures & tables
| GSM8K | MATH-500 | SVAMP | HumanEval † | |||||||||
| Policy | Llama | Qwen | Ministral | Llama | Qwen | Ministral | Llama | Qwen | Ministral | Llama | Qwen | Ministral |
| greedy | 86.01 0.26 | 87.35 0.42 | 89.22 0.10 | 49.91 0.41 | 56.27 2.19 | 54.39 1.33 | 83.91 0.89 | 87.40 1.68 | 88.18 0.67 | 41.67 0.00 | 53.70 8.02 | 46.30 1.60 |
| sample1 | 86.12 0.65 | 87.83 0.91 | 87.35 0.84 | 48.57 1.48 | 56.45 0.93 | 53.85 3.00 | 81.01 1.46 | 87.40 0.67 | 85.85 1.46 | 42.59 3.21 | 52.78 2.78 | 48.15 6.42 |
| SC-10 | 91.35 0.75 | 91.46 0.05 | 92.39 0.17 | 58.51 1.53 | 59.23 1.09 | 61.38 1.09 | 88.57 2.04 | 90.50 1.21 | 91.67 0.67 | 47.22 0.00 | 49.07 1.60 | 54.63 1.60 |
| ASC | 92.05 0.24 | 92.84 0.24 | 92.78 0.17 | 61.02 0.93 | 62.54 1.12 | 62.90 0.71 | 89.15 0.89 | 91.28 1.01 | 91.86 0.58 | 50.93 4.24 | 54.63 5.78 | 51.85 1.60 |
| ESC | 91.83 0.26 | 93.00 0.13 | 92.50 0.10 | 62.46 0.56 | 61.65 1.33 | 64.25 0.71 | 89.15 0.67 | 91.28 0.58 | 92.05 0.34 | 48.15 3.21 | 51.85 4.24 | 51.85 3.21 |
| Task | Accuracy | Calls | Tokens / R1 share | |
|---|---|---|---|---|
| GSM8K | 0 | 92.84 0.10 | 6.14 | 2,586 / 15.59% |
| 0.0025 | 92.67 0.21 | 4.60 | 2,021 / 15.00% | |
| 0.005 | 92.50 0.46 | 2.70 | 1,314 / 12.90% | |
| MATH-500 | 0 | 80.02 0.68 | 5.38 | 5,656 / 55.82% |
| 0.0025 | 80.29 1.53 | 4.69 | 5,226 / 53.58% | |
| 0.005 | 76.79 1.48 | 3.43 | 4,375 / 49.19% |
| GSM8K | MATH-500 | SVAMP | HumanEval † | |||||||||
| Policy | Llama | Qwen | Ministral | Llama | Qwen | Ministral | Llama | Qwen | Ministral | Llama | Qwen | Ministral |
| greedy | 1.00/364 | 1.00/1,740 | 1.00/399 | 1.00/767 | 1.00/2,094 | 1.00/641 | 1.00/331 | 1.00/1,531 | 1.00/323 | 1.00/283 | 1.00/1,755 | 1.00/252 |
| sample1 | 1.00/365 | 1.00/1,757 | 1.00/398 | 1.00/774 | 1.00/2,099 | 1.00/617 | 1.00/320 | 1.00/1,545 | 1.00/309 | 1.00/288 | 1.00/1,766 | 1.00/247 |
| SC-10 | 10.00/3,658 | 10.00/17,557 | 10.00/3,984 | 10.00/7,771 | 10.00/20,975 | 10.00/6,132 | 10.00/3,154 | 10.00/15,458 | 10.00/3,092 | 11.00/3,502 | 11.00/18,644 | 11.00/3,112 |
| ASC | 8.03/3,401 | 10.51/21,019 | 7.35/3,170 | 20.15/19,644 | 24.65/53,650 | 18.99/14,095 | 8.33/3,581 | 10.70/20,547 | 6.36/2,264 | 10.29/3,514 | 5.27/8,659 | 10.56/3,144 |
| ESC | 8.07/3,450 | 14.14/28,672 | 6.98/3,038 | 21.03/20,838 | 28.93/62,812 | 19.86/14,849 | 8.82/3,981 | 12.65/24,638 | 6.44/2,326 | 10.37/3,598 | 5.28/8,640 | 10.68/3,155 |
Inference-Time Compute Allocation
Inference-time compute allocation decides where, when, and how much computation models spend during prediction to improve quality under resource limits.
See all 90 papers