cs.LGNov 5, 2026
SaveTest-Time Self-Play: Scaling without a Verifier
Organizations: ETH Zurich & Tufa Labs · ETH Zurich
Abstract
Test-time scaling methods spend additional compute at deployment to improve a large language model's performance on a hard target problem. Most methods rely on a verifier that scores candidate solutions for correctness, using it to select among or refine the model's outputs. In many deployment regimes, however, a strong verifier on the target problem is unavailable. We introduce Test-Time Self-Play (TTSP), an approach to test-time scaling using asymmetric self-play without reliance on a verifier. On a difficulty-calibrated subset of LiveCodeBench v6, TTSP is the best-performing method at every verifier strength we evaluate, and the gap to the strongest baseline widens as the verifier weakens; in the no-verifier limit, TTSP yields a 35% absolute improvement in mean@8. The gains hold across a range of verifier strengths, except when the base policy has near-zero coverage of the correct answer.
Figures & tables
Figure 1: TTSP outperforms all baseline test-time scaling methods in the weak verifier regime. TTSP and ACE are evaluated at converged checkpoints; FMV uses public and private input test cases.
Figure 2: Overview of Test-Time Self-Play. Given a target problem and previously generated synthetic problems, the proposer policy generates target-conditioned synthetic problems consisting of a problem statement, tests, and a reference solution. The solver policy is rewarded for correctly solving the synthetic problems, while the proposer is rewarded for generating synthetic problems near the solver’s current learning frontier (both via GRPO; see the red arrows). The synthetic problem history provides context for the proposer and fills a replay buffer . After a sufficient number of iterations, the test-time trained solver can then attempt the target problem.
| Target Problem | Concept |
|---|---|
| For each , maximize the GCD of a size- subset containing . | Largest feasible divisor of |
| Synthetic Problems | |
| Find the largest integer that divides at least elements of the array. | Feasibility predicate: |
| For every divisor , compute . | Multiples-sieve divisor counts |
| For each , output the largest divisor with . | Per-element feasible divisor search |
Table 1: Example synthetic curriculum. The synthetic problems are generated by the proposer, teaching the solver concepts which are useful to recover the final strategy needed to solve the target problem.
| Method | Verifier Access |
|---|---|
| Rejection sampling | Inputs + Outputs |
| Context optimization | Inputs + Outputs |
| Majority voting | Inputs Only |
| TTSP (ours) | None |
Table 2: Required verifier access per method. Each method requires test case inputs, test case outputs, neither, or both, from which a learning signal can be derived.
Figure 3: Baselines performance on HardLCBv6 as compute scales. Each line reports the mean@8 with the indicated evaluation-time verifier applied to the validation problem submissions. Left and middle: Qwen 3 8B and FMV scale on the number of sampled rollouts k. Right: ACE scales on the number of generation–reflection–curation steps.
Figure 6: Unfiltered mean@8 of TTSP and FMV binned by rank of the correct functional cluster in the base policy. Bars group target problems by the rank of the correct functional cluster among base-policy rollouts.
Figure 7: Difference in mean@8 between TTSP and Qwen3-8B. The gap is large and positive on MidHardLCBv6 as the verifier weakens. On VeryHardLCBv6, the gap is near zero: only marginally positive under the weakest verifiers.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
| Target Problem | mean@512 | Correct Solution Rank |
| 1035 | 0.686 | 1 |
| 1050 | 0.645 | 1 |
| 976 | 0.633 | 1 |
| 920 | 0.596 | 1 |
| 1047 | 0.555 | 1 |
| 962 | 0.523 | 1 |
Table 3: MidHardLCBv6 target problems. LiveCodeBench v6 target problem identifiers and base model mean@512 of Qwen 3 8B for each selected target problem in the MidHardLCBv6 subset ( ).
| Target Problem | mean@512 | Correct Solution Rank |
| 941 | 0.057 | 3 |
| 1017 | 0.045 | 2 |
| 946 | 0.043 | 3 |
| 919 | 0.018 | 3 |
| 1027 | 0.012 | 2 |
| 947 | 0.010 | 11 |
Table 4: VeryHardLCBv6 target problems. LiveCodeBench v6 target problem identifiers and base model mean@512 of Qwen 3 8B for each selected target problem in the VeryHardLCBv6 subset ( ).
| Verifier | Mean tests used | |||
|---|---|---|---|---|
| Generated | Public | Private | Total | |
| 0.00 | 0.00 | 0.00 | 0.00 | |
| 0.36 | 0.07 | 0.86 | 1.29 | |
| 0.39 | 0.07 | 1.00 | 1.46 | |
| 0.57 | 0.18 | 0.79 | 1.54 | |
| 0.00 | 2.86 | 0.00 | 2.86 | |
Table 5: Tests used by each evaluation-time verifier on HardLCBv6. Values are averages over the 28 target problems and are grouped by test source.
| Setting | Value |
|---|---|
| Model | |
| Backbone | Qwen3-8B |
| Trainable parameters | Separate proposer and solver LoRA adapters |
| LoRA rank / | |
| LoRA target modules | All linear layers |
| Optimization | |
Table 6: Core TTSP training hyperparameters. All values are fixed across target problems. The model is adapted per target through separate proposer and solver LoRA adapters, while the Qwen 3 8B backbone remains frozen.
| Setting | Value |
|---|---|
| Generation | |
| Inference engine | vLLM |
| Sampling temperature | |
| Max prompt / response length | tokens |
| Proposer contexts per step | |
| Proposer rollouts per context | |
Table 7: Curriculum, rollout, and replay hyperparameters. Fresh synthetic problems are generated by the proposer; replayed synthetic problems are sampled from history and re-evaluated under the current solver policy when replayed.
Figure 8: FMV under partial test-input access on HardLCBv6. The final evaluation-time filter is removed, and all submitted candidates are scored directly against the public+private evaluator . The x-axis denotes the test inputs available to FMV for functional clustering, not a post-hoc verifier. TTSP and Qwen 3 8B are shown as unfiltered baselines independent of FMV’s input-access condition.
Figure 9: Synthetic problem rejection diagnostics. Pre-training proposer generations rejected before entering solver training, averaged over target problems and grouped by failure type. Rejection counts decrease over training; invalid generated output is the largest rejection category.
Figure 10: Average and top- proposer reward during TTSP training. The Proposer curve is the mean proposer reward over generated synthetic problems, while Proposer (top-4) averages the four highest proposer rewards at each step. The gap measures within-batch reward concentration. Solver reward and unfiltered target validation are shown for reference. Shaded bands denote SEM across target problems.
Figure 11: Frozen-base and adapted-solver performance on generated synthetic problems. Example trajectory for one target problem. Frozen Qwen 3 8B is the initial model evaluated on the synthetic problems generated at each step; TTSP solver is the adapted solver evaluated on the same synthetic problems; Target validation is the adapted solver’s unfiltered mean@8 on the original target problem under .
Figure 12: Policy entropy during TTSP training. Mean proposer-policy and solver-policy entropy across GRPO steps on HardLCBv6, averaged over target problems. The proposer entropy increases over training, while solver entropy remains stable. Shaded regions show variation across target problems. Shaded bands denote SEM across target problems.
Figure 13: Curriculum novelty and target relatedness. Left: cumulative normalized Vendi gain computed from embeddings of extracted Python reference programs. Right: text-level cosine similarity between generated synthetic problem statements and the original target problem statement. Shaded bands denote SEM across target problems.
Figure 14: Per-target TTSP training curves on MidHardLCBv6. Mean@8 over GRPO steps for each individual target problem, with one panel per target and one line per evaluation-time verifier strength. Many targets show the rapid rise within – steps and stable plateau seen in the aggregate curve of Figure 5 .
Figure 15: Functional output clusters for MidHardLCBv6. Qwen 3 8B functional output cluster distribution for each target problem in MidHardLCBv6. Each panel shows, for one target, the fraction of base rollouts whose execution traces fall into each functional cluster, with the correct cluster highlighted in green.
Figure 16: Per-target TTSP training curves on VeryHardLCBv6. Mean@8 over GRPO steps for each individual target problem, with one panel per target and one line per evaluation-time verifier strength. The flat or noisy curves illustrate the discovery-side failure mode discussed in Section 5.4 when base coverage is below .
Figure 17: Functional output clusters for VeryHardLCBv6. Qwen 3 8B functional output cluster distribution for each target problem in VeryHardLCBv6. Each panel shows, for one target, the fraction of base rollouts whose execution traces fall into each functional cluster, with the correct cluster highlighted in green. The correct cluster typically carries negligible mass, matching the discovery-regime characterization in Figure 7 .