cs.LGNov 17, 2026

Learning Stratified Policies for Inference-Time Objectives

Authors: Mridul Mahajan, Xuezhou Zhang, Aldo Pacchiano

Organizations: Boston University · Broad Institute of MIT and Harvard

Abstract

Inference-time scaling improves the performance of large language models by sampling multiple responses and evaluating their collective utility. However, many existing approaches obtain these samples by repeatedly sampling from a single policy. In this work, we study if the sample budget at inference time can instead be allocated across a population of policies. We introduce stratified policies, where each of kk learned policies contributes one response to a kk-sample inference-time objective. We realize this policy population using separate LoRA adapters on a shared language model backbone and optimize it for pass@kk. We theoretically motivate the approach by analyzing the linear setting; a population of LoRAs can achieve zero best-of-kk approximation error in settings where every single LoRA has strictly positive prediction error. Empirically, we demonstrate that stratified policies consistently improve pass@kk over a single-policy baseline across multiple reasoning tasks. Overall, our results highlight how distributing samples across complementary policies can be a more effective use of sample budget than repeated i.i.d. sampling from a single policy at inference time.

Figures & tables

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Funnel of Thoughts: Efficient Test-Time Scaling via Early Voting and Rollout Pruning

    Aug 15, 2026Chanhee Park, Sungbin Han, Jeongho Yoon +2Efficient Language Model ReasoningInference-Time Scaling

  2. ∇\nabla-Reasoner: LLM Reasoning via Test-Time Gradient Descent in Latent Space

    Mar 5, 2026Peihao Wang, Ruisi Cai, Zhen Wang +4LLM ReasoningTest-Time Optimization