RL for Language Model Reasoning

RL: Reinforcement Learning

This field studies how reward feedback can teach language models to solve multi-step problems, use tools, and decide how much reasoning a task needs. A common approach is reinforcement learning from verifiable rewards (RLVR), often using group-relative policy optimization, where sampled solutions are scored for correctness; other methods add step-level rewards, process verifiers, search, or teacher guidance to address sparse feedback. Work also explores curricula, adaptive rollout budgets, self-distillation, and controls for reasoning length and diversity. Key open tensions include balancing exploration with stable learning, ensuring rewards and verifiers do not encourage shortcuts, and improving generalization without excessive computation or overthinking.

Latest papers

1,115 papers129 papers in the last four weeks, up 4% on the four weeks before. 0.8% of all new papers.

Kongzi: A Historical Large Language Model with Fact Enhancement
Apr 13, 2025Jiashu Yang, Ningning Wang, Yian Zhao +5Language Modeling RL for Language Model Reasoning
Dalian University of Technology · Peking University · Nanyang Technological University
Efficient Reinforcement Finetuning via Adaptive Curriculum Learning
Apr 7, 2025Taiwei Shi, Yiyang Wu, Linxin Song +2RL for Language Model Reasoning Mathematical Reasoning
University of Southern California · Carnegie Mellon University · Mohamed Bin Zayed University of Artificial Intelligence
LearNAT: Learning NL2SQL with AST-guided Task Decomposition for Large Language Models
Apr 3, 2025Weibin Liao, Xin Gao, Tianyu Jia +6LLM Planning RL for Language Model Reasoning
School of Computer Science, Peking University · 2Key Laboratory of High Confidence Software Technologies, Ministry of Education · School of Computer Sciences, Beijing University of Posts and Telecommunications +5
RL-STaR: Theoretical Analysis of Reinforcement Learning Frameworks for Self-Taught Reasoner
Oct 31, 2024Fu-Chieh Chang, Yu-Ting Lee, Hui-Ying Shih +2Reinforcement Learning RL for Language Model Reasoning
MediaTek Research, Taipei, Taiwan · Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan · Department of Mathematical Sciences, National Chengchi University, Taipei, Taiwan +2
Amortizing intractable inference in large language models
Oct 6, 2023Edward J. Hu, Moksh Jain, Eric Elmoznino +4Amortized Inference LLM Fine-Tuning
Mila – Quebec AI Institute, Universit´e de Montr´eal · University of Oxford

Apr 13, 2025J/K move · Enter open · S save

Kongzi: A Historical Large Language Model with Fact Enhancement

Jiashu Yang, Ningning Wang, Yian Zhao, Chaoran Feng+4

Dalian University of Technology · Peking University · Nanyang Technological University

Language Modeling RL for Language Model Reasoning Question Answering

Open paper AskPDF arXiv

Large Language Models (LLMs) can exhibit factual inaccuracies during long-chain reasoning or under misleading prompts, limiting their reliability in historical analysis. We propose Kongzi, a language model for historical understanding and reasoning that combines continued pre-training on historical corpora, supervised fine-tuning, and fact-enhanced reinforcement learning. Its factuality reward compares entities and relations extracted from generated responses with historical references through knowledge-graph similarity. We evaluate 0.5B and 7B models on historical question answering and counterfactual reasoning, using reported ratings from three LLM judges, and on CMMLU-Chinese-History. Kongzi-7B receives the highest historical-accuracy ratings among the compared models under all three judges. Reinforcement learning improves the mean answer scores of both model sizes and their CMMLU-Chinese-History accuracy, although Kongzi-7B remains below the strongest reported baselines on that benchmark. These descriptive results support further study of factuality rewards for historical language models.

Latest papers

Kongzi: A Historical Large Language Model with Fact Enhancement

Efficient Reinforcement Finetuning via Adaptive Curriculum Learning

LearNAT: Learning NL2SQL with AST-guided Task Decomposition for Large Language Models

RL-STaR: Theoretical Analysis of Reinforcement Learning Frameworks for Self-Taught Reasoner

Amortizing intractable inference in large language models