On-Policy

Follow this topic Ask about this topic

Recent momentum

-75%

3 papers in the last 28 days · 0.1% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-14

1 new paper

A weekly snapshot of new work published in On-Policy.

Period ending 2026-09-07

1 new paper

A weekly snapshot of new work published in On-Policy.

48 papers

Latest in On-Policy

Open your feed →

MAGMA-GEN: Validated Recovery Supervision from Ambiguous Failures via Counterfactual Re-Execution
Sep 17, 2026Loan Bernat, Matthieu Grard, Ariane Herbulot +1Long-Horizon Planning On-Policy
Co-Evolving Harnesses and Models: On-Policy Correction Helps Weaker Models Catch Up Where Imitation Fails
Sep 8, 2026Zhou Yu, Bin Bi, Shiva Kumar Pentyala +8Agent Harness Iterative Co-Training
Scaling Bimanual Household Manipulation from 1,500 hours of Demonstrations to On-Policy Corrections
Sep 3, 2026Jiafeng Xu, Qi Li, Yan Shen +7Bimanual Manipulation Manipulation
FlowBalance: Verifier-Grounded Self-Improvement from On-Policy Reasoning Experience
Sep 3, 2026Zixun Huang, Kishan Panaganti, Haitao Mi +1Reinforcement Learning With Verifiable Reward On-Policy
REOPD: Reliability-Adaptive Reward Extrapolation for On-Policy Distillation
Aug 12, 2026Yang Sun, Lichao Ma, Houyuan Qin +5On-Policy Distillation Reward Gradients
RP-OPSD: Reasoning-Pivot-Guided On-Policy Self-Distillation for Multilingual Reasoning Transfer
Aug 6, 2026Xinye Wang, Junxiao Liu, Shujian HuangMultilingual Reasoning Large Language Model Reasoning
On-Policy Self-Distillation without Any Supervision
Aug 6, 2026Yijiang Li, Bingyang Wang, Yijun Liang +3Unsupervised On-Policy Self-Distillation On-Policy Distillation
Continual Learning in Transition
Aug 6, 2026Zhiyan Hou, Dan Zhang, Tao Feng +11Continual Learning Methods Transitions
ReflectRL: Learning from Golden Negative Trajectories via Reflective-to-Direct Reasoning
Aug 4, 2026Jinhe Bi, Chennan Zhou, Zengjie Jin +10Reasoning Trajectory On-Policy
SERL-SQL: Selective Hindsight Distillation for Text-to-SQL Reinforcement Agentic Learning
Aug 1, 2026Tao Liu, Tao Feng, Xiangheng Li +9Text-To-Sql Sql
When Does On-Policy Interaction Help? Representational Tradeoffs in Value-Based Imitation Learning
Jul 31, 2026Luca Viano, Antoine Moulin, Audrey Huang +3Imitation Learning On-Policy
Self-Play Meets Skill Evolution: Self-Evolving Search Agents that Pose, Solve, and Remember
Jul 31, 2026Zenghuang Fu, Zhaoyang Li, Qiuyuan Ai +6Skill Evolution Self-Evolving Agents
On-Policy and Off-Policy Learning for Large Action Spaces
Jul 30, 2026Imad AoualiOff-Policy Evaluation Multi-Objective Policy Optimization
Probing the Origins of Reasoning Performance: Representational Quality for Mathematical Problem-Solving in RL vs. SFT Fine-Tuned Models
Jul 28, 2026Antyabha Rahman, Akshaj Gurugubelli, Omar Ankit +2Large Reasoning Models Supervised Fine-Tuning
Less Data, Better Alignment: Data-Centric Multi-Evaluator Agreement for Preference Optimization
Jul 27, 2026Zhengtao Yao, Runhao Li, Xupeng Chen +12On-Policy Minimal
X $^3$ -OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy AlignmentX^3-OPD: Distilling Reasoning into Large Audio-Language Models via On-Policy Alignment
Jul 23, 2026Dongjie Fu, Di Cao, Xize Cheng +6Large Audio Language Models Rp-Opsd
Better Starts, Better Ends: Bootstrapped Iterative Self-Reasoning Distillation for Compressed Reasoning
Jul 17, 2026Leichao Dong, Dongxu Zhang, Yiding Sun +4Unsupervised On-Policy Self-Distillation Large Reasoning Models
A Few Teacher Steps Go a Long Way: Cost-Efficient On-Policy Data Augmentation for Agent Post-Training
Jul 6, 2026Junze Ye, Jiayi Cheng, Miao Lu +3On-Policy Teacher
DRIFT: Refining Instruction Data via On-Policy Data Attribution
Jun 16, 2026Zefan Wang, Lincheng Li, Tianyu Yu +1Supervised Fine-Tuning Reinforcement Fine-Tuning
Reversal Q-Learning
Jun 16, 2026Aditya Oberai, Seohong Park, Sergey LevineFlow-Based Policy Offline Reinforcement Learning
ExpRL: Exploratory RL for LLM Mid-Training
Jun 15, 2026Violet Xiang, Amrith Setlur, Chase Blagden +2Large Language Model Reasoning Sparse Rewards
DiPOD: Diffusion Policy Optimization without Drifting Apart
Jun 11, 2026Haozhe Jiang, Haiwen Feng, Pieter Abbeel +3Diffusion Policies Diffusion Language Models
SENTINEL: Failure-Driven Reinforcement Learning for Training Tool-Using Language Model Agents
Jun 11, 2026Ziyi Wang, Yuxuan Lu, Yimeng Zhang +8Personalized Large Language Model Agents Deep Reinforcement Learning
World Models Meet Language Models: On the Complementarity of Concrete and Abstract Reasoning
Jun 2, 2026Yucheng Zhou, Wei Tao, Yiwen Guo +1World Models On-Policy
GPU-Parallel Multi-Task Reinforcement Learning with Demonstration Guided Policy Optimization
Jun 2, 2026Rui Zhang, Qiwei Wu, Zhengyu Zhang +5Multi-Task Reinforcement Learning Multi-Objective Policy Optimization
Decision-Focused On-Policy Learning for Contextual Linear Optimization with Partial Feedback
May 31, 2026Wyame Benslimane, Tinghan Ye, Pascal Van Hentenryck +1Linear Contextual Bandits Optimal Policies
The Flip Side of RLHF: On-Policy Feedback for Reward Model Self-Supervised Improvement
May 29, 2026Xiaobo Wang, Tong Wu, Min Tang +3Reinforcement Learning From Human Feedback On-Policy
PRO-CUA: Process-Reward Optimization for Computer Use Agents
May 27, 2026Yifei He, Rui Yang, Hao Bai +2Reliable Medical Process-Level Reward Model Evaluation Framework Agentic Reinforcement-Learning Framework
OISD: On-Policy Internal Self-Distillation of Language Models
May 27, 2026Xinyu Liu, Darryl Cherian Jacob, Yang Zhou +2Unsupervised On-Policy Self-Distillation On-Policy Distillation
VR-DAgger: Immersive VR for Dexterous Data Collection and Uncertainty-Guided On-Policy Correction
May 26, 2026René Zurbrügg, Tifanny Portela, Arjun Bhardwaj +3Teleoperation Data Collection
Efficient Agentic Reinforcement Learning with On-Policy Intrinsic Knowledge Boundary Enhancement
May 26, 2026Dingwei Chen, Zefang Zong, Zhipeng Ma +5Agentic Reinforcement-Learning Framework Llm-Based Agent
Not All Disagreement Is Learnable: Token Teachability in On-Policy Distillation
May 26, 2026Yuanyi Wang, Su Lu, Yanggan Gu +6On-Policy Distillation Token-Level Supervision
Extreme Region Policy Distillation
May 25, 2026Changyu Chen, Xiting Wang, Rui YanOn-Policy Off-Policy Evaluation
ECHO: Terminal Agents Learn World Models for Free
May 23, 2026Vaishnavi Shrivastava, Piero Kauffmann, Ahmed Awadallah +1Agentic Reinforcement-Learning Framework On-Policy
Human-in-the-Loop Contextual Bandits for Short-Term Rental Dynamic Pricing: Structural Equivalence of Historical Warm-Up and Approval-Gated Live Learning
May 22, 2026Oleg MiroshnichenkoLinear Contextual Bandits Prices
On-Policy Consistency Training Improves LLM Safety with Minimal Capability Degradation
May 20, 2026Andy Han, Kristina Fujimoto, Avidan Shah +5Safety Alignment Large Language Model Safety
CopT: Contrastive On-Policy Thinking with Continuous Spaces for General and Agentic Reasoning
May 19, 2026Dachuan Shi, Hanlin Zhu, Xiangchi Yuan +4Large Language Model Reasoning Thinking
Offline Reinforcement Learning with Universal Horizon Models
May 15, 2026Hojun Chung, Junseo Lee, Songhwai OhOffline Reinforcement Learning On-Policy
$f$ -Trajectory Balance: A Loss Family for Tuning GFlowNets, Generative Models, and LLMs with Off- and On-Policy Dataf-Trajectory Balance: A Loss Family for Tuning GFlowNets, Generative Models, and LLMs with Off- and On-Policy Data
May 14, 2026Jake Fawkes, Jason HartfordGenerative Flow Networks Time-Step-Wise Learnable Surrogate Gradient Function
STOP: Structured On-Policy Pruning of Long-Form Reasoning in Low-Data Regimes
May 13, 2026Chenjun Xu, Zhennan Zhou, Zhan Su +3Deepseek On-Policy
Missing Old Logits in Asynchronous Agentic RL: Semantic Mismatch and Repair Methods for Off-Policy Correction
May 12, 2026Zhong Guan, Yongjian Guo, Haoran Sun +5On-Policy Training-Inference Mismatch
Towards On-Policy Data Evolution for Visual-Native Multimodal Deep Search Agents
May 11, 2026Shijue Huang, Hangyu Guo, Guanting Dong +8Multimodal Search Agents Multimodal Learning
Preference-Based Self-Distillation: Beyond KL Matching via Reward Regularization
May 6, 2026Xin Yu, Liuchen Liao, Yiwen Zhang +3Unsupervised On-Policy Self-Distillation On-Policy Distillation
Hindsight-Anchored Policy Optimization: Learning Through Hindsight with Thompson Sampling-Inspired Adaptive Gating
Mar 11, 2026Yuning Wu, Ke Wang, Haoran Liu +3On-Policy Reinforcement Learning With Verifiable Reward
TADPO: Reinforcement Learning Goes Off-road
Mar 6, 2026Zhouchonghao Wu, Raymond Song, Vedant Mundheda +3Proximal Policy Optimization Terrain
HeaPA: Difficulty-Aware Heap Sampling and On-Policy Query Augmentation for LLM Reinforcement Learning
Jan 30, 2026Weiqi Wang, Xin Liu, Binxuan Huang +13On-Policy Prompt Engineering
Retaining by Doing: The Role of On-Policy Data in Mitigating Forgetting
Oct 21, 2025Howard Chen, Noam Razin, Karthik Narasimhan +1Catastrophic Forgetting Forgetting
Rethinking On-policy Optimization for Query Augmentation
Oct 20, 2025Zhichao Xu, Shengyao Zhuang, Xueguang Ma +6Retrieval-Augmented Generation Systems Information Retrieval Metrics

Sep 17, 2026 · cs.AIJ/K move · Enter open · S save

MAGMA-GEN: Validated Recovery Supervision from Ambiguous Failures via Counterfactual Re-Execution

Loan Bernat, Matthieu Grard, Ariane Herbulot, Florent Lamiraux

LAAS-GEPETTO · LAAS-RAP

Hierarchical robotic systems executing long-horizon manipulation tasks must make high-level semantic decisions that orchestrate stochastic low-level skills. In this setting, failed rollouts are ambiguous: a poor downstream state may reflect an invalid high-level decision, partial observation, or a valid decision whose physical execution failed. Traditional supervised learning lacks data for such recovery states, while reinforcement learning struggles with sparse rewards and non-local credit assignment. We propose MAGMA-GEN, an on-policy data-generation pipeline that converts ambiguous failed rollouts into validated recovery supervision. MAGMA-GEN first uses a privileged coach to hypothesize an early decision-level error and propose localized correction or recovery actions. Because this diagnosis is fallible, candidates are retained only if re-execution from the same state under matched conditions improves downstream progress. This produces supervised examples from the agent's own failure distribution without per-step human demonstrations. Evaluated on interactive long-horizon manipulation tasks, MAGMA-GEN improves task success and recovery capabilities, against distillation and trajectory-repair baselines under evolving task constraints in both simulation and real-robot execution.

Long-Horizon Planning On-Policy Counterfactual Reasoning

Open paper Ask about it PDF