Rollout

Follow this topic Ask about this topic

Momentum

14 papers in the last four weeks, up 56% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 66

All topics

Iterative Policy Refinement through Semantic Rollout Analysis
Oct 1, 2026Feiyu Gavin Zhu, Qi Xu, Zhifei Deng +4Imitation Rollout
Carnegie Mellon University · Meta
SERA: Scale-Equalized Rollout Allocation for Maximum Likelihood Reinforcement Learning
Sep 29, 2026Zihao Chen, Fanxiang Xiong, Hongran Ren +6Rollout
Harbin Institute of Technology, Shenzhen · The Chinese University of Hong Kong, Shenzhen · Nanyang Technological University
ROSS: Relearning from Self-Generated Rollouts through Selective Supervision
Sep 28, 2026Zhiwei Zhang, Huayu Deng, Fei Zhao +4Autoregressive Rollout Unsupervised On-Policy Self-Distillation
AllSpark Team
Not All Rollouts Are Worth Learning: On Trajectory Valuation for Post-Training Reinforcement Learning
Sep 28, 2026Xuesong Jia, Ziao Yang, Zhanhe Huang +1Reinforcement Learning Post-Training Trajectory Generation
Department of Computer Science, Brandeis University
One Rollout Is All You Get: Fully Test-Time Adaptation for GUI Agents
Sep 28, 2026Ziqiang Wang, Li Gu, Zhixiang Chi +6Graphical User Interface Agents Test-Time Adaptation
Concordia University · Mila – Qu´ebec AI Institute · University of Toronto +1
TGRL: Temperature-Grouped Reinforcement Learning for Efficient Exploration in LLMs
Sep 27, 2026Zihan Lin, Xiaohan Wang, Jie Cao +4Large Language Model Reinforcement Learning Group-Based Reinforcement Learning
University of Chinese Academy of Sciences · Meituan · MAIS&NLPR, Institute of Automation, Chinese Academy of Sciences
Marginally Correct Tool Caches Can Reverse Group-Normalized Policy Updates
Sep 22, 2026Shivam GuptaCache Rollout
Independent Researcher
Rollout Efficiency in Reinforcement Learning for Reasoning Large Language Models: A Taxonomy and Future Directions
Sep 21, 2026Niloofar Gholipour, Marcos Assuncao, Gursimran Singh +8Large Language Model Reinforcement Learning Rollout
École de technologie supérieure, Univ. of Québec, Canada · Huawei Technologies, Canada · The Univ. of Melbourne, Australia +1
WeightBridge: An Efficient Weight Transfer Library for Reinforcement Learning
Sep 21, 2026Xuanlin Jiang, Samuel Hsia, Michael Kuchnik +3Offline Reinforcement Learning Rollout
FAIR at Meta · Harvard University · Work carried out at Meta
Luck Is Not Skill: When Do Paired Rollouts Help Group-Relative RL of LLM Agents?
Sep 21, 2026Nazmus SakibGroup-Based Reinforcement Learning Rollout
BRAC University, Dhaka, Bangladesh
From Rollout to Reset: A Graph-Based Harness for Autonomous Long-Horizon Manipulation Evaluation
Sep 16, 2026Jing Jiang, Yue Yang, Xinkai Jiang +3Robot State Robot Systems
Karlsruhe Institute of Technology, Germany · University of North Carolina at Chapel Hill, USA
IMPLY: Physically Anchored Consistency for World-Model Rollouts
Sep 14, 2026Aman Mehta, Riya BaviskarSelf-Consistency World Models
Independent Researcher
Expert-Space Exploration in MoE Reinforcement Learning
Sep 14, 2026Hongyi He, Zhenghao Lin, Xiao Liu +3Mixture-Of-Experts Large Language Model Reinforcement Learning
Tsinghua University · Microsoft Research
Mask Forcing: Improving Autoregressive Video Diffusion Distillation via Dual-Noise Masking Rollout
Sep 8, 2026Zhuoran Zhao, Shengju Qian, Tongtong Liang +7Autoregressive Video Diffusion Models Video Diffusion Models
1HKUST(GZ) · 2HKUST · 3LIGHTSPEED +3
LeanGRPO: Eliminating Redundant Recomputation in Diffusion RL
Sep 3, 2026Sijie Wang, Zhiqiang Tan, Xinrui Yang +1Diffusion-Based Reinforcement Learning Methods Reward Gradients
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen
R2S-Eval: Robot Evaluation with Real-to-Sim Calibration via Vision-Language Models
Sep 3, 2026Yidi Wang, Feixiang Ruan, Ruoqu Chen +4Evaluation Benchmarks Sim-To-Real Gap
Sharpa · Nanjing University · Tongji University +1
From Rollouts to Recipes: Self-Contained Post-Training for LLMs
Sep 1, 2026Yifei Li, Lingling Zhang, Muye Huang +3Post-Training Reinforcement Learning Post-Training
School of Computer Science and Technology, Xi’an Jiaotong University, China · Shaanxi Province Key Laboratory of Big Data Knowledge Engineering · Zhongguancun Academy, Beijing, China +1
Group Adaptive Clipping Policy Optimization
Aug 31, 2026Sheng Jia, Xiao Wang, Shiva Prasad Kasiviswanathan +1Gradient Clipping Reinforcement Learning With Verifiable Reward
University of Toronto · Amazon
GCPO: Diagnosing and Constraining Subspace Geometry in Rollout RL for LLMs
Aug 12, 2026Kai Yang, Jingwei Xu, Wanyu Wang +4Autoregressive Rollout Rollout
School of Computer Science, Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · NovaCore +3
Keep the Future, Drop the Rollout: RIFT for World Action Models
Aug 12, 2026Chushan Zhang, Jinguang Tong, Xuesong Li +2World Models Autoregressive Rollout
Australian National University
PAIR: Pairwise-Aware Inclusion Reweighting for Adaptive Rollout Allocation in RLVR
Aug 11, 2026Pixel Nomand, Elena Voss, Marcus Hale +1Reinforcement Learning With Verifiable Reward Verifiable Rewards
University of Wisconsin–Madison · University of Washington
Scheduling Mixed RL Rollouts Beyond Prefix Locality
Aug 11, 2026Zetao Hong, Song Yuan, Yuanhao Ding +4Parallel Rollouts Autoregressive Rollout
State Key Laboratory of Novel Software Technology, Nanjing University · StepFun
Twin Rollouts: Noise-Coupled Counterfactual Branching in Interactive Video World Models
Aug 10, 2026Yu Ma, Hongli Shi, Xinran XuCounterfactual Generation Video World Models
OnEvoMemory: Evolving Memory through Online Robot Rollouts for Pretrained Robot Policies
Aug 9, 2026Zhongxi Chen, Shenqi ZongRobot Policies Long-Horizon Manipulation
Shanghai Jiao Tong University · Tsinghua University
Predicting Task Difficulty Without Rollouts
Aug 6, 2026Stefan Krsteski, Charlotte MeyerAgentic Benchmarks Evaluation Metrics
Andromede AI
Optimizing What Policies Learn From: Recoverability-aware Rollout Intervention Learning
Aug 5, 2026Zheyuan Zhang, Manqing Mao, Hong Wang +8Group-Based Reinforcement Learning Rollout
University of Notre Dame · Amazon, Inc
When Shared Rollouts Fail in Defensive Driving Evaluation: A NAVSIM Score Basis Audit
Aug 5, 2026Ziang Wei, Minjun Yu, Zheyuan Lai +2Rollout Trajectory-Level Credit
Beyond On-Policy Exploration: Integrating External Policy Rollouts for Reinforcement Learning in Diffusion Language Models
Aug 3, 2026Wonseok Lee, Jimyeong Kim, Jungmin Ko +1Large Language Model Reinforcement Learning Autoregressive Rollout
Interdisciplinary Program in Artificial Intelligence, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, South Korea · Artificial Intelligence Institute, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, South Korea · Department of Intelligence and Information, Seoul National University, 1 Gwanak-ro, Gwanak-gu, Seoul, 08826, South Korea
Temporal Concentration from Rollout Errors: Implicit Preference Optimization for Text-to-Video Diffusion
Jul 30, 2026Henglin Liu, Fangyuan Kong, Jing Wang +7Text-To-Video Diffusion Models Video Diffusion Models
Tsinghua University · Kling Team, Kuaishou Technology · Sun Yat-sen University
How Fast Can Reward Models Score? A Systems Study of C++ and PyTorch Inference Runtimes for RLHF
Jul 22, 2026Venkata Naga Sai Vishnu Rohit Pulipaka, Anish Katta, Deva Rohit Reddy PeddireddyDestination-Only Scoring Fast Inference
Stale but Stable: Staleness-Adaptive Trust Regions for Stabilizing Asynchronous Reinforcement Learning
Jul 21, 2026Junyao Yang, Yucheng Shi, Zongxia Li +6Trust Region Gradient Staleness
1Tencent Hy LLM Frontier · 2National University of Singapore · University of Maryland, College Park +3
RRPO: Reference-Relative Policy Optimization with Stratified Conditional Rollouts
Jul 20, 2026Yuxin Xiong, Xunyi Jiang, Rohan Surana +8Reinforcement Learning With Verifiable Reward Rollout
University of California San Diego · 2Adobe Research
WAR: Workload-Aware Rollouts for Synchronous Agentic Reinforcement Learning
Jul 19, 2026Ryan Xu, Atlas Zhao, David Bao +1Agentic Reinforcement Learning Rollout
QUADS: Stabilizing NVFP4 Reinforcement Learning for MoE via QUantization-error Alignment across Dual Sides
Jul 17, 2026Zhengyang Zhuge, Hao Yu, Xin Wang +4Fp8 Autoregressive Rollout
Qwen Team, Alibaba Inc.
Process Reward Informed Tree Rollout for Effective Multi-Turn RL
Jul 17, 2026Xintong Li, Sha Li, Yuwei Zhang +8Autoregressive Rollout Rollout
1UC San Diego · 2Amazon · 3MIT Alumni
Branching Policy Optimization: Sandbox-Native Language Agent Reinforcement Learning
Jul 15, 2026Bowei He, Yankai Chen, Xiaokun Zhang +1Frictive Policy Optimization Offline Reinforcement Learning
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · McGill Univeristy, Montreal, Canada · City University of Hong Kong, Hong Kong SAR.
Prompt-Driven Exploration
Jul 9, 2026Sunshine Jiang, John Marangola, David Zhang +6Large Language Model Reinforcement Learning Rollout
Massachusetts Institute of Technology · Improbable AI Lab · MIT-IBM Computing Research Lab
HELP: Human-Efficient Large-Scale Robot Post-Training with Rollout Segmentation
Jul 8, 2026Shaopeng Zhai, Qi Zhang, Tianyi Zhang +5Humanoid Robotics Rollout
Continuous Learning Team, Shanghai AI Laboratory
The Rollout Infrastructure Tax in Coding-Agent Reinforcement Learning
Jul 1, 2026Daniel Thi Graviet, Lovre Pesut, Ivan Dagelic +2Coding Agents Rollout
Daytona United States
Understanding Rollout Error in Graph World Models
Jun 26, 2026Xinyuan Song, Zekun CaiWorld Models Prediction Error
Emory University, Atlanta, GA, USA · The University of Tokyo, Tokyo, Japan · LocationMind, Tokyo, Japan
RolloutPipe: Overlapping Pipelined Rollout and Training in Disaggregated On-Policy LLM Reinforcement Learning
Jun 25, 2026Rongjian Chen, Jianmin Hu, Kejiang Ye +1Large Language Model Reinforcement Learning Rollout
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China · University of Chinese Academy of Sciences, Beijing, China · Southern University of Science and Technology, Shenzhen, China
Prefix-Guided On-Policy Distillation: Mining Golden Trajectories from Rollouts
Jun 20, 2026Qingfei Zhao, Huan Song, Shuyu Tian +2Unsupervised On-Policy Self-Distillation Rollout
TeleAI · Shanghai Jiao Tong University
EfficientRollout: System-Aware Self-Speculative Decoding for RL Rollouts
Jun 17, 2026Minseo Kim, Minjae Lee, Seunghyuk Oh +7Autoregressive Rollout Parallel Rollouts
FuriosaAI · University of California, Berkeley
TRACE: A Unified Rollout Budget Allocation Framework for Efficient Agentic Reinforcement Learning
Jun 9, 2026Heming Zou, Qi Wang, Yun Qu +9Autoregressive Rollout Agentic Reinforcement Learning
1Tsinghua University · 2LLM Department, Tencent
SALT: When More Rollouts Don't Help in Group-Based Policy Optimization and How to Make Them Matter
Jun 4, 2026Powei Chang, Jinpeng Zhang, Chaoqun Sun +6Reinforcement Learning With Verifiable Reward Rollout
Bilibili Inc. · Fudan University · Zhejiang University
DRIFT: Decoupled Rollouts and Importance-Weighted Fine-Tuning for Efficient Multi-Turn Optimization
May 29, 2026Jian Mu, Tianyi Lin, Chengwei Qin +2Reinforcement Fine-Tuning Offline Reinforcement Learning
The Hong Kong University of Science and Technology (Guangzhou) · The Chinese University of Hong Kong, Shenzhen.
EchoRL: Reinforcement Learning via Rollout Echoing
May 29, 2026Jinhe Bi, Aniri, Minglai Yang +9Reinforcement Learning With Verifiable Reward Large Language Model Training
Huawei Heisenberg Research Center · LMU Munich · Munich Center for Machine Learning +5
Smaller Models are Natural Explorers for Policy-Level Diversity in GRPO
May 29, 2026Yiming Ren, Yiran Xu, Zicheng Lin +8Frictive Policy Optimization Rollout
Tsinghua University · Shanghai AI Laboratory · The Chinese University of Hong Kong +1
Diversifying RLVR Rollouts via First-Token Exploration
May 27, 2026Soeun Kim, Albert NoReinforcement Learning With Verifiable Reward Verifiable Rewards
Department of Artificial Intelligence, Yonsei University
Less is More: Early Stopping Rollout for On-Policy Distillation
May 26, 2026Zhou Ziheng, Jiaqi Li, Huacong Tang +2Efficient On-Policy Distillation Rollout
University of California, Los Angeles · Beijing Institute of General Artificial Intelligence
Spend Your Rollouts Where It Counts: Rollout Allocation for Group-Based RL Post-Training
May 26, 2026Woojeong Kim, Ziyi Yang, Jing Nathan Yan +1Rollout Token Budget Allocation
Databricks · Cornell University
Not All Transitions Matter: Evidence from PPO
May 22, 2026Ajhesh BasnetTransitions Rollout
Department of Artificial Intelligence and Data Science KPR Institute of Engineering and Technology, Coimbatore
Rollout Cards: A Reproducibility Standard for Agent Research
May 12, 2026Charlie Masters, Ziyuan Liu, Stefano V. AlbrechtReproducibility Rollout
Deepflow · Nanyang Technological University
CuSearch: Curriculum Rollout Sampling via Search Depth for Agentic RAG
May 12, 2026Jianghan Shen, Siqi Luo, Xinyu Cheng +6Agentic Retrieval-Augmented Generation Systems Reinforcement Learning With Verifiable Reward
1Nanjing University · 2Shanghai Artificial Intelligence Laboratory · University of Hong Kong +1
Ace-Skill: Bootstrapping Multimodal Agents with Prioritized and Clustered Evolution
May 9, 2026Feng Xiong, Zengbin Wang, Yong Wang +5Self-Evolving Agents Multimodal Agents
1AMAP, Alibaba Group · 2CASIA · 3BNU
Where to Spend Rollouts: Hit-Utility Optimal Rollout Allocation for Group-Based RLVR
May 8, 2026Tao Wang, Shuo Li, Yan Sun +2Reinforcement Learning With Verifiable Reward Repair-Based Group-Relative Policy Optimization
University of Pennsylvania · New Jersey Institute of Technology · University of Tennessee, Knoxville
On Training in Imagination
May 7, 2026Nadav Timor, Ravid Shwartz-Ziv, Micah Goldblum +2Model-Based Reinforcement Learning Rollout
Weizmann Institute of Science · New York University · Columbia University
Selective Rollout: Mid-Trajectory Termination for Multi-Sample Agent RL
May 7, 2026Zhiyuan Zhai, Xin WangGroup-Based Reinforcement Learning Parallel Rollouts
Fudan University
Rollout Pass-Rate Control: Steering Binary-Reward RL Toward Its Most Informative Regime
May 6, 2026Tianshu Zhu, Wenyu Zhang, Xiaoying Zuo +8Rollout Pass-Rate Evaluation
Baidu
Using Common Random Numbers for Simulation-based Planning with Rollouts
May 6, 2026Sandarbh Yadav, Frederic J Maliakkal, Harshad Khadilkar +1Random Simulation-Based Reinforcement Learning
Department of Computer Science and Engineering, IIT Bombay, India

Oct 1, 2026, cs.LGJ/K move · Enter open · S save

Iterative Policy Refinement through Semantic Rollout Analysis

Feiyu Gavin Zhu, Qi Xu, Zhifei Deng, Zhigang Hua+3

Carnegie Mellon University · Meta

Imitation Rollout Refinement Inductive Bias

Open paper AskPDF arXiv

Structured policies improve efficiency, robustness, and interpretability in imitation learning by introducing task-specific inductive bias, but existing structure generation methods rely either on extensive human input or on static domain knowledge encoded in LLMs, which may be inconsistent with the expert demonstrations. We propose a closed-loop framework that iteratively refines structured policies using LLM-guided analysis of policy rollouts. By logging rollouts as semantically meaningful tabular data and prompting the LLM to generate diagnostic analysis code, our method identifies suboptimalities in the policy structure and iteratively corrects them without requiring human instruction. Experiments on car racing and door opening tasks show that our approach improves imitation learning performance by up to 15% over zero-shot LLM-generated structures and requires 75% less compute to achieve the same reinforcement learning performance. These results demonstrate that tabular rollout analysis provides an effective feedback signal to align LLM-generated policy structures with expert demonstrations, and we can utilize it to generate good policy structures automatically.

Momentum

Related topics

Latest papers 66