cs.RONov 8, 2026

Off-Policy RL with Online Inference Tuning: My Solution to the CoRL 2026 RoboEval Challenge

Authors: Ilia Larchenko

Abstract

The CoRL 2026 RoboEval challenge has eight bimanual tasks, and it ranks a policy not only on whether it succeeds but on how: every stage of the task, done safely, efficiently and with both arms in coordination, scored against human demonstrations. My solution has two parts. The generic part, carried over from my LeHome challenge solution, is off-policy RL on a π0.5π_{0.5}-based policy that is also its own value function. It is trained with AWR and RECAP on advantages of the exact official score, in a loop where every rollout is at the same time a Thompson-sampling trial of about 20 inference parameters. Out of the box, it took the policy to 0.85 RDS on held-out seeds in two days (and it was still very far from convergence). The competition-specific part adds data that local exploration cannot find: generated demonstrations that reach the full reward, and recovery episodes that start from the policy's own failures. The submitted policy reaches 0.990 RDS on held-out seeds and 0.987 on the official seeds.

Figures & tables

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards

    Jun 1, 2026Christian Scherer, Joe Watson, Theo Gruner +3Reinforcement LearningRobot Skill Learning

  2. Evo-RL: Iterative Robot Reinforcement Learning from Selective Experience

    Oct 9, 2026Kai Ye, Yifan Jin, Qianhang Luo +7Robotic RLRobot Learning