cs.RONov 15, 2026

Value and Policy Calibration from Execution Feedback for Socially Compatible Driving

Authors: Sen Shen, Ning Liu, Xiangrui Kong, Yingwei Li, Dongkun Han

Organizations: Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong, China. · Department of Electrical, Electronic and Computer Engineering, The University of Western Australia, Perth, Australia.

Abstract

Autonomous vehicle (AV) decisions at intersections affect the delay, braking, and motion smoothness of surrounding human-driven vehicles (HDVs). Learning socially compatible behavior requires connecting imagined interaction outcomes with feedback from executed actions. We propose a world-model reinforcement learning framework combines imagined policy improvement with direct value and policy calibration from finalized interactions, balancing AV progress with HDV interaction burden. An interaction-aware latent model supports imagination, while off-policy correction and replay updates enable observed rewards and transitions to supervise both the critic and actor. Simulation experiments at an unsignalized intersection achieve a 100% success and reductions in following-HDV acceleration variation of 65.5% and 44.0% relative to PIWM and DreamerV3, respectively. Ablations identify complementary roles for value calibration in reducing HDV burden and policy calibration in improving passage efficiency. Tests under out-of-distribution traffic shifts further show lower ego jerk and HDV acceleration variation than PIWM at comparable passage times.

Figures & tables

Explore similar work

CardsList
  1. Modeling Vehicle-Type-Specific Pedestrian Crash Avoidance Behavior in Safety-Critical Interactions Using Smooth-Mamba Deep Reinforcement Learning

    May 27, 2026Qingwen Pu, Kun Xie, Hong Yang +2Collision AvoidancePedestrian Behavior Modeling

  2. Beyond Policy Support: Interaction Constrained Offline Reinforcement Learning for Autonomous Driving

    Oct 7, 2026Mahmoud Selim, Cristina Cipriani, Karl Henrik JohanssonOffline RLDistribution Shift

  3. Characterizing Driver Interactions with Autonomous Vehicles via Response Maps

    Jun 26, 2026Dave Broaddus, Rachel DiPirro, Chishang +4Autonomous Driving