ReinVBC: A Model-based Reinforcement Learning Approach to Vehicle Braking Controller

Haoxin Lin; Junjie Zhou; Daheng Xu; Yang Yu

cs.ROApr 6, 2026

ReinVBC: A Model-based Reinforcement Learning Approach to Vehicle Braking Controller

Authors: Haoxin Lin, Junjie Zhou, Daheng Xu, Yang Yu

Organizations: National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · School of Artificial Intelligence, Nanjing University, Nanjing, China · Polixir Technologies, Nanjing, China

Abstract

Braking system, the key module to ensure the safety and steer-ability of current vehicles, relies on extensive manual calibration during production. Reducing labor and time consumption while maintaining the Vehicle Braking Controller (VBC) performance greatly benefits the vehicle industry. Model-based methods in offline reinforcement learning, which facilitate policy exploration within a data-driven dynamics model, offer a promising solution for addressing real-world control tasks. This work proposes ReinVBC, which applies an offline model-based reinforcement learning approach to deal with the vehicle braking control problem. We introduce useful engineering designs into the paradigm of model learning and utilization to obtain a reliable vehicle dynamics model and a capable braking policy. Several results demonstrate the capability of our method in real-world vehicle braking and its potential to replace the production-grade anti-lock braking system.

Figures & tables

Type	Name	Notation	Unit	Description
dynamic variable	vehicle speed	$v$	$\mathrm{km}/\mathrm{h}$	speed in the vehicle direction
	wheel speed	$\mathbf{w}=(w^{1},w^{2},w^{3},w^{4})$	$\mathrm{km}/\mathrm{h}$	linear speed of the four wheels rotating
	acceleration	$\mathbf{a}=(a_{x},a_{y},a_{z})$	$\mathrm{m}/\mathrm{s}^{2}$	acceleration in the $x$ , $y$ , $z$ directions
	attitude rate	$\dot{\mathbf{\omega}}=(\dot{\theta},\dot{\phi},\dot{\psi})$	$\mathrm{rad}/\mathrm{s}$	change rates of pitch, roll, and yaw angles
	wheel cylinder pressure	$\mathbf{p}=(p^{1},p^{2},p^{3},p^{4})$	$\mathrm{MPa}$	hydraulic pressure within the wheel cylinders
control variable	wheel action	$\mathbf{u}=(u^{1},u^{2},u^{3},u^{4})$	N/A	discrete instruction to control the wheel cylinder pressure

Table 1: Important vehicle variables.

Figure 1: Illustration of ReinVBC: vehicle braking controller by offline model-based reinforcement learning

Body	class	SUV
	length	4462mm
	width	1857mm
	height	1638mm
	wheelbase	2680mm
	clearance	170mm
	curb weight	1630kg

Table 2: Parameters of our experimental vehicle

action	description	operation
$c_{0}$	pressure maintenance	set the switching valve to braking mode, close the inlet valve and the outlet valve
$c_{1}$	pressure increase	set the switching valve to braking mode, open the inlet valve, close the outlet valve
$c_{2}$	pressure decrease	set the switching valve to braking mode, close the inlet valve, open the outlet valve
$c_{3}$	no pressure control	set the switching valve to normal mode

Table 3: Discrete action choices for each wheel.

Figure 2: Comparison between the model roll-out (in orange) and the real-world sequence (in blue).

Figure 3: Learning curves of SAC in the learned vehicle dynamics model. The solid lines indicate the mean while the shaded areas indicate the standard error over five different seeds.

Figure 4: In-distribution test results in real-world (with 40km/h as braking speed), in terms of braking distance (in meter) and braking deviation (in degree). The height of the bars represents the mean, and the error bars indicate the standard deviation, over five experiments. Due to the significant differences in values for braking deviation across different methods, the y-axis of the corresponding chart uses a logarithmic scale.

Scenario	Braking Speed	Braking Distance (m)			Braking Deviation (degree)
Scenario	(km/h)	ReinVBC	ABS	No Controller	ReinVBC	ABS	No Controller
high-adhesion straight	$100$	$34.9\pm 0.5$	$\mathbf{27.6\pm 0.6}$	$63.3\pm 0.3$	$<0.1$	$<0.1$	$<0.1$
low-adhesion straight	$55$	$65.0\pm 0.7$	$70.7\pm 0.5$	$\mathbf{56.8\pm 0.4}$	$\mathbf{0.3\pm 0.1}$	$\mathbf{0.3\pm 0.1}$	$439.3\pm 7.2$
high-to-low straight	$45$	$\mathbf{30.5\pm 0.6}$	$32.3\pm 0.5$	$46.8\pm 0.5$	$\mathbf{0.2\pm 0.0}$	$\mathbf{0.2\pm 0.0}$	$68.4\pm 5.1$
low-to-high straight	$45$	$\mathbf{25.7\pm 0.4}$	$\mathbf{25.6\pm 0.5}$	$41.6\pm 0.4$	$\mathbf{0.2\pm 0.0}$	$\mathbf{0.2\pm 0.0}$	$66.3\pm 7.2$
split-friction straight	$60$	$\mathbf{53.9\pm 3.2}$	$\mathbf{54.7\pm 2.3}$	$57.5\pm 1.4$	$0.6\pm 0.1$	$\mathbf{0.5\pm 0.1}$	$473.7\pm 9.1$

Table 4: hardware-in-loop Out-of-distribution Test Results

Scenario	Braking Speed	Braking Distance	Braking Deviation	Wheel Lock-up
Scenario	(km/h)	(m)	(deg)	(Yes/No)
high-adhesion straight	$100$	$47.7\pm 1.9$	$1.7\pm 0.8$	No
low-adhesion straight	$55$	$57.9\pm 2.0$	$0.9\pm 0.2$	No
high-to-low straight	$45$	$26.0\pm 2.9$	$0.7\pm 0.2$	No
low-to-high straight	$45$	$21.9\pm 0.8$	$1.2\pm 0.2$	No
split-friction straight	$60$	$33.8\pm 1.0$	$8.9\pm 0.4$	No

Table 5: Real-world Out-of-distribution Test Results

Figure 5: Speed curves during braking on the split-friction straight in the hardware-in-loop simulation. We separately compare the vehicle speed sequence with each wheel’s speed sequence. The solid lines indicate the wheel speed curves of each wheel, while the dashed line indicates the vehicle speed for reference.

Figure 6: Speed curves during braking on the split-friction straight in the real world. We separately compare the vehicle speed sequence with each wheel’s speed sequence. The solid lines indicate the wheel speed curves of each wheel, while the dashed line indicates the vehicle speed for reference.

Scenario	Braking Speed	Braking Distance (m)			Braking Deviation (degree)
Scenario	(km/h)	Simulation	Real World	Gap	Simulation	Real World	Gap
high-adhesion straight	$100$	$34.9$	$47.7$	$+12.8$	$<0.1$	$1.7$	$+1.6$
low-adhesion straight	$55$	$65.0$	$57.9$	$-7.1$	$0.3$	$0.9$	$+0.6$
high-to-low straight	$45$	$30.5$	$26.0$	$-4.5$	$0.2$	$0.7$	$+0.5$
low-to-high straight	$45$	$25.7$	$21.9$	$-3.8$	$0.2$	$1.2$	$+1.0$
split-friction straight	$60$	$53.9$	$33.8$	$-20.1$	$0.6$	$8.9$	$+8.3$

Table 6: Performance Comparison of Testing in Simulation and Real World

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Hyper-parameter	Value	Description
network	GRU(128)+FC(128,128)	a GRU layer followed by fully connected layers
$h$	20	the number of steps for observation stacking
$p_{\mathrm{dropout}}$	0.1	dropout rate
$lr_{\mathrm{model}}$	$1\times 10^{-4}$	learning rate
$m$	500	maximum roll-out length
optimizer	Adam	optimizer of the dynamics model

Appendix

Table 7: Hyper-parameters used to train our vehicle dynamics model

Hyper-parameter	Value	Description
$N_{Q}$	2	the number of Q networks
actor network	FC(256,256)	fully connected (FC) layers
critic network	FC(256,256)	fully connected (FC) layers
$\tau$	$5\times 10^{-3}$	target network smoothing coefficient
$\gamma$	0.99	discount factor
$lr_{\mathrm{policy}}$	$3\times 10^{-4}$	learning rate of the actor and the critic

Appendix

Table 8: Hyper-parameters used to optimize the policy

Figure 7: Comparison between the model roll-out and the real-world sequence. This sequence is sampled on a split-friction straight road.

Figure 8: Comparison between the model roll-out and the real-world sequence. This sequence is sampled on a split-friction straight road.

Figure 9: Comparison between the model roll-out and the real-world sequence. This sequence is sampled on a split-friction straight road.

Figure 10: Comparison between the model roll-out and the real-world sequence. This sequence is sampled on a split-friction straight road.

Figure 11: Comparison between the model roll-out and the real-world sequence. This sequence is sampled on a high-adhesion straight road.

Figure 12: Comparison between the model roll-out and the real-world sequence. This sequence is sampled on a low-adhesion straight road.

Figure 13: The braking process of the vehicle with our controller on a high-adhesion straight.

Figure 14: The braking process of the vehicle with our controller on a low-adhesion straight.

Figure 15: The braking process of the vehicle with our controller on a high-to-low straight.

Figure 16: The braking process of the vehicle with our controller on a low-to-high straight.

Figure 17: The braking process of the vehicle with our controller on a split-friction straight.

Figure 18: The braking process of the vehicle with our controller on a split-friction curve.

RL Control

Reinforcement learning control uses reward-driven learning to make decisions in dynamic systems, from robots and vehicles to energy and infrastructure.

See all 269 papers

ReBound: Reset-Free Reinforcement Learning for Agile Driving via Reset-Aware Semi-Markov Bootstrapping
Apr 9, 2026Kohei Honda, Aiki Kada, Hirotaka HosogayaReinforcement Learning RL for Autonomous Driving
Department of Mechanical Systems Engineering, Nagoya University, Aichi, Japan · CyberAgent AI Lab, Tokyo, Japan
Optimal Control with Learned Critics under Unmodeled State Dependencies
Oct 4, 2026Philipp Schoch, Markus RyllResidual Dynamics Learning Robot Dynamics Learning
Autonomous Aerial Systems Lab Technical University of Munich
Beyond Safety Filtering: Control Barrier Function-Informed Reinforcement Learning for Connected and Automated Vehicles
May 16, 2026Jianye Xu, Bassam AlrifaeeReward Shaping Control Barrier Functions
Department of Computer Science, RWTH Aachen University, Germany · Department of Aerospace Engineering, University of the Bundeswehr Munich, Germany

Apr 9, 2026J/K move · Enter open · S save

ReBound: Reset-Free Reinforcement Learning for Agile Driving via Reset-Aware Semi-Markov Bootstrapping

Kohei Honda, Aiki Kada, Hirotaka Hosogaya

Department of Mechanical Systems Engineering, Nagoya University, Aichi, Japan · CyberAgent AI Lab, Tokyo, Japan

Reinforcement Learning RL for Autonomous Driving

Open paper AskPDF arXiv

We present ReBound (reset-aware semi-Markov bootstrapping), a value-learning method for reset-free reinforcement learning (RL) that learns agile driving through real-world RL without human intervention after training starts. Reset-free RL explores by alternating between a forward policy that performs the task and a reset policy that restores a drivable state. Standard reset-free RL treats collisions as episode terminations in value-function bootstrapping, truncating the return at each collision. This conflicts with the reset-free objective, under which driving resumes after recovery, and acts as a hidden collision penalty absent from the reward function. ReBound instead applies semi-Markov bootstrapping: it treats the collision-causing action and the subsequent recovery as a single semi-Markov transition and bootstraps from the post-recovery value, discounted by the actual recovery time. We further apply reward centering to stabilize value learning in the resulting continuing task. In simulation and on a 1/10-scale vehicle, ReBound substantially outperforms conventional value-learning methods and model-based control.

Abstract

RL Control

Explore similar work

ReBound: Reset-Free Reinforcement Learning for Agile Driving via Reset-Aware Semi-Markov Bootstrapping

Optimal Control with Learned Critics under Unmodeled State Dependencies

Beyond Safety Filtering: Control Barrier Function-Informed Reinforcement Learning for Connected and Automated Vehicles