Precision at Speed: Sample-Efficient Online Model-Based Reinforcement Learning for Hydraulic Excavator Control

Claudio Canales; Fang Nan; Marco Hutter; Javier Ruiz-del-Solar

cs.ROSep 25, 2026

Precision at Speed: Sample-Efficient Online Model-Based Reinforcement Learning for Hydraulic Excavator Control

Authors: Claudio Canales, Fang Nan, Marco Hutter, Javier Ruiz-del-Solar

Organizations: Robotic Systems Lab, ETH Zürich, 8092 Zürich, Switzerland · Advanced Mining Technology Center (AMTC) and the Department of Electrical Engineering, Universidad de Chile, Santiago, Chile

Abstract

Precise, high-speed control remains challenging for robots with complex actuation dynamics. Learning directly on hardware is further constrained by the cost of real-world interaction. We present an online model-based reinforcement learning framework that learns a probabilistic dynamics ensemble model from scratch for sampling-based model predictive control. A precision-gated contouring objective conditions the progress reward on path accuracy, prioritizing precision over speed. In a data-driven excavator simulator, the framework achieves higher sample efficiency than the evaluated model-based reinforcement learning baselines. We validate the framework by learning directly on an 11.5-ton Menzi Muck M445 hydraulic excavator, without demonstrations or simulation pretraining. After 20 minutes of interaction, the controller reaches tracking accuracy comparable to prior learned controllers trained on 100-150 minutes of data. After 40 minutes, it sustains sub-centimeter mean path error at high operating speeds.

Figures & tables

Fig. 1: Online model-based control framework. Data collected on the machine update a probabilistic dynamics ensemble, through which sampling-based MPC plans under a task-specific reward. The same learned dynamics support time-indexed tracking and contouring.

Fig. 2: Contouring. The current state is projected onto the forward reference window

m\in[\phi^{-},\phi^{-}+K]

(blue). The minimizer

\phi

defines the weighted contour cost

c

and normalized progress

\Delta\phi

; the precision gate attenuates rewarded progress as

c

increases.

Fig. 3: Top: one-step model MSE. Middle: tracking MSE. Bottom: end-effector trajectories after 20, 100, and 400 training episodes. For the latent-space models, we plot the latent-space prediction MSE loss of TD-MPC2 and the decoded state-space prediction MSE of Dreamer, as the latter method uses categorical latent variables in its open-source implementation. The model errors are scaled based on their value at iteration 0 to normalize the scale of different prediction spaces.

Fig. 4: Experimental platform: the 11.5-ton Menzi Muck M445 hydraulic excavator, whose four arm joints are controlled through the learned dynamics model.

Fig. 5: Training from scratch on the machine, one run per reward mode. Top: (a) contour error $e_{cont}$ , (b) joint contour error $e_{joint}$ , (c) end-effector speed, (d) time per trajectory $T_{traj}$ , over interaction time $T_{int}$ ; 5-episode rolling means; dashed: time tracking under its own time-indexed objective; red marks: the snapshots below. Bottom: $K{=}7$ end-effector paths at 3, 10, and 20 minutes of training and at the frozen 60-minute evaluation.

System	Control rate	$25\text{\,}\mathrm{H}\mathrm{z}$	Planner ( MPPI )	Samples $N$	3000
	Controlled joints	4		Horizon $T$	30 ( $1.2\text{\,}\mathrm{s}$ )
	Command	valve current		Iterations	3
	Reference	$6\text{\,}\mathrm{s}$ quintic		Temperature $\lambda$	0.05
	Trajectories/episode	10		Sampling std	0.5
	Warm start	$2\times$ 60\text{,}\mathrm{s}$$		EMA $\alpha$	0.18
	Authority $\beta_{e}$	$0.5\rightarrow 1.0$ , $+0.1$ /ep.

TABLE I: System, model, planner, and reward parameters.

		$\rho_{M}$	$v$ [cm/s]		$e_{time}$ [cm]			$e_{cont}$ [cm]			$T_{int}$
		[s]	avg	max	avg	p95	max	avg	p95	max	[min]
M545	Egli & Hutter [ 6 ]	0.13	20	53	2.8	–	6.7	–	–	–	100
	Nan & Hutter [ 7 ]	0.27	37.5	60	8.56	–	16.2	–	–	–	sim+10
	Nan et al. [ 9 ]	0.10	52	129	4.5	–	12.4	–	–	–	150
Ours, M445	Time tracking, 20	0.12	28	137	2.7	7.9	16.3	1.1	3.1	7.5	20
	$K{=}1$ , 20	0.05 †	26	94	–	–	–	0.87	2.5	5.1	20

TABLE II: Comparison with learned excavator controllers;

\rho_{M}=|e|_{max}/v_{max}

is the speed-normalized error metric.

Fig. 6: Precision-gate ablation with the frozen 60-minute time-tracking model ( $\sigma=0.05$ ; auxiliary penalties disabled). (a) Mean contour error versus end-effector speed across $\rho$ ; open markers indicate below 90% completion. (b) Measured p95 contour cost and theoretical break-even thresholds without fitting. (c),(d) Ungated and gated paths at $\rho=160$ .

Fig. 7:

3{\times}3

cross-evaluation at 60 minutes (lines: trained model; dots: seed means). The deployed objective sets performance; the models separate only under the

K{=}7

evaluation.

Robotic Control

Robotic control studies how machines use sensing and computation to produce reliable movement and interaction in changing environments.

See all 584 papers

IL-ACT: Imitation Learning with Adaptive Cartesian Tracking Control for a 30-ton Excavator
Sep 15, 2026Mehdi Heydari Shahna, Seihun Kim, Soyi Jung +3Robotic Control Robot Trajectory Tracking
Faculty of Engineering and Natural Sciences, Tampere University, Tampere, Finland · Department of Defense Convergence Technology, Korea University, Seoul, Republic of Korea · Department of Electrical and Computer Engineering, Korea University, Seoul, Republic of Korea +2
Learning a System-Level Surrogate for Hydraulic Excavators: A Simulation-to-Real LSTM Approach
Jul 17, 2026Shuai Wang, Shen Wang, Qiang Wang +4Neural Surrogate Modeling Robotic Construction
National Engineering Research Center of Mobile Network Technologies, Beijing University of Posts and Telecommunications, Beijing 100876, China · Yaowu (Shenzhen) Technology Co., Ltd., Shenzhen 518052, China
Size Doesn't Matter: Material-State Reinforcement Learning for Excavator Transferable Soil Manipulation
Sep 11, 2026Lennart Werner, Pol Eyschen, Sean Costello +3RL for Robotics Deformable Object Manipulation
Robotic Systems Lab, ETH Zürich, Leonhardstrasse 21, 8092 Zürich, Switzerland · Hexagon Innovation Hub GmbH, Heinrich-Wild-Strasse 201, 9435 Heerbrugg, Switzerland

Sep 15, 2026J/K move · Enter open · S save

IL-ACT: Imitation Learning with Adaptive Cartesian Tracking Control for a 30-ton Excavator

Mehdi Heydari Shahna, Seihun Kim, Soyi Jung, Soohyun Park+2

Faculty of Engineering and Natural Sciences, Tampere University, Tampere, Finland · Department of Defense Convergence Technology, Korea University, Seoul, Republic of Korea · Department of Electrical and Computer Engineering, Korea University, Seoul, Republic of Korea+2

Robotic Control Robot Trajectory Tracking Robot Imitation Learning Adaptive Control Robotic Excavation

Open paper AskPDF arXiv

Autonomous excavator control is challenged by coupled kinematics, actuation lag, and uncertainty. We propose imitation learning and adaptive Cartesian tracking (IL-ACT), a novel motion control framework for a 30-ton-class excavator. An anchored, 14-input imitation policy pretrained on operator demonstrations generates nominal joint rates; adaptive Cartesian feedback and gated gain/bias estimation correct these commands before a stopping-distance governor constrains joint-reference generation. Simscape evaluation covers 100 sequential goals and spiral, figure-eight, and rounded-raster tracking, including 88 additional runs across three training seeds, two initializations, and speeds, under hydraulic response and sensing conditions. Compared with Teacher+ACT, IL-ACT completes all goals with shorter duration and lower terminal errors under both response conditions. Telemetry-initialized IL-ACT lowers RMSE in all 24 figure-eight and rounded-raster seed comparisons and lowers additional-load spiral mean RMSE by approximately 29%. Original spiral RMSE also improves over IL-only and PID. Under a shared sensor-noise realization, telemetry-initialized IL-ACT achieves 27.67% lower mean RMSE than Teacher+ACT; enabling estimation reduces mean RMSE by

22.44\%

relative to the frozen estimator. Pretrained-weight effects remain mixed, and the original teacher comparison exhibits a spiral RMSE--maximum-error tradeoff. Analysis establishes bounded adaptive states and Cartesian feedback, with reference admissibility conditional on governor feasibility.

Abstract

Robotic Control

Explore similar work

IL-ACT: Imitation Learning with Adaptive Cartesian Tracking Control for a 30-ton Excavator

Learning a System-Level Surrogate for Hydraulic Excavators: A Simulation-to-Real LSTM Approach

Size Doesn't Matter: Material-State Reinforcement Learning for Excavator Transferable Soil Manipulation