Learning the Right Abstraction: Neural Reduced Dynamics for Complex Robot Control

Harry Zhang; Dan Negrut

cs.ROAug 19, 2026

Learning the Right Abstraction: Neural Reduced Dynamics for Complex Robot Control

Authors: Harry Zhang, Dan Negrut

Organizations: Department of Mechanical Engineering, University of Wisconsin–Madison, Madison, 53706, WI, USA

Abstract

High-fidelity embodied AI simulators provide realistic evaluation of complex robotic systems, but their computational cost limits their direct use for large-scale reinforcement learning campaigns. We advocate the use of less accurate but more expeditious simulations, which might draw on data-driven, e.g., neural dynamics, models. This contribution argues that the practical value of a neural dynamics model for complex robot control lies in learning the right abstraction: a reduced state that preserves the control-relevant physics of the high-fidelity system while enabling high-throughput policy learning. We develop a neural reduced dynamics (NRD) framework that separates the state the model propagates from what can be supplied as an input or recovered analytically, trains policies entirely inside the frozen learned model, and validates them back in the high-fidelity simulator. Two case studies instantiate it across three control tasks: terrain-aware HMMWV trajectory tracking on rigid, bumpy and deformable Continuum Representation Model (CRM) terrain; and goal reaching for a stock tracked vehicle and its front-mounted articulated arm. Every policy transfers back to the high-fidelity simulator. A single policy trained inside the terrain-conditioned dynamics model, and given no terrain input of its own, attains lower median and mean tracking error than both single-terrain specialists on all three terrains, including zero-shot bumpy terrain. Quantitatively, the tracked vehicle reaches 100 of 100 goals and the arm 97 of 100, with zero contacts or joint-limit violations. The NRD models advance roughly four orders of magnitude faster in simulated time than the high-fidelity simulator scenes they replace, making iterative on-policy learning practical and supporting neural reduced dynamics as a bridge between accurate but expensive physics simulation and scalable robot learning.

Figures & tables

Dataset	Role	Scale	Code
Flat rigid	Train, val, in-domain test	$\approx$ 82k episodes ( $\sim$ 300 GiB); 329 M / 81 M transitions	$[1,0]$
CRM soil	Train, val, in-domain test	2,000 episodes; 2.28 M / 0.60 M transitions	$[0,1]$
Bumpy rigid	Zero-shot OOD test only	20 episodes are selected for chrono closed-loop policy evaluation	$[1,0]$

Table 1: HMMWV datasets and their roles. Sizes are episodes and processed 15-D transitions (training / validation). Bumpy rigid terrain keeps the rigid regime code and is held out for zero-shot evaluation only.

Setting	Rigid (flat, bumpy)	CRM (soil)
Tire model	TMEASY (analytical)	RIGID_MESH (FSI)
Terrain / contact	SMC rigid surface	SPH–FSI deformable soil
Tire-force source	analytical tire report	FSI spindle force
Friction / soil	$\mu=0.9$	coh. $5$ kPa, $\mu_{s}{=}0.8$ , $\rho{=}1700$ kg/m 3 , $E{=}1$ MPa
Integration step	2 ms	0.5 ms
Terrain extent	flat / $500{\times}500$ m heightmap ( $\pm 0.6$ m)	$150{\times}150{\times}0.25$ m box, $0.08$ m spacing

Table 2: Simulation settings by terrain regime. All regimes share the fixed HMMWV_Full vehicle (SMC contact, SHAFTS engine, automatic transmission, AWD, Pitman-arm steering) and record at 100 Hz; they differ only in the tire–terrain contact model.

	One-step loss		10 s rollout err/dist
Dynamics model	Rigid	CRM	Rigid	CRM
Generalist (75/25)	0.0025	0.094	3.7%	5.4%
Rigid-only	0.0018	1.27 †	3.8%	194.1% †
CRM-only	0.72 †	0.11	39.6% †	4.2%

Table 3: Terrain-conditioned generalist vs. single-terrain specialists (rigid-only, CRM-only), each at its selected checkpoint. One-step loss is the channel-reweighted Huber validation loss; rollout err/dist is the 10 s open-loop position error normalized by distance traveled. † zero-shot: specialist evaluated on the terrain it was not trained on.

Item	Tracked vehicle	Arm
Physical task	planar goal reach	end-effector reach
Reduced-state dim	$3$	$8$
Action dim	$3$	$4$
Sampling interval	$0.02$ s	$0.02$ s
Context length	$16$	$16$
Transformer layers	$3$	$5$

Table 4: Task-specific NRD-model configurations. Both instantiate the same causal next-state framework, while their state definitions and model capacities are selected according to their respective tasks. The arm column is the dynamics model used to train the reaching policy.

Model	One-step	Open-loop err/dist
Tracked Vehicle NRD model	$v_{x}\,0.024$ , $v_{y}\,0.013$ m/s; $r\,0.0069$ rad/s	$5.85\%$ / $0.105$ m XY
Arm NRD model	$1.2$ mm (ee)	$1.2\%$ ee drift

Table 5: NRD-model accuracy. Vehicle one-step is the per-channel one-step RMSE of all three reduced states (

v_{x}

v_{y}

in m/s; yaw rate

r

in rad/s); vehicle open-loop is the

5

s pose-integrated position RMSE / err/dist. Arm one-step is the end-effector position RMSE (forward kinematics on the predicted joints); arm open-loop is the end-effector-drift err/dist at

2

s (

0.3\%

0.5

s). All arm numbers are for the deployed

5

-layer model used to train the reaching policy (Table 4 ).

Setting	Vehicle policy	Arm policy
Parallel environments	$2{,}048$	$4{,}096$
Control rate	$10$ Hz	$50$ Hz
Observation / action dim	$11$ / $3$	$26$ / $4$
Actor/critic hidden	$[256,128,64]$	$[256,128,64]$
Learning rate	$3\!\times\!10^{-4}$	$1\!\times\!10^{-4}$
Adaptive KL target	$0.01$	$0.005$

Table 6: Task-specific PPO configurations using a common implementation and actor–critic topology, each trained inside its frozen NRD model; both use tanh -squashed actions, GAE (

\lambda=0.95

), a clip ratio of

0.2

, and empirical observation normalization. ‡ the vehicle run is scheduled for

3{,}000

iterations but converges to

\approx 100\%

NRD-model success within the first

\sim 150

iterations; the iteration-

1{,}500

checkpoint (long converged) is used for transfer.

Task	Trials	Success	Median final err
Vehicle goal reaching	$100$	$100\%$	$0.69$ m
Arm reaching	$100$	$97\%$	$0.042$ m

Table 7: Chrono transfer summary over the seeded

100

-goal stress batteries (Appendix G ). Vehicle:

100/100

within

0.75

m, median closest approach

0.69

m (worst

0.748

m), median time-to-success

20.2

s, median path efficiency

0.959

. Arm:

97/100

within

0.05

m (median reached error

4.2

cm), all three misses timeouts at under-sampled deep lower-workspace goals with zero contacts or joint-limit violations. Error scales differ because the two tasks use very different success tolerances.

Control task	Learned recurrent state	Action / context	Evaluated outside the NRD model	Why this abstraction is needed	Chrono evidence
HMMWV tracking	Body velocities, attitude, and angular rates; per-wheel normal loads and speeds ( $15$ -D)	Steering, throttle, braking; rigid/CRM terrain code	Vehicle pose $(x,y,\psi)$ ; body frame error from reference trajectories	Exposes wheel slip, load transfer, and the domain-dependent tire–terrain response	Lower median and mean tracking error than both single-terrain specialist policies on flat, CRM, and zero-shot bumpy terrain
Tracked vehicle reaching	Planar velocities and yaw rate ( $3$ -D)	Steering, throttle, braking	Vehicle pose $(x,y,\psi)$ ; body frame error from goal point	Skid-steer chassis response in the plane is pose-invariant; pose is integrable	$100/100$ goals within $0.75$ m
Arm reaching	Joint positions and velocities ( $8$ -D)	Joint target command $q^{\mathrm{cmd}}$	End-effector $p^{\mathrm{ee}}=\mathrm{FK}(q)$ and contact clearance	Joint response is dynamic and depends on $q$ ; task geometry is known analytically	$97/100$ goals within $0.05$ m; zero contacts or joint-limit violations

Table 8: The same design pattern across three control tasks: each keeps a task-specific reduced state, treats commands and context as inputs, and recovers known kinematics analytically—yet every one yields a policy that transfers into high-fidelity Chrono.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Configuration	$\Delta t$	Steps/s	RTF	Cost
HMMWV — rigid flat	$2.0$ ms	$3{,}657$	$7.31\times$	$239\times$
HMMWV — bumpy rigid	$2.0$ ms	$3{,}324$	$6.65\times$	$263\times$
HMMWV — CRM soil	$0.5$ ms	$304$	$0.152\times$	$11{,}500\times$
HMMWV — NRD model ( $8$ -layer) ‡	$10$ ms	$175{,}000$	$1{,}750\times$	$1\times$

Appendix

Table 9: Case Study I (terrain-aware HMMWV) throughput. Chrono rows share the fixed HMMWV_Full configuration and differ only in the tire–terrain contact model. Cost is normalized to the NRD model row (

1\times

). ‡ The NRD model row is an aggregate batched-GPU figure, not a single-stream process like the three Chrono rows above: Steps/s is dynamics forward passes/s summed across the

2{,}048

parallel rollouts used during PPO training (Sec. 4.5 ), read from the rsl-rl training logs ( Perf/total_fps , consistently

\approx 35{,}000

policy-control steps/s across all three

8

-layer policies,

\times

action_repeat

=5

); RTF is the corresponding aggregate simulated-seconds advanced per wall-clock second.

Configuration	$\Delta t$	Steps/s	RTF	Cost
Tracked vehicle $+$ arm — rigid	$0.5$ ms	$558$	$0.279\times$	$58{,}400\times$ / $8{,}300\times$
Tracked vehicle — NRD model ‡	$20$ ms	$815{,}000$	$16{,}300\times$	$1\times$
Arm — NRD model ‡	$20$ ms	$116{,}000$	$2{,}320\times$	$1\times$

Appendix

Table 10: Case Study II (tracked vehicle

+

arm) throughput, measured the same way as Table 9 . The single Chrono vehicle-plus-arm scene serves both control modes; each NRD model row is its own

1\times

reference, and the Chrono Cost cell lists the wall-clock ratio over the drive-mode / reach-mode NRD model respectively. ‡ as in Table 9 , both NRD model rows are aggregate batched-GPU figures (not single-stream like the Chrono row): Steps/s is dynamics forward passes/s during PPO, read from the rsl-rl logs as Perf/total_fps

\times

action_repeat. Drive mode uses tracked_transformer_v1 (

2{,}048

rollouts, Perf/total_fps

\approx 163{,}000

\times

action_repeat

5

); reach mode uses the larger arm_transformer_8d_v1 (

4{,}096

rollouts, Perf/total_fps

\approx 116{,}000

, action_repeat

1

). The Chrono scene simulates the full coupled vehicle-plus-arm system, whereas each NRD model represents only the subsystem its mode controls.

Terrain	Policy	Median	Mean	IQR
Rigid flat	Mixture generalist	0.157	0.184	0.099–0.236
	Rigid-only	0.174	0.219	0.163–0.277
	CRM-only	0.232	0.259	0.189–0.328
CRM	Mixture generalist	0.180	0.249	0.134–0.321
	Rigid-only	0.854	1.000	0.365–1.447
	CRM-only	0.231	0.361	0.174–0.521

Appendix

Table 11: Closed-loop Chrono tracking statistics (XY RMSE, meters) for the mixture generalist and the rigid-only / CRM-only specialists, on held-out rigid flat, CRM, and zero-shot bumpy references; these are the numbers summarized by Fig. 5 . Median, mean, and IQR are over

20

references at the

1000

-iteration checkpoint, all

20/20

successful. Bold marks the best median and mean in each terrain block; on this

8

-layer backbone the mixture generalist takes every bold cell, i.e. it achieves the lowest mean and median XY RMSE on all three evaluated terrains. † bumpy is a strict zero-shot test: rigid code, no bumpy data in training or selection.

Model	$n_{\mathrm{layer}}$	$n_{\mathrm{embd}}$	head_dim	ctx	One-step loss		10 s rollout err/dist		$S$
					Flat	CRM	Flat	CRM
Ablation model 1 (used)	8	256	32	128	0.0025	0.094	3.7%	5.4%	4.6%
Ablation model 2	6	256	32	256	0.0021	0.114	6.1%	3.7%	4.9%
Ablation model 3	6	512	32	128	0.0020	0.103	4.6%	5.3%	4.9%
Ablation model 4	6	384	32	128	0.0030	0.095	5.6%	4.9%	5.2%
Ablation model 5	6	256	16	128	0.0021	0.097	6.7%	3.9%	5.3%

Appendix

Table 12: All

14

configurations in the dynamics-architecture sweep, sorted by selection score

S

(Eq. 9 ; lower is better), each at its own best epoch. Each model varies exactly one of depth (

n_{\mathrm{layer}}

), width (

n_{\mathrm{embd}}

, with the attention head count co-scaled so

\mathrm{head\_dim}=n_{\mathrm{embd}}/n_{\mathrm{head}}

stays fixed at

32

), attention-head width (head_dim, at fixed

n_{\mathrm{embd}}=256

—the axis that actually changes head_dim), or train-time context (ctx) from the shared

6

-layer/

256

-embedding/head_dim-

32

128

-context configuration (Ablation model 10). One-step loss is the channel-reweighted Huber validation loss as training (Eq. 8 ); rollout err/dist is the

10

s open-loop position error normalized by distance traveled. The top row (bold), Ablation model 1, is the

8

-layer configuration used throughout this paper.

	One-step loss		10 s rollout err/dist
Data	Flat	CRM	Flat	CRM	$S$
20%	0.0023	0.138	4.2%	9.6%	6.9%
40%	0.0022	0.124	8.1%	4.8%	6.5%
60%	0.0021	0.114	7.0%	5.7%	6.3%
80% †	0.0020	0.105	3.9%	4.6%	4.3%
100% (used)	0.0025	0.094	3.7%	5.4%	4.6%

Appendix

Table 13: Data-quantity ablation at the

8

-layer backbone: selection score

S

(Eq. 9 ) and its one-step-loss/rollout-err components at each run’s own best epoch, as a function of the fraction of training episodes used. †

80\%

attains a marginally lower single-seed

S

than the full-data point; see discussion below.

Model	One-step loss		10 s open-loop err/dist
	Flat	CRM	Flat	CRM
Baseline (20-D)	0.00248	0.09378	0.0373	0.0538
No one-hot (18-D)	0.00236	0.09395	0.0826	0.0486
No terramechanics (12-D)	0.00341	0.06198 †	0.0341	0.0800

Appendix

Table 14: Reduced-state and context ablation at the

8

-layer backbone, metrics at each run’s own best-validation checkpoint. One-step loss is the channel-reweighted Huber validation loss; 10 s open-loop err/dist is the open-loop rollout position error normalized by distance traveled,

12

episodes/domain. † computed over

7

readout channels instead of

15

and not comparable to the other two rows (see text); the open-loop column is comparable across all three, since it is integrated from

v_{x},v_{y},\omega_{z}

, which every variant retains.

Robotic Control

Robotic control studies how machines use sensing and computation to produce reliable movement and interaction in changing environments.

See all 583 papers

Oct 6, 2026

FlashNeRD: Performance-First Contact-Rich Neural Robot Dynamics

Compared with analytical physics, learned dynamics models promise robot simulation that is faster, inherently differentiable, and easily adaptable to real data. Neural Robot Dynamics (NeRD) pursues this by keeping collision detection analytical and replacing a simulator's numerical dynamics for the robot with a learned model. Three limitations remain. NeRD offers little speedup over the simulator it learned from, accepts contact only at predefined points, and has no two-way coupling with objects it manipulates. FlashNeRD removes all three with a parallel streaming architecture that makes each prediction faster and more accurate, an encoder that accepts contacts wherever they occur, and two-way coupling with objects simulated by analytical solvers. Experiments across five robots show faster and more accurate dynamics, faster policy learning, and faster inference-time planning. Across three robots, FlashNeRD's dynamics model is up to

55\times

faster than an optimized NeRD and more accurate over long rollouts. This speedup extends to policy learning, where PPO trains an ANYmal locomotion policy in 34 seconds,

3.8\times

faster than the analytical simulator and

2.7\times

faster than an optimized NeRD. With DIAL-MPC, a sampling-based MPC method, the robot climbs all six test platforms where fixed-contact NeRD manages one, at approximately half the analytical simulator's planning time. Cube-reorientation policies trained with FlashNeRD complete within 2% of the simulator-trained policy's target count.

Mohammadmehdi Ataei, Mohammad Amin Nabian, Jie Xu +9

NVIDIA

Robot Dynamics Learning Robotics Simulation Robotic Control

Ask PDF

May 2, 2026

Dynamics Distillation for Efficient and Transferable Control Learning

Robust control policy learning for autonomous driving requires training environments to be both physically realistic and computationally scalable, properties that existing simulators provide only in isolation. We introduce Sim2Sim2Sim, a framework that bridges high-fidelity vehicle simulation and scalable reinforcement learning by distilling simulator dynamics into a highly parallelizable learned dynamics model. By training control policies purely within this distilled environment and deploying them back into the high-fidelity source simulator, we demonstrate more efficient policy optimization and reliable transfer under challenging dynamics. We further show that predictive accuracy alone does not fully characterize a learned dynamics model's suitability as a reinforcement learning training environment, which should also be assessed by the quality of the policies it enables.

Xunjiang Gu, Kashyap Chitta, Mahsa Golchoubian +2

University of Toronto · Vector Institute · NVIDIA +1

Vehicle Dynamics Transfer Learning RL for Autonomous Driving Autonomous Driving Simulation Sample-Efficient RL

Ask PDF

Apr 16, 2026

Abstract Sim2Real through Approximate Information States

In recent years, reinforcement learning (RL) has shown remarkable success in robotics when a fast and accurate simulator is available for a given task. When using RL and simulation, more simulator realism is generally beneficial but becomes harder to obtain as robots are deployed in increasingly complex and widescale domains. In such settings, simulators will likely fail to model all relevant details of a given target task and this observation motivates the study of sim2real with simulators that leave out key task details. In this paper, we formalize and study the abstract sim2real problem: given an abstract simulator that models a target task at a coarse level of abstraction, how can we train a policy with RL in the abstract simulator and successfully transfer it to the real-world? Our first contribution is to formalize this problem using the language of state abstraction from the RL literature. This framing shows that an abstract simulator can be grounded to match the target task if the grounded abstract dynamics take the history of states into account. Based on the formalism, we then introduce a method that uses real-world task data to correct the dynamics of the abstract simulator. We then show that this method enables successful policy transfer both in sim2sim and sim2real evaluation.

Yunfu Deng, Yuhao Li, Josiah P. Hanna

Department of Computer Sciences, University of Wisconsin–Madison, Madison, WI 53706 USA · Manning College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, MA 01003 USA

Sim-to-Real Transfer State Abstraction Robotic RL

Ask PDF

Abstract

Robotic Control

Explore similar work

FlashNeRD: Performance-First Contact-Rich Neural Robot Dynamics

Dynamics Distillation for Efficient and Transferable Control Learning

Abstract Sim2Real through Approximate Information States