FastRoverVLA: Fast Vision-Language-Action Execution for Off-Road Autonomous Driving
Authors: Changtao Zhao, Yingxue Guan, Li Wang, Shijie Zhao, Yuhang Zhang, Shengkai Zhang, Qing Wang, Ziying Song
Organizations: School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081, China · Chongqing Innovation Center, Beijing Institute of Technology, Chongqing 401120, China · School of Mechanical Engineering and Intelligent Equipment, Chang’an University · School of Electronic and Information Engineering, Beijing Jiaotong University · School of Artificial Intelligence (School of Software), Yanshan University
Off-road vision-language-action (VLA) models generate scene-conditioned plans, but frequent trajectory updates can repeatedly invoke expensive multimodal inference. Reusing a prediction reduces this demand while leaving its commands exposed to changes in vehicle motion. FastRoverVLA addresses this mismatch by decoupling visual plan generation from motion-conditioned execution. Its training-free interface maintains decoded plans between visual predictions. A speed-curvature action buffer is aligned with measured motion at each update, integrated into a local trajectory, and advanced by consuming the current action. The corrected remainder is written back to the buffer, preserving motion corrections across successive updates. A bounded refresh rule normalizes speed and curvature variations within a newly decoded plan and assigns a short or long reuse budget according to the resulting plan-roughness score. The interface requires neither backbone features nor an additional learned driving policy. Over a 3 s horizon, independent real-request evaluation achieves average displacement error (ADE) and final displacement error (FDE) of 0.328/0.596 m on ORAD-3D and 0.266/0.526 m on RELLIS-3D, reducing FDE by 18.0% and 36.7% at request rates of 6.0% and 4.8%, respectively. These runs retain lower errors across four backbones and both datasets and reduce Qwen3-VL request-processing cost by factors of 16.6 and 20.8 under serial replay, while mechanism and motion-only controls identify persistent state correction as the main source of the observed accuracy benefit. The source code and evaluation materials are publicly available at https://github.com/ADEPT-ORG/FastRoverVLA.
Figures & tables
Figure 1: Motivation of FastRoverVLA. (a) Frame-synchronous prediction. (b) State mismatch during direct reuse. (c) Recurrent state alignment and write-back between plan refreshes. The separated paths depict distinct planning and trajectory-update rates; the arrows indicate scheduling dependencies.
Figure 2: FastRoverVLA framework. A frozen VLM initializes a speed–curvature buffer from images, ego motion, and an instruction. Consume–align–write-back updates produce integrated waypoints between budgeted refreshes (Section 3.3 ). The control panel denotes a potential downstream interface; evaluation uses waypoints and logged states, without a vehicle controller.
Figure 3: Plan-variation-based bounded refresh. Speed and curvature variations are normalized and aggregated into the plan-roughness score zt . The threshold τ selects a short reuse budget Ks for higher variation and a long reuse budget Kl otherwise.
Method
L 2 error ↓
ADE ↓
RMSE ↓
P95FDE ↓
1 s
2 s
3 s
OFF-EMMA [ 41 ]
0.168
0.428
0.805
0.395
0.336
2.068
ReCogDrive [ 11 ]
0.292
0.693
1.365
0.692
0.586
3.306
AutoVLA [ 8 ]
0.293
1.217
3.089
1.235
1.159
5.293
SimLingo [ 22 ]
2.411
8.201
14.067
6.955
5.951
20.018
DriveMoE [ 10 ]
1.347
3.763
6.219
3.206
2.684
10.480
Table 1: RELLIS-3D system comparison under the input and horizon adaptations in Section 4.1 . FastRoverVLA has the lowest displayed errors. Frozen weights; 3 s horizon; errors: m ( ↓ ); bold: column minima.
Figure 4: Frame-level 3 s ADE on RELLIS-3D under the adaptations in Table 1 . Curves: peak-normalized densities in log-error space; thick bars: interquartile ranges; thin lines: 5th–95th percentiles; ticks: medians; diamonds: arithmetic means.
Backbone
Execution
ORAD-3D
RELLIS-3D
ADE ↓
FDE ↓
P95FDE ↓
U/R ↑
ADE ↓
FDE ↓
P95FDE ↓
U/R ↑
Qwen3-VL-2B
Every update
0.384
0.726
1.736
1.0
0.387
0.820
1.933
1.0
Fixed interval
0.330
0.599
1.478
12.2
0.289
0.584
1.507
8.8
FastRoverVLA
0.327
0.593
1.445
17.2
0.266
0.526
1.375
21.0
MiniCPM-V-2.6
Every update
0.390
0.740
1.732
1.0
0.457
0.905
1.886
1.0
Fixed interval
0.327
0.595
1.478
12.2
0.292
0.574
1.454
8.8
Table 2: Recurrent execution improves all eight backbone–dataset pairs under shared frozen predictions. Errors: m ( ↓ ); U/R: updates per admitted plan ( ↑ ). Bold: within-backbone displayed optima, including ties.
Control
ORAD-3D
RELLIS-3D
ADE ↓
FDE ↓
RMSE ↓
P95FDE ↓
ADE ↓
FDE ↓
RMSE ↓
P95FDE ↓
Shared VLA-derived refresh times
FastRoverVLA
0.327
0.593
0.267
1.445
0.266
0.526
0.222
1.375
Direct stale reuse
0.500
0.906
0.404
2.260
0.471
0.901
0.387
2.646
No correction during reuse
0.507
0.924
0.411
2.330
0.455
0.876
0.375
2.511
No per-update alignment ( β=1 )
0.454
0.852
0.374
2.064
0.419
0.844
0.353
2.360
Table 3: State alignment and write-back improve reuse. Upper block: shared refresh times; the no-reuse-correction control also shares refresh-time processing. Lower block: no requests or scheduled refresh. Errors: m ( ↓ ); bold: displayed column minima, including ties.
Figure 5: Matched Qwen3-VL replay. (a,b) ADE densities use a shared within-dataset bandwidth and reflection at zero. (c,d) Mean FDE by reuse age; both policies have identical refresh-time trajectories, but the control omits subsequent correction. Shading marks ages beyond the six-action horizon.
Image input
All frames
Turning subset
ADE
FDE
ADE
FDE
ORAD-3D
True
0.385
0.727
0.468
1.014
Gray
0.384
0.726
0.464
1.005
Donor
0.385
0.728
0.465
1.008
RELLIS-3D
Table 4: Image-content controls at fresh-plan initialization. Turning subsets: Section 4 . Errors: m ( ↓ ); bold: dataset/group minima.
Setting
ORAD-3D
RELLIS-3D
ADE ↓
FDE ↓
P95FDE ↓
Rate (%) ↓
ADE ↓
FDE ↓
P95FDE ↓
Rate (%) ↓
Timing and cross-dataset rule transfer
Fixed interval
0.330
0.599
1.478
8.2
0.289
0.584
1.507
11.4
Uniform, matched budget
0.329
0.597
1.519
5.8
0.268
0.530
1.377
4.8
Local plan-conditioned rule
0.327
0.593
1.445
5.8
0.266
0.526
1.375
4.8
Transferred frozen rule
0.331
0.603
1.531
5.3
0.272
0.541
1.391
5.7
Table 5: Refresh timing, signals, and reuse caps in frozen-Qwen3 replay. R/C/Q: initial-action residual, output incompleteness, and plan roughness. Errors: m ( ↓ ); rate: admissions/update (%); bold: displayed block minima.
Figure 6: ORAD-3D trajectories from matched Qwen3-VL predictions. Camera and BEV views show six waypoints; curves interpolate them for display. Boxes: schematic ego poses; gray points: synchronized LiDAR. Camera projections use per-frame calibration and a 0.65 m display lift. Labels give every-update / FastRoverVLA FDE.
Dataset
Metric
Reduction
Interval (95%)
Improved
ORAD-3D
ADE
0.055
[0.030, 0.078]
10/11
FDE
0.125
[0.073, 0.175]
10/11
RELLIS-3D
ADE
0.111
[0.092, 0.138]
3/3
FDE
0.271
[0.239, 0.328]
3/3
Table 6: Sequence-balanced error reductions (every-update minus FastRoverVLA, m ↑ ). Intervals: 10,000 whole-sequence bootstrap resamples; improved: sequences with lower error.
Controlled execution
Archived heading
Past-only heading
ADE ↓
FDE ↓
ADE ↓
FDE ↓
Every update
0.387
0.820
0.438
0.903
Fixed interval
0.289
0.584
0.339
0.666
FastRoverVLA
0.266
0.526
0.316
0.606
No per-update alignment
0.419
0.844
0.449
0.883
No correction write-back
0.371
0.738
0.414
0.805
Table 7: Past-only heading preserves the execution effect on RELLIS-3D. Only the scoring frame changes. Errors: m ( ↓ ); bold: upper-panel column minima and lower-panel within-backbone minima.
Backbone
Execution
ORAD-3D
RELLIS-3D
ADE ↓
FDE ↓
Req. (%) ↓
C ↓
ADE ↓
FDE ↓
Req. (%) ↓
C ↓
Qwen3-VL-2B
Every update
0.385
0.727
100.0
355.4
0.391
0.831
100.0
443.9
Fixed interval
0.330
0.599
8.2
28.0
0.287
0.581
11.4
50.1
FastRoverVLA
0.328
0.596
6.0
21.4
0.266
0.526
4.8
21.3
MiniCPM-V-2.6
Every update
0.397
0.754
100.0
1336.1
0.364
0.753
100.0
1392.8
Fixed interval
0.332
0.604
8.2
108.1
0.280
0.558
11.4
157.3
Table 8: Independent real-request execution improves accuracy and reduces cost relative to every-update prediction. Errors: m ( ↓ ); Req.: requests/update (%); C: request-processing ms/update. Bold: within-backbone minima, including ties.
Autonomous driving develops vehicles that perceive road scenes, predict interactions, and plan and control safe movement with limited or no human input.