NPU Offloading of a Frozen Visual Encoder for Robot Policy Training
Authors: Hyojun Yun, Seungjae Won, Hyungpil Moon
Organizations: Department of Mechanical Engineering, Sungkyunkwan University, Suwon, Korea · Department of Intelligent Robotics, Sungkyunkwan University, Suwon, Korea
When a robot policy is trained for a new task or dataset, its visual encoder can be frozen and only its action generation module trained, reducing training cost. Freezing removes the encoder's backward pass, but its forward pass must still run at every training step because the input images change, so it keeps consuming GPU compute. We therefore ask whether moving this computation to a low power AI accelerator such as an NPU can reduce total energy despite the added data transfer and longer training time, and how it affects policy performance. We built an asynchronous training pipeline that uses both a GPU and an NPU for the AR-Actor specialist. The frozen visual encoder runs in A8W8 INT8 on a Mobilint Aries2 NPU, while the FP32 action expert is trained on an NVIDIA GeForce RTX 5060 Ti GPU. We compared a GPU-only baseline with four conditions, L1 to L4, which gradually extend NPU offloading from one to four Transformer encoder layers. Each condition was trained for 30,000 steps with three random seeds. We measured GPU board power for the GPU-only condition and combined GPU and NPU board power for the NPU conditions. Energy per sample decreased by 17.1% in L1, which offloaded ResNet18 and the first encoder layer, and by 27.9% in L4, which offloaded ResNet18 and all four encoder layers. In contrast, training time per sample increased by 15.2% in L1 and 37.7% in L4, and peak allocated GPU memory decreased by 19.8 to 20.7%. The 15 resulting policies were each evaluated with the same 300 environment seeds, for a total of 4,500 simulator rollouts. The combined success rate was 93.33% for GPU-only and 91.44 to 92.89% for the NPU conditions. These results show that NPU offloading of a frozen visual encoder can reduce training energy, but it increases training time and lowers policy success rate by 0.44 to 1.89 percentage points compared with GPU-only training.
Figures & tables
Part
Components
Number of parameters
State
Visual encoder
ResNet18, image projection, four Transformer encoder layers
28,771,520
Frozen
Action expert
State projection, four autoregressive Transformer decoder layers, one parallel decoder layer, and action head
22,743,708
Newly initialized and trained
TABLE I : PARAMETER COUNT AND TRAINING STATE OF EACH POLICY MODULE
Condition
Runs on NPU
Frozen computation remaining on GPU
Trained on GPU
GPU-only
None
Entire visual encoder
Action expert
L1
ResNet18 + projection + encoder 0
Encoders 1–3 + K/V reconstruction
Action expert
L2
L1 range + encoder 1
Encoders 2–3 + K/V reconstruction
Action expert
L3
L2 range + encoder 2
Encoder 3 + K/V reconstruction
Action expert
L4
L3 range + encoder 3
K/V reconstruction + final normalization
Action expert
TABLE II : OFFLOADING BOUNDARIES AND THE COMPUTATION THAT REMAINS ON THE GPU
Fig. 1 : Time breakdown of asynchronous GPU and NPU overlap by condition. While the GPU processes batch N, the NPU runs the frozen visual encoder forward pass for batch N+1 in advance.
Item
Setting
CPU
Intel Core i5-10400 @ 2.90 GHz
GPU
NVIDIA GeForce RTX 5060 Ti 16 GB, driver 580.173.02
TABLE III : HARDWARE, SOFTWARE, AND PRECISION SETTINGS
Mean GPU
Mean NPU
Mean combined
Total training
Energy per
vs.
Time per
Throughput
Condition
board power
board power
board power
time
sample
GPU-only
sample
(W)
(W)
(W)
(min)
(J)
(%)
(ms)
(sample/s)
GPU-only
138.59
—
138.59
26.43
0.91512
—
6.607
151.4
L1
80.85
18.81
99.66
30.46
0.75831
−17.1
7.613
131.3
L2
70.67
18.88
89.55
32.94
0.73671
−19.5
8.224
121.6
L3
62.84
19.08
81.92
34.31
0.70244
−23.2
8.578
116.6
TABLE IV : MEAN BOARD POWER, ENERGY, AND TRAINING TIME
Fig. 2 : Accelerator board energy per sample by offloading boundary. Gray lines connect the conditions measured with the same training seed. Colored points show the mean across the three training seeds.
Fig. 3 : Peak allocated GPU memory by condition. Each bar shows the mean across three training seeds.
Fig. 4 : Simulator policy success rate by condition. Large colored points and the dark line show the success rate over 900 evaluations for each condition. Small open points and gray lines connect checkpoint results from the same training seed. The training seed is shown at the right end of each gray line. Values in parentheses show the difference from GPU-only.