cs.ROOct 25, 2026

TRIAGE: Efficient Vision-Language-Action Inference Needs Both Reuse and Pruning

Authors: Sooyoung Kwon, Carlee Joe-Wong, Kyunghan Lee

Organizations: Dept. of ECE, Seoul National University. · Dept. of ECE, Carnegie Mellon University.

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Vision-language-action (VLA) models are increasingly used for robot manipulation because they map camera images and a language instruction directly to actions. Robots that cannot be tethered to a workstation, such as mobile manipulators and humanoids, must run them on onboard embedded boards, where the prefill latency of the language backbone grows with the hundreds of visual tokens processed at every control step. To meet real-time limits, training-free accelerators recompute only a subset of the visual tokens and skip the rest, either by pruning them or by reusing their cached key-value (KV) states. The two options save nearly the same computation, yet existing methods apply only one of them to every skipped token. Through a token-level diagnostic, we show that which option keeps the action closer to full recomputation depends on the token's drift, the change in its shallow-layer value since its cache was computed. On both backbones tested, reusing tokens below a drift threshold was no worse than pruning them, whereas beyond it, reuse yields up to 4.4x the action error of pruning; the same diagnostic calibrates this threshold per backbone. Building on this finding, we propose TRIAGE, which deems a cache valid when its drift is below this threshold and it is not too old. TRIAGE recomputes a fixed budget of tokens, prioritizing important tokens with invalid caches, reuses every other token with a valid cache, and prunes the rest; prune-only and reuse-only methods are the two extremes of this partition. Because the budget fixes the number of recomputed tokens, it directly sets the prefill latency. On LIBERO and SimplerEnv, TRIAGE stays within 3 pp of full recomputation in average success rate while cutting prefill latency on a Jetson AGX Orin by up to 48%, and with only 7% of visual tokens recomputed, it beats the strongest baseline by 24.4 pp on LIBERO-Object.

Figures & tables

Fig. 1: Recompute only what neither reuse nor pruning can handle. Blue: important tokens; red: tokens changed since frame t−1t-1 (stale caches). Reuse-only recomputes every changed token (18) and prune-only every important token (17). TRIAGE recomputes only tokens that are both (7), reuses valid caches, and prunes the rest.
Fig. 2: Drift decides between reuse and pruning. (a, b) Reuse −- prune action error Δ\Delta (Eq. ( 2 ), ×10−3\times 10^{-3} ) versus drift of the probed tokens on LIBERO-Spatial; blue region: reuse is better. Black: pooled over cache ages (95% bootstrap CI); thin: individual ages. Reuse is never worse below δ=0.3\delta=0.3 (dashed) but clearly worse above 0.5. (c) Drift grows with cache age (median, solid; 90th percentile, dashed), so the age limit TT rarely binds.
Fig. 3: Overview of TRIAGE. Shallow layers mark each token as important (blue) and its cache as invalid (red). The budget BB goes to important, invalid tokens first; every other token is reused if valid and pruned otherwise, so BB alone sets the prefill latency.
Fig. 4: Only combining pruning and reuse survives small budgets. LIBERO-Object, OpenVLA (500 episodes per budget; Wilson 95% intervals). x-axis: visual tokens recomputed per query, out of 256. Drop-all / keep-all prune / reuse every non-recomputed token. Drop-all degrades steadily and keep-all collapses below B=46B=46 , while TRIAGE leads VLA-Pruner by 24.4 pp at B=18B=18 .
Fig. 5: TRIAGE lies on the Pareto frontier in every setting. Success rate versus prefill latency on a Jetson AGX Orin (top) and an RTX 3090 (bottom), averaged over the LIBERO suites or the SimplerEnv settings; dashed: Pareto frontier. No baseline is both more accurate and faster.
       Latency (ms)
       Jetson AGX OrinRTX 3090
MethodSpatialObjectGoalLongAvgFLOPsPrefillE2Ep99PrefillE2Ep99
OpenVLA
Full recomputation85.086.079.256.076.51.00 ×\times229.51261.31307.1112.9281.4286.3
VLA-Cache [ 14 ]84.887.278.250.475.20.78 ×\times211.31365.71415.387.7278.7289.5
VLA-Pruner [ 10 ]84.483.676.249.273.30.39 ×\times204.51368.71428.551.2246.1315.9
TABLE I: Results on LIBERO: TRIAGE stays within 3 pp of full recomputation with the lowest Jetson p99 latency
 Move NearCoke Can Latency (ms)
     Jetson AGX OrinRTX 3090
MethodVMVAVAAvgPrefillE2Ep99PrefillE2Ep99
Full recomputation50.060.453.954.8228.21230.61263.4112.3275.4278.7
VLA-Pruner [ 10 ]40.853.850.548.4208.61385.01447.252.7242.6312.2
SpecPrune-VLA ‡ [ 11 ]45.456.748.850.3175.01189.21254.946.6236.0305.7
TRIAGE (ours)50.461.251.954.5169.01179.41224.248.3238.6279.5
TABLE II: OpenVLA on SimplerEnv (Google Robot): without retuning, TRIAGE matches full recomputation
Fig. 6: TRIAGE is robust to its hyperparameters. LIBERO-Goal, OpenVLA, B=28B=28 (default: 70.6%). Each row changes one setting from the default ( δ=0.3\delta=0.3 , T=10T=10 , μ=0.06\mu=0.06 , signal layer 2); paired difference with 95% bootstrap interval over 500 episodes. Every interval contains zero.
DriftAge 1Age 5Age 10Age 20
Layer 15
[0,0.1)[0,0.1)0.040.030.020.03
[0.1,0.2)[0.1,0.2)0.060.100.090.09
[0.2,0.3)[0.2,0.3)0.110.140.190.21
Layer 31
[0,0.1)[0,0.1)0.080.080.060.07
TABLE III: Error of reused caches grows with age only just below δ\delta (OpenVLA, median relative change in attention)
More in this topic

Efficient VLA Model Inference

Efficient vision-language-action (VLA) model inference reduces robot-control latency, memory use, and energy while preserving task performance.

See all 190 papers

Explore similar work

CardsList
Oct 6, 2026

VLA-ACL: Action-Consistent Visual Token Pruning for Efficient Vision-Language-Action Models

Vision-Language-Action (VLA) models achieve strong robotic manipulation performance but incur high computational costs from processing long token sequences at every control step, limiting real-time deployment. Visual token pruning offers a direct solution, as visual patches dominate the input sequence and contain considerable redundancy. Existing approaches, however, either rely on indirect training-free heuristics, such as attention scores and motion thresholds, or require costly fine-tuning of the base VLA model. We introduce VLA-ACL (Action Consistency Learning), which learns a lightweight visual token pruning policy through action-level supervision while keeping the base VLA model entirely frozen. The training objective encourages actions produced from pruned visual contexts to remain consistent with the full-context teacher, with ground-truth actions as auxiliary supervision. This directly ties token selection to its effect on the downstream control output. Experiments on LIBERO and real-world manipulation tasks show that VLA-ACL prunes up to 87.5% of visual tokens while retaining competitive performance, reduces computation by up to 75%, and achieves a 1.5x inference speedup. These results establish a stronger performance-efficiency trade-off than existing frozen-VLA pruning methods and demonstrate the value of action-level supervision for visual token selection. Code is available at https://github.com/du-owen/VLA-ACL.

Owen Du, Yang Yue, Jie Zhang +3

ETH Zurich · Tsinghua University

Efficient VLA ModelsVLMs for RoboticsVisual Token Pruning
AskPDF
Oct 15, 2026

FastVLA: Task-Conditioned and Temporally Adaptive Visual Token Compression for Efficient Vision-Language-Action Models

Vision-Language-Action (VLA) models present a promising paradigm for general-purpose robot manipulation, but closed-loop deployment is often limited by inference efficiency: at each timestep, high-resolution, often multi-view images are encoded into hundreds of visual tokens that dominate transformer computation and latency. Token compression via attention is a natural solution but is challenging for VLA control because (1) decision-critical tokens are determined by the task, not generic visual saliency, and (2) the token budget varies substantially across steps, so static metrics and budgets waste computation or discard control-critical tokens. We introduce FastVLA, a training-free, task-aware, and temporally adaptive visual token compression framework. At each step, FastVLA scores token importance via task-to-vision attention and adaptively sets the compression rate from the importance-distribution uniformity, measured by an inverse-Simpson metric with online sliding-window calibration. We evaluate FastVLA on OpenVLA, OpenVLA-OFT, and CogACT across LIBERO and SIMPLER, and on a physical Franka Research 3 robot. In simulation, it reduces LLM-backbone FLOPs by 35-59% and action latency by 9-32% (a 1.09×\times-1.46×\times speedup) while maintaining task success, achieving a comparable or better efficiency-performance trade-off than existing plug-and-play token compression/pruning/reuse baselines. On the real robot, it cuts action-prediction latency by 20% (1.25×\times speedup) at comparable success. Code is available at https://github.com/jiuyixu25/FastVLA.

Jiuyi Xu, Chengyuan Qian, Zhengzhong Tu +2

Colorado School of Mines · Texas A&M University · Microsoft AI

Efficient VLA Model InferenceVisual Token CompressionVision-Language-Action ModelsVLMs for Robotics
AskPDF
Jul 7, 2025

VOTE: Vision-Language-Action Optimization with Trajectory Ensemble Voting

Recent large-scale Vision Language Action (VLA) models have shown superior performance in robotic manipulation tasks guided by natural language. However, current VLA models suffer from two drawbacks: (i) generation of massive tokens leading to high inference latency and increased training cost, and (ii) insufficient utilization of generated actions resulting in potential performance loss. To address these issues, we develop a training framework to finetune VLA models for generating significantly fewer action tokens with high parallelism, effectively reducing inference latency and training cost. Furthermore, we introduce an inference optimization technique with a novel voting-based ensemble strategy to combine current and previous action predictions, improving the utilization of generated actions and overall performance. Our results demonstrate that we achieve superior performance compared with state-of-the-art VLA models, achieving significantly higher success rates and 39×\times faster inference than OpenVLA with 46 Hz throughput on edge platforms, demonstrating practical deployability. The code is available at https://github.com/LukeLIN-web/VOTE.

Juyi Lin, Amir Taherin, Arash Akbari +11

Northeastern University, Boston, USA · EmbodyX,San Mateo,USA

Robotic ControlEfficient VLA Model InferenceEfficient VLA ModelsVision-Language-Action ModelsVLMs for RoboticsVLA Model Training
AskPDF