T3R: Training-Free Two-Stage Token Refinement Towards Efficient and Robust VLA Models

Mohammed Hassan; Zhenyang Chen; Zheng Wang; Zhixin Zhu; Tianlong Chen; Yingyan; Lin; Chaojian Li

cs.RONov 5, 2026

T3R: Training-Free Two-Stage Token Refinement Towards Efficient and Robust VLA Models

Authors: Mohammed Hassan, Zhenyang Chen, Zheng Wang, Zhixin Zhu, Tianlong Chen, Yingyan, Lin, Chaojian Li

Organizations: Celine · Georgia Institute of Technology, Atlanta, Georgia, United States · Hong Kong University of Science and Technology, Hong Kong, Hong Kong

Abstract

Vision-Language-Action (VLA) models enable end-to-end robot control, but inference is increasingly expensive. Each decision processes hundreds of visual tokens, often across multiple cameras, leading to substantial computation while allowing irrelevant background content to distract action prediction under distribution shift. We argue that these issues stem from two complementary forms of redundancy: many visual tokens are spatially irrelevant, while attention over the remaining tokens is often misallocated. Based on this observation, we propose T3R, a Training-Free Two-Stage Token Refinement framework. In the first stage, segmentation-guided token pruning removes irrelevant visual patches to reduce computation and eliminate distracting context. In the second stage, saliency-guided attention biasing reallocates attention toward action-critical visual regions and language instructions within the retained tokens. Together, pruning produces a cleaner candidate set, while attention biasing further emphasizes the most decision-relevant evidence, yielding a better accuracy-efficiency frontier than either stage alone. On LIBERO-Spatial, T3R removes 85% of scene-camera visual tokens while essentially matching the unpruned success rate, and improves success by up to 7 percentage points under out-of-distribution object-placement perturbations. The timing comparison attributes 7.8 ms per inference call to pruning and attention-bias application. As input resolution increases, LLM-forward profiling shows up to a 3.3x speedup over the unpruned baseline. Evaluations across LIBERO, SIMPLER, and RoboTwin with three VLA backbones show the strongest accuracy-efficiency benefits in multi-camera settings. These results demonstrate that training-free token refinement is a practical approach to efficient and robust VLA inference.

Figures & tables

Figure 1. Two-stage T3R pipeline. Stage 1 (pruning): POS-weighted SigLIP text–patch similarity localizes the target; the top-scoring patch positions (with SigLIP/DINOv2 features) prompt EfficientTAM to produce an object mask, which is converted to a patch-level pruning decision while preserving original RoPE positions, so only the “interested tokens” enter the LLM. Stage 2 (biasing): Integrated Gradients over the instruction identifies salient words; an additive attention bias ( $\beta{=}2.0$ ) is injected into the middle decoder layers for action-token queries, steering attention toward salient text tokens and retained visual patches before the action chunk is decoded. A closed-loop robot-control diagram. Camera observations and the instruction feed image and text encoders. In the first stage, similarity-selected image patches and DINOv2 features guide EfficientTAM to generate a mask; only selected visual tokens enter the language model. In the second stage, Integrated Gradients assigns importance to instruction words, illustrated by bowl and plate, and attention biases emphasize those words and selected visual tokens. The model outputs action tokens, the robot executes the actions, and a new observation starts the next iteration.

Figure 2. Qualitative pruning on a sparse LIBERO-Spatial scene (“pick up the bowl between the plate and the ramekin”). Top: input, POS-weighted SigLIP similarity, and the EfficientTAM mask with its point prompts. Middle: T3R keeps only 13/256 patches (95% pruned), tightly localized on the target. Bottom: TeamVLA’s similarity+expansion+merge retains 226/256 (12% pruned), leaving most of the background in the sequence through the early decoder layers. A three-by-three panel comparison of patch selection in a tabletop scene containing a robot gripper, bowls, a plate, and a ramekin. The top row shows the original image, a colored similarity heatmap, and a green segmentation mask with positive and negative point prompts. The middle row shows a small retained region around the target, the image with discarded regions darkened, and the original image for comparison. The bottom row shows TeamVLA's similarity, expansion, and final merging masks, which retain much more background. Green patches are retained and red patches are discarded.

Figure 3. Most visual tokens are near-inert. (a) Across all 32 decoder layers, 93.9% of visual tokens receive $<$ 0.2% of the action queries’ attention each. (b) Per-patch action-query attention concentrates on a few task-relevant patches (here the gripper and target bowls), while the background receives negligible weight. Two panels summarize visual-token attention. The left bar chart shows that 30.7 percent of tokens receive less than 0.1 percent attention each, 63.2 percent receive 0.1 to 0.2 percent, and 6.1 percent receive more than 0.2 percent. The right panel overlays a patch-grid heatmap on the scene. Most patches have low attention, with a few high-attention regions near the gripper and target bowls. The color scale runs from blue for low attention to red for high attention.

Method	Reduction	SR (%)	Latency (ms)
Baseline (no pruning)	0%	97	107.5
TeamVLA ( Ye et al., 2025 )	$\sim$ 15%	94	102.6
FastV ( Chen et al., 2024 ) ( $k{=}1$ , $r{=}0.85$ )	85%	14	–
ADP ( Pei et al., 2025 )	85%	66	–
ADP ( Pei et al., 2025 ) ∗	33.5%	97	–
T3R (ours)	85%	96	92.5

Table 1. LIBERO-Spatial (OpenVLA-OFT). “Reduction” is scene-camera tokens pruned before decoder execution; latency is per VLA inference call. ∗ ADP native operating point (67% of steps

\times

50%/step

\approx

33.5% reduction).

Figure 4. OOD robustness under object-placement perturbation (21 noise levels, 10 episodes/task). T3R (Full Pipeline) tracks the unpruned baseline for small shifts and pulls ahead beyond

{\sim}2

cm, leading by up to 7 pp at 4 cm, while TeamVLA degrades fastest. The crossover is consistent with pruning reducing misleading background context. A line chart plots success rate against placement perturbation from zero to five centimeters. Blue circles represent the unpruned baseline, green squares represent the full T3R pipeline, and red triangles represent TeamVLA. The curves begin close together at high success rates and generally decline as perturbation increases. Beyond about two centimeters, the green curve is above or equal to the blue curve, while the red curve declines most strongly.

Method (keep ratio)	Google Robot	WidowX/Bridge
Baseline (100%)	73.2	10.4
ADP ( Pei et al., 2025 ) (75% / $\sim$ 87%)	71.0	16.7
FastV ( Chen et al., 2024 ) ( $\sim$ 60%)	73.3	15.6
TeamVLA ( Ye et al., 2025 ) (31.25%)	69.5	12.5
T3R (ours, 60%)	67.7	10.4

Table 2. Cross-backbone generalization. Left: CogACT on SIMPLER (single camera), average SR (%); each keep ratio corresponds to the respective method’s native operating point. Right:

\pi_{0.5}

on RoboTwin (three cameras), SR (%) vs. per-camera token budget (base

=33.3

). Best per column/row in bold.

Efficient VLA Model Inference

Efficient vision-language-action (VLA) model inference reduces robot-control latency, memory use, and energy while preserving task performance.

See all 205 papers

TRIAGE: Efficient Vision-Language-Action Inference Needs Both Reuse and Pruning
Oct 25, 2026Sooyoung Kwon, Carlee Joe-Wong, Kyunghan LeeEfficient VLA Model Inference Visual Token Pruning
Dept. of ECE, Seoul National University. · Dept. of ECE, Carnegie Mellon University.
When and What to Prune? Stage-Aware Visual Token Pruning for Efficient VLA
Oct 4, 2026Tianjun Shi, Haotian Xiong, Ziyu Gong +2Efficient VLA Model Inference Vision-Language-Action Models
University of Macau
2D or 3D: Who Governs Salience in VLA Models? -- Tri-Stage Token Pruning Framework with Modality Salience Awareness
Apr 10, 2026Zihao Zheng, Sicheng Tian, Zhihao Mao +8Efficient VLA Model Inference Vision-Language-Action Models
School of Computer Science, Peking University · School of Artificial Intelligence, Beijing Normal University · School of Computer Science, China University of Geosciences (Wuhan) +2

Oct 25, 2026J/K move · Enter open · S save

TRIAGE: Efficient Vision-Language-Action Inference Needs Both Reuse and Pruning

Sooyoung Kwon, Carlee Joe-Wong, Kyunghan Lee

Dept. of ECE, Seoul National University. · Dept. of ECE, Carnegie Mellon University.

Efficient VLA Model Inference Visual Token Pruning Robotic Manipulation

Open paper AskPDF arXiv

Vision-language-action (VLA) models are increasingly used for robot manipulation because they map camera images and a language instruction directly to actions. Robots that cannot be tethered to a workstation, such as mobile manipulators and humanoids, must run them on onboard embedded boards, where the prefill latency of the language backbone grows with the hundreds of visual tokens processed at every control step. To meet real-time limits, training-free accelerators recompute only a subset of the visual tokens and skip the rest, either by pruning them or by reusing their cached key-value (KV) states. The two options save nearly the same computation, yet existing methods apply only one of them to every skipped token. Through a token-level diagnostic, we show that which option keeps the action closer to full recomputation depends on the token's drift, the change in its shallow-layer value since its cache was computed. On both backbones tested, reusing tokens below a drift threshold was no worse than pruning them, whereas beyond it, reuse yields up to 4.4x the action error of pruning; the same diagnostic calibrates this threshold per backbone. Building on this finding, we propose TRIAGE, which deems a cache valid when its drift is below this threshold and it is not too old. TRIAGE recomputes a fixed budget of tokens, prioritizing important tokens with invalid caches, reuses every other token with a valid cache, and prunes the rest; prune-only and reuse-only methods are the two extremes of this partition. Because the budget fixes the number of recomputed tokens, it directly sets the prefill latency. On LIBERO and SimplerEnv, TRIAGE stays within 3 pp of full recomputation in average success rate while cutting prefill latency on a Jetson AGX Orin by up to 48%, and with only 7% of visual tokens recomputed, it beats the strongest baseline by 24.4 pp on LIBERO-Object.

Abstract

Efficient VLA Model Inference

Explore similar work

TRIAGE: Efficient Vision-Language-Action Inference Needs Both Reuse and Pruning

When and What to Prune? Stage-Aware Visual Token Pruning for Efficient VLA

2D or 3D: Who Governs Salience in VLA Models? -- Tri-Stage Token Pruning Framework with Modality Salience Awareness