Catch Me If You Can: Real-Time Feedback Denoising for Responsive VLAs

Yiheng Ji; Xingru Zhou; Luis Sentis; Mingyo Seo

cs.ROSep 17, 2026

Catch Me If You Can: Real-Time Feedback Denoising for Responsive VLAs

Authors: Yiheng Ji, Xingru Zhou, Luis Sentis, Mingyo Seo

Organizations: The University of Texas at Austin · University of Central Florida

Abstract

Vision-Language-Action (VLA) models have shown strong generalization in robotic manipulation by combining semantic knowledge from pretrained vision-language models with expressive action-generation policies. Diffusion-based action generators are particularly effective for modeling temporally coherent action chunks, but these chunks are typically executed open-loop after inference. This limits responsiveness when objects move, contacts change, or the scene evolves during execution. We propose VLA-Feedback, a two-timescale architecture that combines low-frequency diffusion planning with high-frequency visual feedback. Rather than fully denoising an action chunk before execution, VLA-Feedback retains its final denoising step as a lightweight feedback interface, allowing each action to be corrected using the latest observation before it is executed. This design preserves the expressiveness of the diffusion planner while enabling real-time action correction without rerunning the full vision-language diffusion model. VLA-Feedback matched GR00T on static LIBERO tasks while improving average success on dynamic simulation tasks from 27.5% to 85.0%. On real-robot tasks, it improved average success from 51% to 73%. Additional materials can be found on our project page: https://vla-feedback.github.io.

Figures & tables

Figure 1: Overview of VLA-Feedback. Traditional VLAs (a) execute a generated action chunk open-loop and therefore cannot incorporate observations received during execution. VLA-Feedback (b) retains the VLM-DiT planner in the slow pathway and uses a lightweight Feedback Denoising Module to correct each action using the latest observation, enabling intra-chunk feedback without rerunning the planner.

Figure 2: Architecture of VLA-Feedback. A low-frequency VLM-DiT planner generates a near-final action chunk and caches its encoded action representation. At every control step, the Feedback Denoising Module fuses the cached action representation with the latest high-frequency observation, applies visual residual scaling, and predicts the final denoising velocity to refine the current action online.

Method	LIBERO	LIBERO	Pick up 1D	Pick up 2D	Drop Ball to	Catch	1D Robot	Drop Ball	Drop Ball
	Goal	Obj.	Robot	Robot	Cup	Block ∗	Speed ∗	Color ∗	Speed ∗
OpenVLA [ 21 ]	78	88.5	5	60	0	0	0	0	0
FiS-VLA [ 9 ]	41.5	53.5	0	67.5	80	0	0	60	0
GR00T [ 2 ]	92	97.5	47.5	20	15	60	0	2.5	0
VLA-Feedback	92	95.5	80	75	100	67.5	50	97.5	57.5

Table 1: Success rates on static LIBERO, dynamic tasks, and unseen tasks

Figure 3: Simulation tasks. ( a ) Example tasks from the LIBERO benchmark: the top two were from LIBERO-Goal, and the bottom two were from LIBERO-Object. ( b ) Time-lapse rollouts of VLA-Feedback on three dynamic tasks: “Pick up the red robot toy,” “Pick up the green robot toy,” and “Drop the ball into the cup.”

Method	S:F:A	$T_{s}$	$T_{f}$	$R$	$\bar{T}_{\mathrm{react}}$
OpenVLA [ 21 ]	$1{:}0{:}1$	160 ms	–	1	$160+\frac{1}{2}\Delta t$
GR00T [ 2 ]	$1{:}0{:}16$	80 ms	–	16	$80+8\Delta t$
FiS-VLA [ 9 ]	$1{:}4{:}4$	73 ms	40 ms	1	$40+\frac{1}{2}\Delta t$
VLA-Feedback	$1{:}16{:}16$	79 ms	2 ms	1	$2+\frac{1}{2}\Delta t$

Table 2: Timing and reaction latency of VLA policies. S:F:A denotes slow-planner calls, fast-module updates, and executed actions per chunk.

Figure 4: Real-robot experiments on Panda. ( Left ) execution timelines for three tasks: (a) “Pick up the bread,” (b) “Drop the lemonade into the cup,” and (c) “Catch the rolling can.” ( Right ) success rates across the three tasks.

Panel of Figure 5 — Figure 5: Ablation study of VLA-Feedback. ( a ) Ablations compared VLA-Feedback with direct action residual correction, no visual feedback, and replacement of the Transformer fusion block with an MLP. Removing visual feedback or using direct action residual correction most strongly degraded dynamic-task performance, while static performance remained similar. ( b ) Reducing the feedback update frequency lowered dynamic-task success rate, showing that frequent intra-chunk feedback was important for non-stationary manipulation.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Speed	GR00T	VLA-Feedback
Original	15%	100%
$+10\%$	5%	92.5%
$+20\%$	5%	85%
$+30\%$	0%	57.5%
$+40\%$	0%	5%

Appendix

Table 3: Drop-ball success rate versus speed.

Method	Backbone / Head	Obs.	Horizon	Schedule	Training / Notes
OpenVLA [ 21 ]	Transformer VLA action head	Agent	1	$1{:}0{:}1$	Released finetuning; agent-view setup.
GR00T [ 2 ]	VLM-DiT flow-matching head	Agent + hand	16	$1{:}0{:}16$	Released finetuning; open-loop chunks.
FiS-VLA [ 9 ]	Slow–fast VLA pathways	Agent + hand	1	$1{:}4{:}4$	Frozen vision backbone; LoRA language finetuning; released default horizon.
VLA-Feedback	VLM-DiT + feedback denoising	Agent + hand	16	$1{:}16{:}16$	Same planner as GR00T; feedback final denoising.

Appendix

Table 4: Baseline implementation details. Schedule denotes slow-planner calls, fast-module updates, and executed actions per chunk.

Component	Simulation	Hardware
Camera capture/exposure	n/a	62.00
IPC	0.03	4.00
Preprocessing	0.06	0.06
Feedback inference	1.95	1.95
Robot communication	n/a	2.00
Actuation	n/a	1.00

Appendix

Table 5: Feedback-loop latency in simulation and hardware. All values are in milliseconds.

Closed-Loop Control of VLA Models

See all 15 papers

Reducing Temporal Redundancy for Efficient Vision-Language-Action Inference
Jul 14, 2026Yuzhou Wu, Yuxin Zheng, Muchun Niu +6Efficient VLA Model Inference Efficient VLA Models
1Tianji KernalMind co ltd · 2Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · 3Shanghai Jiao Tong University, Shanghai, China +2
ReactVLA: Fast and Lightweight Reactive Robot Manipulation via Improved Mean Flow Action Generation
Jun 12, 2026Yanzhao Guo, Wenkai Chen, Jianwei ZhangLanguage-Conditioned Robot Manipulation Efficient VLM Inference
Shanghai Jiao Tong University · Technical Aspects of Multimodal Systems (TAMS), Department of Informatics, Universität Hamburg
TokenPB: Enabling VLA Action Self-Verification via VLM Token Probability Bucketing
Mar 18, 2026Chen Zhao, Zhuoran Wang, Haoyang Li +6Vision-Language-Action Models VLA Model Training
School of Information, Renmin University of China · Key Laboratory of Data Engineering and Knowledge Engineering, Beijing, China · Beijing University of Posts and Telecommunications +1

Jul 14, 2026J/K move · Enter open · S save

Reducing Temporal Redundancy for Efficient Vision-Language-Action Inference

Yuzhou Wu, Yuxin Zheng, Muchun Niu, Yishan Yang+5

1Tianji KernalMind co ltd · 2Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · 3Shanghai Jiao Tong University, Shanghai, China+2

Efficient VLA Model Inference Efficient VLA Models Robotic Manipulation Vision-Language-Action Models Diffusion Model Inference Acceleration

Open paper AskPDF arXiv

Vision-Language-Action (VLA) models exhibit strong generalization for robotic manipulation, yet their high inference latency limits real time deployment. We identify two primary sources of temporal redundancy in existing VLA pipelines: repeated visual encoding of highly similar consecutive frames and multi step iterative sampling in diffusion based policies. To address this, we propose a system level acceleration strategy that reduces computation in both perception and action generation. On the perception side, we incrementally update only tokens corresponding to dynamic scene regions instead of re-encoding entire frames. On the policy side, we compress diffusion sampling into a compact 2-step schedule through efficiency oriented training while preserving action precision. Experiments on Libero, RobotWin, and Real Robot Platforms demonstrate over 2 times speedup while maintaining high performance, achieving up to 98% success rate on general manipulation benchmarks. Our codes will be released on Github.

Abstract

Closed-Loop Control of VLA Models

Explore similar work

Reducing Temporal Redundancy for Efficient Vision-Language-Action Inference

ReactVLA: Fast and Lightweight Reactive Robot Manipulation via Improved Mean Flow Action Generation

TokenPB: Enabling VLA Action Self-Verification via VLM Token Probability Bucketing