RawVLA: Embodied Neural Image Signal Processor For Robotic Manipulation

Shuhong Liu; Heng Zhou; Lingfeng Qian; Yuhao Fang; Xianbao Hou; Qianyu Zhou; Lin Gu; Wei Sui

cs.ROSep 29, 2026

RawVLA: Embodied Neural Image Signal Processor For Robotic Manipulation

Authors: Shuhong Liu, Heng Zhou, Lingfeng Qian, Yuhao Fang, Xianbao Hou, Qianyu Zhou, Lin Gu, Wei Sui, +2 more

Organizations: UTokyo · D-Robotics · TohokuU · NTU · HKUSTGZ

Abstract

Vision-language-action (VLA) models typically operate on RGB images produced by a fixed camera image signal processor (ISP), leaving the imaging pipeline outside the learning and evaluation loop. We systematically examine the consequences of this overlooked design choice across five fundamental ISP dimensions: gain, sensor noise, chromatic response, tonal response, and bit depth. Our analysis reveals that RAW-to-RGB processing materially shapes both action prediction and manipulation success, with different ISP dimensions exerting substantially different effects. Guided by these findings, we introduce RawVLA, a streaming neural ISP that adaptively renders RAW observations for frozen VLA policies while concentrating its capacity on the imaging factors relevant to embodied behavior. We further present RawVLA-Bench, a RAW-domain manipulation benchmark to expose image processing as an explicit evaluation variable across clean and adverse acquisition conditions. Experiments on RawVLA-Bench show that RawVLA preserves performance under standard conditions while substantially improving robustness under degraded imaging, establishing adaptive RAW processing as an effective interface between physical cameras and embodied policies.

Figures & tables

Figure 1: We propose RawVLA, an embodied and adaptive neural ISP module for VLA models, and RawVLA-Bench, a RAW-domain manipulation benchmark based on LIBERO and RoboTwin 2.0.

Figure 2: ISP sensitivity across simulation benchmarks. The left panel shows LIBERO results for fine-tuned VLA and WAM policies provided by StarVLA ( StarVLA Community, 2026 ) . The right panel shows Qwen3-OFT and FastWAM fine-tuned on RoboTwin 2.0 Easy and Hard (Hard adds randomized clutter, lighting, textures, and tabletop heights), which moderately improves robustness.

Figure 3: Overview of RawVLA. A causal linear RAW burst is summarized by factorized luminance and chroma descriptors, fused with previous ISP parameters through decoupled recurrent states, and decoded into structured exposure, white-balance, color-correction, and monotonic tone controls. The rendered image is passed to a frozen VLA policy for action prediction.

Figure 4: Qualitative results in simulation and the real world. The first and second rows compare agent-view observations rendered by the Default ISP and RawVLA on LIBERO and RoboTwin 2.0, respectively. The final row shows a real-world task completed under low-light conditions.

Illumin.	Backbone	Default			DarkISP			RAM			RAWAdapter			RAWild			Ours
Illumin.	Backbone	SOG	Long	Avg	SOG	Long	Avg	SOG	Long	Avg	SOG	Long	Avg	SOG	Long	Avg	SOG	Long	Avg
Extreme Lowlight	PI-0	7.87 7.87	3.20 3.20	6.70 6.70	1.93	0.00	1.45	1.80	0.00	1.35	2.80	0.00	2.10	2.87 2.87	0.00	2.15 2.15	46.00 46.00	24.80 24.80	40.70 40.70
	PI-0.5	2.60	6.60	3.60	5.80 5.80	9.00 9.00	6.60 6.60	1.73	0.00	1.30	0.07	0.00	0.05	3.93 3.93	8.00 8.00	4.95 4.95	41.63 41.63	30.00 30.00	38.72 38.72
	Qwen3-OFT	0.00	0.00	0.00	4.13 4.13	2.40 2.40	3.70 3.70	0.00	0.00	0.00	0.00	0.00	0.00	0.87 0.87	2.60 2.60	1.30 1.30	15.20 15.20	10.60 10.60	14.05 14.05
Lowlight	PI-0	68.67 68.67	44.60 44.60	62.65 62.65	2.67	0.00	2.00	1.80	0.00	1.35	2.60	0.00	1.95	6.67 6.67	0.40 0.40	5.10 5.10	90.47 90.47	66.00 66.00	84.35 84.35
	PI-0.5	62.47 62.47	50.80 50.80	59.55 59.55	53.60	43.00	50.95	1.13	0.00	0.85	0.20	0.00	0.15	72.93 72.93	60.40 60.40	69.80 69.80	91.67 91.67	81.80 81.80	89.20 89.20

Table 1: Suite-resolved success rates (%) of ISP modules on LIBERO. SOG averages Spatial, Object, and Goal; Long denotes LIBERO-10; Avg averages all four suites.

Illumin.	Backbone	Default	DarkISP	RAM	R.Adp.	RAWild	Ours
Extreme Lowlight	PI-0	23.62 23.62	4.72	12.60 12.60	1.57	8.66	32.28 32.28
Extreme Lowlight	PI-0.5	17.32 17.32	1.57	3.15	0.00	5.51 5.51	25.98 25.98
Lowlight	PI-0	40.94 40.94	0.79	10.24 10.24	3.15	6.30	59.06 59.06
Lowlight	PI-0.5	51.18 51.18	0.00	2.36	0.00	4.72 4.72	63.78 63.78
Normal	PI-0	55.91 55.91	34.65	17.32	56.69 56.69	28.35	57.48 57.48
Normal	PI-0.5	70.87 70.87	35.43	20.47	41.73 41.73	40.94	67.72 67.72

Table 2: Success rates (%) on RoboTwin 2.0 across five illumination levels. Default denotes the default SAPIEN tone-mapped RGB baseline.

Illumin.	Backbone	Default	DarkISP	RAM	R.Adp.	RAWild	Ours
Extreme Lowlight	PI-0	23.62 23.62	4.72	12.60 12.60	1.57	8.66	32.28 32.28
Extreme Lowlight	PI-0.5	17.32 17.32	1.57	3.15	0.00	5.51 5.51	25.98 25.98
Lowlight	PI-0	40.94 40.94	0.79	10.24 10.24	3.15	6.30	59.06 59.06
Lowlight	PI-0.5	51.18 51.18	0.00	2.36	0.00	4.72 4.72	63.78 63.78
Normal	PI-0	55.91 55.91	34.65	17.32	56.69 56.69	28.35	57.48 57.48
Normal	PI-0.5	70.87 70.87	35.43	20.47	41.73 41.73	40.94	67.72 67.72

Table 2: Success rates (%) on RoboTwin 2.0 across five illumination levels. Default denotes the default SAPIEN tone-mapped RGB baseline.

Illumin.	Task	Default	DarkISP	Ours
Normal	pick & place	96.00 96.00	66.00 66.00	96.00 96.00
	drug drawer	92.00 92.00	52.00 52.00	94.00 94.00
	pick flower	72.00 72.00	44.00 44.00	66.00 66.00
	stack & cover	46.00 46.00	38.00 38.00	44.00 44.00
	Avg	76.50 76.50	50.00 50.00	75.00 75.00
Lowlight	pick & place	0.00 0.00	22.00 22.00	92.00 92.00

Table 3: Real-world success rates (%) on four tasks using PI-0.5. Default denotes the constant default ISP baseline.

Photometric descriptors
	LIBERO / Qwen3-OFT						RoboTwin 2.0 / PI-0.5
Variant	Ext.Low	Low	Normal	Over	Ext.Over	Avg.	Ext.Low	Low	Normal	Over	Ext.Over	Avg.
w/o spatial feature $g_{t}$	0.00	20.35	97.10 97.10	51.15	13.15	36.35	14.96	12.60	62.99 62.99	60.32	59.20	41.93
w/o luminance descriptor $h_{t}^{L}$	27.75 27.75	95.05 95.05	96.80	50.80	25.95	59.27	21.26	47.24	50.39	56.35	52.80	45.57
w/o chroma descriptor $h_{t}^{C}$	0.00	0.00	96.65	48.75	16.60	32.40	23.62 23.62	57.48 57.48	48.03	60.32	53.60	48.58
Recurrent estimator

Table 5: Ablations of RawVLA on LIBERO with Qwen3-OFT and RoboTwin 2.0 with PI-0.5. The two blocks isolate photometric descriptors and recurrent module structures.

Appendix figures & tables25 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: RawVLA-Bench observations on LIBERO ( Liu et al., 2023 ) across five ambient-light intensity levels ranging from extreme low light and low light to normal , overexposure , and extreme overexposure . Each level shows the original RGB reference, synthesized RAW observation, and its default-ISP rendering. Agent and wrist views use the same unprocessing procedure.

Figure 6: RawVLA-Bench observations on RoboTwin 2.0 ( Chen et al., 2026b ) across five ambient-light intensity levels ranging from extreme low light and low light to normal , overexposure , and extreme overexposure . Each level shows the original RGB reference, synthesized RAW observation, and its default-ISP rendering. Agent and wrist views use the same unprocessing procedure.

Figure 7: Synthetic sensor noise on LIBERO ( Liu et al., 2023 ) . The figure shows noisy RAW observations and their corresponding RGB renderings after ISP gain amplification and subsequent processing across progressively lower illumination levels. Noise becomes increasingly pronounced as the illumination decreases.

Figure 8: Burst-denoising comparison on agent-view observations from LIBERO ( Liu et al., 2023 ) (left) and RoboTwin 2.0 ( Chen et al., 2026b ) (right). The top row compares a single noisy frame, direct averaging of $K=6$ consecutive frames, and our frequency-domain burst denoising with $K=6$ . The bottom row shows the corresponding error maps relative to the noise-free clean image. Direct averaging removes substantial sensor noise but blurs moving arms and objects, whereas burst denoising better preserves dynamic scene content while suppressing noise.

Frozen VLA policy	$\lambda_{a}$
Qwen3-PI	0.0015
Qwen3-OFT	0.20
WM4A-Cosmos-GR00T	0.01
WM4A-Wan-OFT	0.02
PI0	0.15
PI0.5	0.10

Appendix

Table 6: Action-loss coefficients used to train RawVLA with each frozen VLA policy.

Figure 9: Processed agent-view and wrist-view observations from the LIBERO portion of RawVLA-Bench under low, normal, and overexposed illumination. The rows compare the Default ISP with neural ISP methods using Qwen3-OFT as the policy backbone.

Figure 10: Processed agent-view and wrist-view observations from the RoboTwin 2.0 portion of RawVLA-Bench under low, normal, and overexposed illumination. The rows compare the Default ISP with neural ISP methods using PI-0.5 as the policy backbone.

Figure 11: Representative agent-view and wrist-view observations from LIBERO under severe haze. Each triplet shows the original simulator RGB image, the hazy input generated with $\alpha_{\mathrm{h}}=0.95$ and $I_{\mathrm{air}}=235$ , and the corresponding RGB output produced by RawVLA. The same enhanced observations are provided to the frozen PI-0 policy during closed-loop evaluation.

Method	Spatial	Object	Goal	LIBERO-10	Avg.
PI-0	78.0	84.0	82.0	22.0	66.5
PI-0 with RawVLA	98.0 98.0	96.0 96.0	96.0 96.0	63.6 63.6	88.4 88.4
Improvement (pp)	+20.0	+12.0	+14.0	+41.6	+21.9

Appendix

Table 7: PI-0 success rates (%) on LIBERO under severe haze (

\alpha_{\mathrm{h}}=0.95

I_{\mathrm{air}}=235

), evaluated with 50 rollouts per task. Avg. is the macro-average over the four suites.

Figure 12: Temporal rendering stability on an overexposed LIBERO ( Liu et al., 2023 ) trajectory with Qwen3-OFT ( StarVLA Community, 2026 ) . RAW-Adapter ( Cui et al., 2026 ) exhibits pronounced frame-to-frame appearance fluctuations that destabilize the policy and lead to task failure, whereas RawVLA maintains temporally consistent observations and successfully completes the task.

	Training objective			LIBERO / Qwen3-OFT						RoboTwin 2.0 / PI-0.5
Variant	$\mathcal{L}_{\rm action}$	$\mathcal{L}_{\rm bright}$	$\mathcal{L}_{\rm color}$	Ext.Low	Low	Normal	Over	Ext.Over	Avg.	Ext.Low	Low	Normal	Over	Ext.Over	Avg.
Action loss only	✓			0.00	0.00	97.75 97.75	44.50 44.50	29.25 29.25	34.30	0.00	0.00	67.72 67.72	54.76 54.76	46.40 46.40	33.70
w/o action loss		✓	✓	9.50 9.50	72.75 72.75	97.50 97.50	44.25 44.25	25.50 25.50	49.90 49.90	11.81 11.81	55.12 55.12	62.20 62.20	59.52 59.52	51.20 51.20	47.94 47.94
Brightness only		✓		4.00 4.00	39.50 39.50	97.25 97.25	41.25	21.50 21.50	40.70 40.70	7.87 7.87	41.73 41.73	65.35 65.35	53.17	51.20 51.20	43.83 43.83
Color only			✓	0.00	0.00	97.50 97.50	43.50 43.50	20.00	32.20	0.00	12.60	53.54	52.38	26.40	23.73
Full RawVLA	✓	✓	✓	14.05 14.05	81.35 81.35	96.95	43.40	20.85	51.32 51.32	25.98 25.98	63.78 63.78	67.72 67.72	65.08 65.08	63.20 63.20	57.12 57.12

Appendix

Table 8: Ablation of the three training objectives. The full objective is compared with action-free auxiliary supervision and each loss used in isolation. Loss weights are fixed and are not swept.

Variant	Ext.Low			Low			Normal			Over			Ext.Over			Overall
Variant	SOG	Long	Avg	SOG	Long	Avg	SOG	Long	Avg	SOG	Long	Avg	SOG	Long	Avg	SOG	Long	Avg
w/o Exposure	6.87	3.60	6.05	42.80	22.00	37.60	98.53 98.53	92.20 92.20	96.95 96.95	92.34 92.34	60.00 60.00	84.25 84.25	38.47	1.00 1.00	29.10	55.80	35.76	50.79
w/o White-balance	15.87 15.87	12.60 12.60	15.05 15.05	87.46 87.46	62.00 62.00	81.10 81.10	98.53 98.53	94.20 94.20	97.45 97.45	91.67	53.00	82.00	42.80	2.00 2.00	32.60	67.27 67.27	44.76 44.76	61.64 61.64
w/o CCM	12.20 12.20	12.60 12.60	12.30 12.30	86.46	62.00 62.00	80.35	97.86 97.86	95.20 95.20	97.20 97.20	92.67 92.67	58.00 58.00	84.00 84.00	43.13	2.00 2.00	32.85	66.47	45.96 45.96	61.34
w/o Tone-mapping	7.53	5.60 5.60	7.05	73.46	46.00	66.60	97.53 97.53	91.20	95.95	90.67	54.00	81.50	45.13 45.13	2.00 2.00	34.35 34.35	62.87	39.76	57.09
$\llcorner$ Shared tone	15.20 15.20	10.60 10.60	14.05 14.05	86.80 86.80	63.00 63.00	80.85 80.85	98.53 98.53	92.20 92.20	96.95 96.95	92.00 92.00	54.00	82.50 82.50	47.47 47.47	1.00 1.00	35.85 35.85	68.00 68.00	44.16 44.16	62.04 62.04

Appendix

Table 9: Ablation study of RawVLA’s modules on LIBERO with Qwen3-OFT ( StarVLA Community, 2026 ) . We report success rates (%) across five illumination levels and overall. Each ablated variant removes one ISP module while keeping the remaining architecture unchanged. Shared tone denotes the shared-head equivalence control.

Variant	Ext.Low	Low	Normal	Over	Ext.Over	Avg.
w/o Exposure	12.60	18.11	65.35	61.11 61.11	60.00 60.00	43.35
w/o White-balance	25.98 25.98	58.27 58.27	66.14 66.14	65.08 65.08	54.40	53.96 53.96
w/o CCM	25.20 25.20	61.42 61.42	69.29 69.29	57.14 57.14	62.40 62.40	55.06 55.06
w/o Tone-mapping	22.05 22.05	50.39	64.57	65.08 65.08	60.00 60.00	52.37
Full RawVLA	25.98 25.98	63.78 63.78	67.72 67.72	65.08 65.08	63.20 63.20	57.12 57.12

Appendix

Table 10: Inference-only ISP component ablations on RoboTwin 2.0 using PI-0.5. All ablated variants use the same frozen checkpoint and force one ISP component to identity. Full (inference-only) keeps all components enabled, while Full RawVLA reports the main benchmark result.

Benchmark	Ablation	Setting	Ext.Low		Low		Normal		Average
Benchmark	Ablation	Setting	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$
LIBERO Qwen3-OFT	Burst length	$K=1$	18.93	0.117	19.99	0.261	36.69	0.863	25.21	0.414
		$K=3$	20.15	0.144	21.34 21.34	0.304 0.304	37.66 37.66	0.893	26.38 26.38	0.447
		$K=6$	21.98 21.98	0.195 0.195	23.27 23.27	0.373 0.373	37.57 37.57	0.921 0.921	27.61 27.61	0.497 0.497
	Fusion strength	$\eta=0.2$	19.63	0.132	20.79	0.286	37.44	0.882	25.95	0.433
		$\eta=0.5$	21.98 21.98	0.195 0.195	23.27 23.27	0.373 0.373	37.57 37.57	0.921 0.921	27.61 27.61	0.497 0.497

Appendix

Table 11: Burst-denoising ablations. We study the burst length

K

, global fusion weight

\eta

, and frequency-wise reliability decomposition. We compare the rendered output

\hat{I}

against the corresponding noise-free simulation ground truth

I_{\rm GT}

using PSNR (dB) and SSIM. Average is computed over the ExtremeLow, Low, and Normal illumination regimes. Underlined settings denote the default configuration.

Figure 13: Real-world dual-arm platform and camera setup.

Figure 14: Objects used in the four real-world tasks.

Figure 15: Action-loss curves for PI0.5 ( Physical Intelligence et al., 2025 ) fine-tuning on the four real-world tasks, using 909 demonstrations and 263,863 frames in total.

Task	Language prompt
pick_and_place	Place the green apple, pink object, and yellow banana in the blue bowl.
drag_the_drawer	Open the drawer and place the yellow block inside.
pick_flower	Pick up the two flowers and place them in the white vase.
stack_and_cover	Stack the green and blue blocks and cover them with the pink cup.

Appendix

Table 12: Language prompts for the four real-world tasks.

Figure 16: Pick & place executions under normal and low-light illumination. The third-person frames receive postprocessing enhancement only for visualization and are not provided to the policy. The lower strips show the rendered agent views produced by Dark-ISP ( Guo et al., 2025 ) and RawVLA under low light.

Figure 17: Drug drawer executions under normal and low-light illumination. The third-person frames receive postprocessing enhancement only for visualization and are not provided to the policy. The lower strips show the rendered agent views produced by Dark-ISP ( Guo et al., 2025 ) and RawVLA under low light.

Figure 18: Pick flower executions under normal and low-light illumination. The third-person frames receive postprocessing enhancement only for visualization and are not provided to the policy. The lower strips show the rendered agent views produced by Dark-ISP ( Guo et al., 2025 ) and RawVLA under low light.

Figure 19: Stack & cover executions under normal and low-light illumination. The third-person frames receive postprocessing enhancement only for visualization and are not provided to the policy. The lower strips show the rendered agent views produced by Dark-ISP ( Guo et al., 2025 ) and RawVLA under low light.

Illumin.	Backbone	Default	DarkISP	RAM	RAWAdapter	RAWild	Ours
Extreme Lowlight	WM4A-Wan	0.00	0.05	0.00	0.00	0.00	0.75
Extreme Lowlight	WM4A-Cosmos	0.00	0.00	0.00	0.00	0.00	0.20
Lowlight	WM4A-Wan	0.05	0.05	0.05	0.00	0.00	0.55
Lowlight	WM4A-Cosmos	0.05	0.00	0.00	0.00	0.00	0.75
Normal	WM4A-Wan	76.10	76.50	0.00	38.05	75.15	78.20
Normal	WM4A-Cosmos	85.75	0.75	1.75	2.45	4.25	83.45

Appendix

Table 13: Success rates (%) of ISP modules across five illumination levels on LIBERO using WAM backbones. Avg reports the mean across illumination levels for each backbone.

Figure 20: Representative ISP perturbations on LIBERO ( Liu et al., 2023 ) . In each row, the first column shows the default RGB rendering and the remaining columns show six perturbed samples. Exposure uses EV offsets $\{-3,-2,-1,+1,+2,+3\}$ . White-point and relative-color response each use six sampled chromatic transformations. Tone mapping uses six contrast settings, and bit depth decreases from 7 to 2 bits.

Figure 21: Representative ISP perturbations on RoboTwin 2.0 ( Chen et al., 2026b ) . In each row, the first column shows the default RGB rendering and the remaining columns show six perturbed samples. Exposure uses EV offsets $\{-3,-2,-1,+1,+2,+3\}$ . White-point and relative-color response each use six sampled chromatic transformations. Tone mapping uses six contrast settings, and bit depth decreases from 7 to 2 bits.

Explore similar work

TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM
Jul 29, 2026Hengyi Xie, Chenfei Yao, Xianjin Wu +4Diffusion-Based Vision-Language-Actions Nvidia
Huazhong University of Science and Technology
E-VLA: Event-Augmented Vision-Language-Action Model for Dark and Blurred Scenes
Apr 6, 2026Jiajun Zhai, Hao Shi, Shangwei Guo +2Diffusion-Based Vision-Language-Actions Generalist Vision--Language--Action
Zhejiang University · Ant Group · Hunan University
G $^3$ VLA: Geometric inductive bias for Vision-Language-Action Models
Jun 23, 2026Yue Peng, Yongzhe Zhao, Artur Habuda +5Diffusion-Based Vision-Language-Actions Visual Geometry Grounded Transformer
New York University Shanghai · Technical University of Denmark · MBZUAI - Mohamed bin Zayed University of Artificial Intelligence

Jul 29, 2026, cs.CVJ/K move · Enter open · S save

TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM

Hengyi Xie, Chenfei Yao, Xianjin Wu, Yingying Zhu+3

Huazhong University of Science and Technology

Diffusion-Based Vision-Language-Actions Nvidia

Open paper AskPDF arXiv

Vision-language-action (VLA) models commonly adopt an LLM-centric

V \to L \to A

pathway, processing visual observations and language instructions through a large language model before predicting robot actions. Although effective, this design incurs substantial computation and memory overhead. In this work, we introduce TurboVLA, a compact VLA architecture built on a direct

V + L \to A

mapping. Instead of using a large language model as the central interface between perception and action, TurboVLA independently encodes visual observations and language instructions, directly exchanges information between them through lightweight bidirectional vision-language interaction, and predicts continuous action chunks with a compact decoder. This simple design directly constructs task-conditioned representations while avoiding the overhead of an LLM-centered execution pathway. On LIBERO, TurboVLA achieves 97.6% average success with only 0.2B parameters, 31.2 ms inference latency, and 0.9 GiB inference VRAM on a consumer-grade RTX 4090. Notably, a 0.4B TurboVLA achieves 88.06% success on RoboTwin 2.0, even matching or outperforming substantially larger VLA policies. These results demonstrate that the simple

V + L \to A

design of TurboVLA can achieve high performance without requiring an LLM-centric execution pathway, offering a new perspective on how vision, language, and action can be connected for efficient robotic manipulation. Code is available at https://github.com/H-EmbodVis/TurboVLA.

Abstract

Explore similar work

TurboVLA: Real-Time Vision-Language-Action Model at 32 Hz on an RTX 4090 with <1 GB VRAM

E-VLA: Event-Augmented Vision-Language-Action Model for Dark and Blurred Scenes

G3^33VLA: Geometric inductive bias for Vision-Language-Action Models

G $^3$ VLA: Geometric inductive bias for Vision-Language-Action Models