Visual Autoregressive Priors for RAW-to-sRGB Image Signal Processing

Tailai Chen; Xiaotong Luo; Yuan Gao; Xin Jin; Wenjun Zeng

cs.CVSep 16, 2026

Visual Autoregressive Priors for RAW-to-sRGB Image Signal Processing

Authors: Tailai Chen, Xiaotong Luo, Yuan Gao, Xin Jin, Wenjun Zeng

Organizations: OmniVision-IDT Joint Laboratory for Intelligent Image Sensing, Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative, Ningbo Institute of Digital Twin, Eastern Institute of Technology, Ningbo; Zhejiang Key Laboratory of Industrial Intelligence and Digital Twin · The Hong Kong Polytechnic University

Abstract

RAW-to-sRGB image signal processing (ISP) must recover perceptually faithful colors and fine details from sensor measurements, often under imperfect spatial alignment and missing camera metadata. This paper presents, to the best of our knowledge, the first application of visual autoregressive (VAR) next-scale prediction over a discrete image codebook to the RAW-to-sRGB ISP task. We adapt a frozen 1.10,B-parameter VAR backbone for RAW-conditioned ISP with only 32.93,M trainable parameters (2.99%), and propose a frequency-decomposed color loss that separately supervises low-frequency tone via wavelet LL cosine similarity and chromatic edges via detail-band $\ell_1$ . On the Zurich RAW-to-sRGB benchmark, the method improves PSNR-Y from 21.31 to 21.89,dB and reduces LPIPS from 0.276 to 0.218 on the full 1,204-image test set. Diagnostic experiments show that the VAR prior preserves structure well, but continuous color transfer remains the dominant bottleneck: oracle affine correction recovers 3.8,dB, while learned color heads yield marginal gains.

Figures & tables

Figure 1: Overview of the proposed RAW-to-sRGB formulation. The frozen VQ-VAE tokenizes the target into multi-scale codes for training supervision. The frozen VAR transformer predicts codes conditioned on a RAW-derived signal through trainable adapters. A frequency-decomposed color loss supervises tone and chromatic edges in the wavelet domain.

Configuration	PSNR-Y $\uparrow$	SSIM-Y $\uparrow$	LPIPS $\downarrow$	CLIPIQA $\uparrow$	MUSIQ $\uparrow$
Baseline	21.312	0.7272	0.2764	0.4538	47.80
+ Spatial color loss	21.681	0.7356	0.2543	0.4229	44.97
+ Freq-decomposed loss	21.891	0.7562	0.2176	0.4107	43.52

Table 1: Ablation on the full ZRR test set (1,204 images). Each row adds one component to the baseline.

Method	Evaluation	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
PyNet [ 5 ]	RGB, published	21.19	0.747	0.193
AWNet-R [ 1 ]	RGB, published	21.42	0.748	0.198
AWNet-D [ 1 ]	RGB, published	21.53	0.749	0.212
MW-ISPNet [ 4 ]	RGB, published	21.42	0.754	0.213
LiteISPNet [ 16 ]	RGB, published	21.55	0.749	0.187
FourierISP [ 2 ]	RGB, published	21.65	0.755	0.182

Table 2: Published ZRR landscape under nonidentical evaluation protocols. Prior-work values are RGB-channel results quoted from the corresponding papers, whereas ours is evaluated on the Y channel over the full test set. The two blocks provide context and must not be numerically ranked across protocols.

Figure 2: Qualitative results on ZRR. Each row: RAW input (left), our output (middle), ground truth (right). The model preserves structure and edges; residual errors are global color, brightness, and contrast shifts.

Figure 3: VQ-VAE reconstruction ceiling. Each row: ground truth (left), VQ-VAE encode-decode reconstruction (right). The codebook preserves structure, color, and most detail; artifacts appear only in smooth gradient regions.

Setting	PSNR-Y $\uparrow$	SSIM-Y $\uparrow$	LPIPS $\downarrow$
Oracle affine, before	21.129	0.7123	0.2892
Oracle affine, after	24.950	0.7269	0.2815
Low-freq residual, before	21.681	0.7357	0.2543
Low-freq residual, after	21.694	0.7357	0.2543
Adaptive CCM, before	21.678	0.7356	0.2543
Adaptive CCM, after	21.391	0.7359	0.2648

Table 3: Post-hoc color diagnostics on a 100-image ZRR subset. The oracle affine uses each test target to fit a per-image affine transform and is not deployable; it estimates the ceiling of color-only correction.

Figure 4: Oracle affine correction. Each row, left to right: RAW input, model output, oracle-corrected output, ground truth. The oracle recovers target color and contrast, confirming the output contains necessary structural detail.

Explore similar work

RPBA-Net: An Interpretable Residual Pyramid Bilateral Affine Network for RAW-Domain ISP Enhancement
May 5, 2026Yucheng Xin, Wu Chen, Xiang Chen +7Image Enhancement Residual Learning
School of Information Science and Engineering, Shandong Normal University, Jinan, China · National University of Defense Technology, Changsha, China · Nanjing University of Science and Technology, Nanjing, China +2
RawVLA: Embodied Neural Image Signal Processor For Robotic Manipulation
Sep 29, 2026Shuhong Liu, Heng Zhou, Lingfeng Qian +7Robotic Manipulation Vision-Language-Action Models
UTokyo · D-Robotics · TohokuU +2
ExtraVAR: Stage-Aware RoPE Remapping for Resolution Extrapolation in Visual Autoregressive Models
May 11, 2026Feihong Yan, Shaoyu Liu, Haixuan Wang +4Rotary Positional Embeddings Visual Autoregressive Models
Beijing Institute of Technology · Xidian University · Northeastern University at Qinhuangdao +3

May 5, 2026J/K move · Enter open · S save

RPBA-Net: An Interpretable Residual Pyramid Bilateral Affine Network for RAW-Domain ISP Enhancement

Yucheng Xin, Wu Chen, Xiang Chen, Guangwei Gao+6

School of Information Science and Engineering, Shandong Normal University, Jinan, China · National University of Defense Technology, Changsha, China · Nanjing University of Science and Technology, Nanjing, China+2

Image Enhancement Residual Learning

Open paper AskPDF arXiv

To address module fragmentation, uninterpretable mappings, and deployment constraints in RAW-domain demosaicing, color correction, and detail enhancement, this paper proposes RPBA-Net, an interpretable residual pyramid bilateral affine network for RAW-domain ISP enhancement. Given packed RAW as input, the method performs residual affine base reconstruction by estimating a base RGB representation and learning identity-guided residual affine corrections, thereby unifying demosaicing and enhancement. It further builds pyramid bilateral affine grids and combines guide-driven autoregressive adaptive slicing with adaptive cross-layer fusion to hierarchically model global tone restoration and local texture enhancement. In addition, smoothness, cross-scale consistency, and magnitude regularization terms are introduced to improve model stability, controllability, and structural interpretability. Extensive experiments demonstrate that RPBA-Net surpasses representative RAW-to-sRGB methods and achieves state-of-the-art performance in reconstruction fidelity and perceptual quality, while maintaining low model complexity and strong deployment potential for mobile and embedded platforms.

Abstract

Explore similar work

RPBA-Net: An Interpretable Residual Pyramid Bilateral Affine Network for RAW-Domain ISP Enhancement

RawVLA: Embodied Neural Image Signal Processor For Robotic Manipulation

ExtraVAR: Stage-Aware RoPE Remapping for Resolution Extrapolation in Visual Autoregressive Models