SpiralDiff: Spiral Diffusion with LoRA for RGB-to-RAW Conversion Across Cameras
Authors: Huanjing Yue, Shangbin Xie, Cong Cao, Qian Wu, Lei Zhang, Lei Zhao, Jingyu Yang
Organizations: School of Electrical and Information Engineering, Tianjin University · State key laboratory of Smart Power Distribution Equipment and System, Tianjin University · Individual
RAW images preserve superior fidelity and rich scene information compared to RGB, making them essential for tasks in challenging imaging conditions. To alleviate the high cost of data collection, recent RGB-to-RAW conversion methods aim to synthesize RAW images from RGB. However, they overlook two key challenges: (i) the reconstruction difficulty varies with pixel intensity, and (ii) multi-camera conversion requires camera-specific adaptation. To address these issues, we propose SpiralDiff, a diffusion-based framework tailored for RGB-to-RAW conversion with a signal-dependent noise weighting strategy that adapts reconstruction fidelity across intensity levels. In addition, we introduce CamLoRA, a camera-aware lightweight adaptation module that enables a unified model to adapt to different camera-specific ISP characteristics. Extensive experiments on four benchmark datasets demonstrate the superiority of SpiralDiff in RGB-to-RAW conversion quality and its downstream benefits in RAW-based object detection. Our code and model are available at https://github.com/Chuancy-TJU/SpiralDiff.
Figures & tables
Figure 1: Comparison of noise schedule (top) and visualization of the noisy xT (bottom) in diffusion: ResShift [ 45 ] uses uniform Gaussian noise, whereas our SpiralDiff introduces a signal-dependent noise schedule based on pixel intensity ( wt ).
Figure 2: Relationship between RGB pixel intensity and residual magnitude between RGB and RAW images across channels on the FiveK Nikon dataset. Colored lines and shaded regions represent the mean and standard deviation, respectively.
Figure 3: Overview of the proposed SpiralDiff with CamLoRA. (a) SpiralDiff introduces a signal-dependent noise schedule via a weight map set {wt}t=1T that aligns with the RAW-to-RGB conversion process. The noise level depends on local pixel intensity and diffusion timestep t : darker regions ( △ ) receive less noise, and brighter regions ( ◯ ) receive more. (b) The spiral structure visualizes how noise scales with wt for different pixel intensities. (c) shows the framework of SpiralDiff. The noisy image xt , RGB image y0 and camera label are fed into denoising U-Net, which iteratively samples to refine the RAW output. (d) The camera label selects the camera-specific LoRA layer in CamLoRA, enhancing adaptation to each camera’s characteristic.
Methods
Separate
Combined
FiveK Canon
FiveK Nikon
NOD Nikon
NOD Sony
FiveK Canon
FiveK Nikon
NOD Nikon
NOD Sony
CycleISP [ 47 ]
37.93 / 0.9913
40.18 / 0.9920
50.11 / 0.9985
46.57 / 0.9975
38.06 / 0.9911
38.86 / 0.9918
48.68 / 0.9984
46.12 / 0.9971
InvISP [ 40 ]
36.81 / 0.9814
34.30 / 0.9163
48.29 / 0.9954
44.76 / 0.9922
35.37 / 0.9828
36.60 / 0.9850
47.73 / 0.9948
44.43 / 0.9907
ReRAW [ 3 ]
35.08 / 0.9608
33.09 / 0.9265
48.25 / 0.9906
45.50 / 0.9788
32.70 / 0.9684
33.24 / 0.9648
46.39 / 0.9715
45.51 / 0.9817
RAW-Diffusion [ 33 ]
39.96 / 0.9890
39.68 / 0.9866
50.52 / 0.9954
47.31 / 0.9908
40.63 / 0.9889
40.35 / 0.9916
49.60 / 0.9950
46.80 / 0.9897
SpiralDiff
42.82 / 0.9936
41.72 / 0.9925
53.64 / 0.9990
50.46 / 0.9980
41.99 / 0.9931
43.19 / 0.9947
52.06 / 0.9987
49.52 / 0.9974
Table 1: Performance comparison of different methods trained on separate and combined datasets, measured by PSNR( ↑ ) and SSIM( ↑ ). +CamLoRA denotes that CamLoRA is applied to the combined training setting. Best results under each setting are highlighted in bold .
Methods
FiveK Canon
NOD Nikon
PSNR
SSIM
PSNR
SSIM
CycleISP [ 47 ]
27.75
0.9833
35.86
0.9957
InvISP [ 40 ]
25.15
0.9699
36.66
0.9905
ReRAW [ 3 ]
26.36
0.9664
38.56
0.9691
RAW-Diffusion [ 33 ]
30.60
0.9739
40.05
0.9926
SpiralDiff
31.10
0.9856
40.79
0.9973
Table 2: Quantitative comparison on the over-exposed test set.
Figure 4: Qualitative comparison with state-of-the-art RGB-to-RAW conversion methods on the FiveK dataset (top two rows) and the NOD dataset (bottom two rows). For each result, the left half is the predicted RAW, and the right half is the error map. The proposed SpiralDiff shows better conversion results, especially in bright regions.
Variant
FiveK Canon
NOD Nikon
PSNR
SSIM
PSNR
SSIM
Baseline
41.40
0.9906
53.48
0.9989
y0
40.06
0.9912
53.42
0.9990
wt
42.82
0.9936
53.64
0.9990
Table 3: Ablations on the noise weighting strategies. The baseline uses spatially uniform noise. y0 ( wt ) means the noise is weighted according to the intensity of y0 ( wt ).
Variant
FiveK Canon
NOD Nikon
PSNR
SSIM
PSNR
SSIM
Uncond.
41.99
0.9931
52.06
0.9987
Embed.
40.97
0.9917
51.44
0.9984
CamLoRA
42.46
0.9934
52.62
0.9988
Table 4: Ablations on three conditioning strategies: Uncond. (no camera info), Embed. (camera embedding), and CamLoRA .
Figure 5: Residual-intensity relationship on rawpy ISP (FiveK Nikon) and real ISP (iPhone and Samsung).
Variant
1-shot
5-shot
PSNR
SSIM
PSNR
SSIM
Scratch
39.83
0.9895
43.40
0.9951
CamLoRA
42.85
0.9942
43.92
0.9954
Table 5: Few-shot RGB-to-RAW conversion on the SID Sony dataset. We compare training from scratch with CamLoRA adaptation under 1-shot and 5-shot settings.
Method
iPhone
Samsung
RAW-Diffusion (DDPM)
22.74 / 0.7503
30.28 / 0.9140
RAW-Diffusion (SpiralDiff)
30.11 / 0.8810
34.07 / 0.9322
Table 6: Plug-in experiment on the real-ISP dataset.
Dataset
NOD Nikon
NOD Sony
AP
AP 50
AP 75
AP
AP 50
AP 75
RGB-only
19.1
37.0
18.5
19.7
38.3
18.0
RAW-only
18.4
35.1
17.0
17.6
35.7
15.1
RAW + CS-RAW
24.5
46.2
23.5
25.9
49.3
24.5
RAW + BDD-RAW
26.7
49.5
25.1
29.0
55.1
26.5
Table 7: Comparison of object detection results on the NOD test set with different training datasets.
Figure 13
Method
FiveK Canon
FiveK Nikon
NOD Nikon
NOD Sony
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
CycleISP [ 47 ]
27.75
0.9833
30.19
0.9872
35.86
0.9957
32.52
0.9890
InvISP [ 40 ]
25.15
0.9699
29.11
0.9764
36.66
0.9905
30.85
0.9782
ReRAW [ 3 ]
26.36
0.9664
27.51
0.9520
38.56
0.9691
34.96
0.9713
RAW-Diffusion [ 33 ]
30.60
0.9739
32.95
0.9850
40.05
0.9926
35.58
0.9792
SpiralDiff
31.10
0.9856
34.42
0.9908
40.79
0.9973
36.11
0.9919
Table 8: Quantitative comparison results on the over-exposed test set.
Rank
FiveK Canon
FiveK Nikon
NOD Nikon
NOD Sony
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
r=0
41.99
0.9931
43.19
0.9947
52.06
0.9987
49.52
0.9974
r=2
42.64
0.9928
43.52
0.9948
52.23
0.9987
49.78
0.9977
r=4
42.34
0.9931
43.95
0.9948
52.47
0.9988
49.82
0.9977
r=8
42.46
0.9934
43.82
0.9950
52.62
0.9988
50.08
0.9977
r=16
42.00
0.9932
42.82
0.9945
52.36
0.9989
49.60
0.9978
Table 9: Ablation study on the LoRA rank r . We set r=8 as the default setting for all experiments.
Variant
FiveK Canon
FiveK Nikon
NOD Nikon
NOD Sony
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
PSNR
SSIM
RAW-Diffusion (DDPM)
39.96
0.9890
39.68
0.9866
50.52
0.9954
47.31
0.9908
RAW-Diffusion (SpiralDiff)
41.53
0.9912
42.34
0.9920
50.54
0.9954
47.32
0.9907
Table 10: RAW-Diffusion (DDPM) denotes the original RAW-Diffusion model with its DDPM-based diffusion process, while RAW-Diffusion (SpiralDiff) replaces only this diffusion process with our SpiralDiff formulation, keeping the network architecture unchanged.
Cameras capture scene-referred linear raw images, which are processed by onboard image signal processors (ISPs) into display-referred 8-bit sRGB outputs. Although raw data is more faithful for low-level vision tasks, collecting large-scale raw datasets remains a major bottleneck, as existing datasets are limited and tied to specific camera hardware. Generative models offer a promising way to address this scarcity -- however, existing diffusion frameworks are designed to synthesize photo-finished sRGB images rather than physically consistent linear representations. This paper presents RawGen, to our knowledge the first diffusion-based framework enabling text-to-raw generation for arbitrary target cameras, alongside sRGB-to-raw inversion. RawGen leverages the generative priors of large-scale sRGB diffusion models to synthesize physically meaningful linear outputs, such as CIE XYZ or camera-specific raw representations, via specialized processing in latent and pixel spaces. To handle unknown and diverse ISP pipelines and photo-finishing effects in diffusion-model training data, we build a many-to-one inverse-ISP dataset where multiple sRGB renditions of the same scene generated using diverse ISP parameters are anchored to a common scene-referred target. Fine-tuning a conditional denoiser and specialized decoder on this dataset allows RawGen to obtain camera-centric linear reconstructions that effectively invert the rendering pipeline. We demonstrate RawGen's superior performance over traditional inverse-ISP methods that assume a fixed ISP. Furthermore, we show that augmenting training pipelines with RawGen's scalable, text-driven synthetic data can benefit downstream low-level vision tasks.
Dongyoung Kim, Junyong Lee, Abhijith Punnappurath +4
AI Center - Toronto, Samsung Electronics · Yonsei University
Raw images preserve linear sensor measurements and high bit-depth information crucial for advanced vision tasks and photography applications, yet their storage remains challenging due to large file sizes, varying bit depths, and sensor-dependent characteristics. Existing learned lossless compression methods mainly target 8-bit sRGB images, while raw reconstruction approaches are inherently lossy and rely on camera-specific assumptions. To address these challenges, we introduce RAWIC, a bit-depth-adaptive learned lossless compression framework for Bayer-pattern raw images. We first convert single-channel Bayer data into a four-channel RGGB format and partition it into patches. For each patch, we compute its bit depth and use it as auxiliary input to guide compression. A bit-depth-adaptive entropy model is then designed to estimate patch distributions conditioned on their bit depths. This architecture enables a single model to handle raw images from diverse cameras and bit depths. Experiments show that RAWIC consistently surpasses traditional lossless codecs, achieving an average 7.7% bitrate reduction over JPEG-XL. Our code is available at https://github.com/chunbaobao/RAWIC.
Chunhang Zheng, Tongda Xu, Mingli Xie +2
School of Electronics, Peking University, Beijing, 100871, China · Institute for AI Industry Research (AIR), Tsinghua University, Beijing, 100084, China · the University of Chinese Academy of Sciences, Beijing, 100190, China
RAW images captured by different camera sensors exhibit substantial domain shifts due to varying spectral responses, noise characteristics, and tone behaviors, complicating their direct use in downstream computer vision tasks. Prior methods address this problem by training domain-specific RAW-to-RAW translators for each source-target pair, but such approaches do not scale to real-world scenarios involving multiple types of commercial cameras. In this work, we introduce MERIT, the first unified framework for multi-domain RAW image translation, which leverages a single model to perform translations across arbitrary camera domains. To address domain-specific noise discrepancies, we propose a sensor-aware noise modeling loss that explicitly aligns the signal-dependent noise statistics of the generated images with those of the target domain. We further enhance the generator with a conditional multi-scale large kernel attention module for improved context and sensor-aware feature modeling. To facilitate standardized evaluation, we introduce MDRAW, the first dataset tailored for multi-domain RAW image translation, comprising both paired and unpaired RAW captures from five diverse camera sensors across a wide range of scenes. Extensive experiments demonstrate that MERIT outperforms prior models in both quality (5.56 dB improvement) and scalability (80% reduction in training iterations).
Wenjun Huang, Shenghao Fu, Yian Jin +10
University of California, Irvine · University of Pennsylvania · Purdue University Northwest +1