RASLF: Representation-Aware State Space Model for Light Field Super-Resolution
Authors: Zeqiang Wei, Kai Jin, Kuan Song, Xiuzhuang Zhou, Wenlong Chen, Min Xu
Organizations: Capital Normal University Information Engineering College, Beijing 100048, China · Bigo Technology Pte. Ltd., Beijing 100020, China · Explorer Global (Suzhou) Artificial Intelligence Technology Co., Ltd., Suzhou, Jiangsu 215123, China · Beijing University of Posts and Telecommunications, Beijing 100088, China
Current SSM-based light field super-resolution (LFSR) methods often fail to fully leverage the complementarity among various LF representations, leading to the loss of fine textures and geometric misalignments across views. To address these issues, we propose RASLF, a representation-aware state-space framework that explicitly models structural correlations across multiple LF representations. Specifically, a Progressive Geometric Refinement (PGR) block is created that uses a panoramic epipolar representation to explicitly encode multi-view parallax differences, thereby enabling integration across different LF representations. Furthermore, we introduce a Representation Aware Asymmetric Scanning (RAAS) mechanism that dynamically adjusts scanning paths based on the physical properties of different representation spaces, optimizing the balance between performance and efficiency through path pruning. Additionally, a Dual-Anchor Aggregation (DAA) module improves hierarchical feature flow, reducing redundant deeplayer features and prioritizing important reconstruction information. Experiments on various public benchmarks show that RASLF achieves the highest reconstruction accuracy while remaining highly computationally efficient.
Figures & tables
Fig. 1: Overall architecture of the proposed RASLF. (a) The Progressive Geometric Refinement (PGR) paradigm sequentially refines features using representation-specific VSSM units, and a Dual-Anchor Aggregation module fuses multi-stage features via spatial and geometric anchors. (b) Representation-Aware Asymmetric Scanning (RAAS) tailors SS2D scanning paths Φ and representation transforms T,T−1 for SAI, MacPI, and EPI, reducing redundant computation while preserving geometry-aware dependencies.
Fig. 2: Illustration of the Panoramic Epipolar Representation and the corresponding representation-aware scanning paths (indicated by red solid arrows).
Method
Scale
EPFL
HCINew
HCIold
INRIA
STFgantry
Average
RCAN [ 33 ]
× 2
33.156/0.9635
35.022/0.9603
41.125/0.9875
35.036/0.9769
36.670/0.9831
36.202/0.9743
resLF [ 34 ]
× 2
33.617/0.9706
36.685/0.9739
43.422/0.9932
35.395/0.9804
38.354/0.9904
37.495/0.9817
LFSSR [ 35 ]
× 2
33.671/0.9744
36.802/0.9749
43.811/0.9938
35.279/0.9832
37.944/0.9898
37.501/0.9832
LF-ATO [ 36 ]
× 2
34.272/0.9757
37.244/0.9767
44.205/0.9942
36.171/0.9842
39.636/0.9929
38.306/0.9847
MEG-Net [ 37 ]
× 2
34.312/0.9773
37.424/0.9777
44.097/0.9942
36.103/0.9849
38.767/0.9915
38.141/0.9851
DistgSSR [ 25 ]
× 2
34.809/0.9787
37.959/0.9796
44.943/0.9949
36.586/0.9859
40.404/0.9942
38.940/0.9867
TABLE I: PSNR/SSIM results compared with SOTA methods for 2 × and 4 × LFSR tasks. The best and second-best results are, respectively, in bold and underlined.
Fig. 3: Qualitative visualization results for 4 × LFSR compared to other methods. Here, we showed the error maps of the reconstructed center-view images, with representative regions indicated by arrows. PSNR/SSIM values for the corresponding region are provided below.
Method
Scale
Params.
FLOPs(G)
Time(ms)
Avg. PSNR/SSIM
LFSSR [ 35 ]
× 2
0.89M
25.70
10.0
37.501/0.9832
MEG-Net [ 37 ]
× 2
1.69M
48.40
31.2
38.141/0.9851
LF-ATO [ 36 ]
× 2
1.22M
597.66
85.6
38.306/0.9847
LF-IINet [ 39 ]
× 2
5.04M
56.16
20.6
38.763/0.9860
DistgSSR [ 25 ]
× 2
3.53M
64.11
24.2
38.940/0.9867
HLFSR-SSR [ 40 ]
× 2
13.72M
167.81
31.1
39.303/0.9874
TABLE II: Comparison of parameters, flops, time, and average PSNR/SSIM values for × 2 and × 4 SR. Flops and time are calculated on an input LF with a size of 5×5×32×32 .
Fig. 4: Comparison of computational efficiency between our method and SOTA models on the 4× LFSR task. The area of each circle represents memory consumption.
PSNR / SIMM
Params
FLOPS
w./o. DDA
32.637 / 0.9466
1.086
37.928
w. EPI-H & EPI-V
32.588 / 0.9466
0.953
31.200
w. EPIs-H & EPIs-V
32.628 / 0.9468
0.953
31.200
RASLF (Our)
32.690 / 0.9472
0.953
31.200
TABLE III: Ablation study on different components for 4 × LFSR.
SAI
MacPI
H-EPI
V-EPI
PSNR / SIMM
Params
FLOPS
(a)
Φ4-path
Φ4-path
{Φrow,Φrow′}
{Φcol,Φcol′}
32.703 / 0.9473
1.013
36.050
(b)
{Φrow,Φcol}
Φ4-path
{Φrow,Φrow′}
{Φcol,Φcol′}
32.700 / 0.9471
0.983
33.625
(c)
{Φrow,Φcol}
Φ4-path
{Φcol}
{Φcol}
32.696 / 0.9471
0.953
31.200
(d)
{Φrow,Φcol}
{Φrow,Φcol}
{Φcol}
{Φcol}
32.329 / 0.9440
0.923
28.775
TABLE IV: Ablation study on different representation scanning strategies for 4 × LFSR.
M
C
PSNR / SIMM
Params
FLOPS
4
64
32.690 / 0.9472
0.953
31.200
8
64
32.901 / 0.9492
1.654
55.756
12
64
32.972 / 0.9497
2.310
76.676
16
64
33.070 / 0.9502
2.996
100.020
4
128
33.039 / 0.9498
3.486
101.185
4
256
33.129 / 0.9508
13.412
367.205
TABLE V: Investigation on the scalability of the proposed RASLF with respect to network depth (number of PGR blocks M ) and width (feature dimensions C ).
Department of EE, IIT Kharagpur, India · Rekhi Centre of Excellence for the Science of Happiness, IIT Kharagpur, India · Department of E&ECE, IIT Kharagpur, India
School of Computer Science and Technology, Xi’an Shiyou University · School of Electronic Engineering, Xi’an Shiyou University · Artificial Intelligence Research Institute, Shenzhen University of Advanced Technology