RASLF: Representation-Aware State Space Model for Light Field Super-Resolution
Organizations: Capital Normal University Information Engineering College, Beijing 100048, China · Bigo Technology Pte. Ltd., Beijing 100020, China · Explorer Global (Suzhou) Artificial Intelligence Technology Co., Ltd., Suzhou, Jiangsu 215123, China · Beijing University of Posts and Telecommunications, Beijing 100088, China
Abstract
Current SSM-based light field super-resolution (LFSR) methods often fail to fully leverage the complementarity among various LF representations, leading to the loss of fine textures and geometric misalignments across views. To address these issues, we propose RASLF, a representation-aware state-space framework that explicitly models structural correlations across multiple LF representations. Specifically, a Progressive Geometric Refinement (PGR) block is created that uses a panoramic epipolar representation to explicitly encode multi-view parallax differences, thereby enabling integration across different LF representations. Furthermore, we introduce a Representation Aware Asymmetric Scanning (RAAS) mechanism that dynamically adjusts scanning paths based on the physical properties of different representation spaces, optimizing the balance between performance and efficiency through path pruning. Additionally, a Dual-Anchor Aggregation (DAA) module improves hierarchical feature flow, reducing redundant deeplayer features and prioritizing important reconstruction information. Experiments on various public benchmarks show that RASLF achieves the highest reconstruction accuracy while remaining highly computationally efficient.
Figures & tables
| Method | Scale | EPFL | HCINew | HCIold | INRIA | STFgantry | Average |
|---|---|---|---|---|---|---|---|
| RCAN [ 33 ] | 2 | 33.156/0.9635 | 35.022/0.9603 | 41.125/0.9875 | 35.036/0.9769 | 36.670/0.9831 | 36.202/0.9743 |
| resLF [ 34 ] | 2 | 33.617/0.9706 | 36.685/0.9739 | 43.422/0.9932 | 35.395/0.9804 | 38.354/0.9904 | 37.495/0.9817 |
| LFSSR [ 35 ] | 2 | 33.671/0.9744 | 36.802/0.9749 | 43.811/0.9938 | 35.279/0.9832 | 37.944/0.9898 | 37.501/0.9832 |
| LF-ATO [ 36 ] | 2 | 34.272/0.9757 | 37.244/0.9767 | 44.205/0.9942 | 36.171/0.9842 | 39.636/0.9929 | 38.306/0.9847 |
| MEG-Net [ 37 ] | 2 | 34.312/0.9773 | 37.424/0.9777 | 44.097/0.9942 | 36.103/0.9849 | 38.767/0.9915 | 38.141/0.9851 |
| DistgSSR [ 25 ] | 2 | 34.809/0.9787 | 37.959/0.9796 | 44.943/0.9949 | 36.586/0.9859 | 40.404/0.9942 | 38.940/0.9867 |
| Method | Scale | Params. | FLOPs(G) | Time(ms) | Avg. PSNR/SSIM |
|---|---|---|---|---|---|
| LFSSR [ 35 ] | 2 | 0.89M | 25.70 | 10.0 | 37.501/0.9832 |
| MEG-Net [ 37 ] | 2 | 1.69M | 48.40 | 31.2 | 38.141/0.9851 |
| LF-ATO [ 36 ] | 2 | 1.22M | 597.66 | 85.6 | 38.306/0.9847 |
| LF-IINet [ 39 ] | 2 | 5.04M | 56.16 | 20.6 | 38.763/0.9860 |
| DistgSSR [ 25 ] | 2 | 3.53M | 64.11 | 24.2 | 38.940/0.9867 |
| HLFSR-SSR [ 40 ] | 2 | 13.72M | 167.81 | 31.1 | 39.303/0.9874 |
| PSNR / SIMM | Params | FLOPS | |
|---|---|---|---|
| w./o. DDA | 32.637 / 0.9466 | 1.086 | 37.928 |
| w. EPI-H & EPI-V | 32.588 / 0.9466 | 0.953 | 31.200 |
| w. EPIs-H & EPIs-V | 32.628 / 0.9468 | 0.953 | 31.200 |
| RASLF (Our) | 32.690 / 0.9472 | 0.953 | 31.200 |
| SAI | MacPI | H-EPI | V-EPI | PSNR / SIMM | Params | FLOPS | |
|---|---|---|---|---|---|---|---|
| (a) | 32.703 / 0.9473 | 1.013 | 36.050 | ||||
| (b) | 32.700 / 0.9471 | 0.983 | 33.625 | ||||
| (c) | 32.696 / 0.9471 | 0.953 | 31.200 | ||||
| (d) | 32.329 / 0.9440 | 0.923 | 28.775 |
| M | C | PSNR / SIMM | Params | FLOPS |
|---|---|---|---|---|
| 4 | 64 | 32.690 / 0.9472 | 0.953 | 31.200 |
| 8 | 64 | 32.901 / 0.9492 | 1.654 | 55.756 |
| 12 | 64 | 32.972 / 0.9497 | 2.310 | 76.676 |
| 16 | 64 | 33.070 / 0.9502 | 2.996 | 100.020 |
| 4 | 128 | 33.039 / 0.9498 | 3.486 | 101.185 |
| 4 | 256 | 33.129 / 0.9508 | 13.412 | 367.205 |
Image Super-Resolution
Image super-resolution reconstructs high-resolution images from lower-resolution observations, balancing detail, fidelity, and computational cost.
See all 248 papers