Organizations: School of Cyber Science and Engineering, Southeast University, Nanjing 210096, China. · Department of Electronic and Computer Engineering, the Hong Kong University of Science and Technology, Hong Kong, China. · College of Electronic and Information Engineering, Southwest University, Chongqing, China. · Institute of AI for Industries, Chinese Academy of Sciences, Nanjing, China. · Surrey Institute for People-Centred Artificial Intelligence and the Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, Guildford, U.K.
Generating consistent multi-view RGB images and normal maps from a single input image provides complementary appearance and geometric cues for 3D reconstruction. However, maintaining cross-view consistency requires information exchange across viewpoints, and dense cross-view attention becomes computationally expensive as the number of views and spatial resolution increase. In this paper, we propose Global-Local Proxy Attention (GLPA) to reduce cross-view communication costs while preserving fine-grained within-view processing. Specifically, GLPA retains full-resolution interactions within each view and uses compact global proxies for cross-view communication. To further leverage the exchanged information, GLPA incorporates iterative proxy refinement, which rebuilds the proxies from updated features and reuses the same attention parameters for subsequent interactions within a single denoising step. We evaluate GLPA on two structurally different diffusion backbones, FLUX and Stable Diffusion. Experiments on FLUX show that GLPA achieves a 2.65× speedup in attention computation per denoising step over dense attention, while improving reconstruction IoU from 0.6849 to 0.7019 and reducing NVS LPIPS from 0.086 to 0.076. Results on Stable Diffusion further demonstrate the effectiveness of GLPA, with a 1.80× attention speedup and improvements across all five quality metrics.
Figures & tables
Fig. 1: Consistent multi-view generation with GLPA. By combining global-local proxy interaction with iterative proxy refinement, GLPA improves cross-view consistency compared with reduced attention while requiring less attention computation than dense attention.
Fig. 2: Overall pipeline incorporating GLPA. GLPA combines global-local proxy interaction with iterative proxy refinement to preserve full-resolution within-view interactions and enable efficient cross-view communication. With GLPA integrated into multi-view diffusion, the pipeline generates consistent multi-view RGB images and normal maps from a single input image for downstream textured 3D reconstruction.
Fig. 3: Visualization of a single round of global-local proxy interaction in GLPA. Each row shows a query cell on the front view (red box), the within-view attention at this location, and the global attention of this cell over the proxies of three other views, displayed at their associated spatial locations. The proxy pathway reaches the corresponding parts across viewpoints, while the local pathway preserves fine within-view structure.
Fig. 4: Qualitative comparison with baseline methods on novel view synthesis.
Methods
CD ↓
IoU ↑
LPIPS ↓
SSIM ↑
PSNR ↑
RealFusion [ 46 ]
0.0819
0.2714
0.283
0.722
15.26
Zero-1-to-3 [ 6 ]
0.0339
0.5035
0.166
0.779
18.93
One-2-3-45 [ 47 ]
0.0629
0.4086
-
-
-
Shap-E [ 48 ]
0.0436
0.3584
-
-
-
Magic123 [ 49 ]
0.0516
0.4528
-
-
-
SyncDreamer [ 8 ]
0.0261
0.5421
0.146
0.798
20.05
TABLE I: Quantitative Comparison on the GSO Benchmark Using Chamfer Distance (CD) and IoU for 3D Reconstruction, and LPIPS, SSIM, and PSNR for Novel View Synthesis.
Fig. 5: Qualitative comparison of 3D reconstruction results on the GSO dataset. The GLPA-based pipeline generates the best results with more views and high resolution, compared with existing methods.
Fig. 6: Qualitative comparison of 3D reconstructions on in-the-wild data. Compared with previous methods, the GLPA-based pipeline produces more complete and higher-quality 3D models.
Methods
N-view
Resolution
Attention runtime/step (s)
CD ↓
IoU ↑
LPIPS ↓
SSIM ↑
PSNR ↑
FLUX.1-dev Backbone
Dense
8
512
1.637
0.0193
0.6849
0.086
0.922
28.03
GLPA (w/o refinement)
8
512
0.390
0.0246
0.6156
0.101
0.914
26.75
GLPA (reuse proxy)
8
512
0.598
0.0254
0.6246
0.087
0.920
27.44
GLPA
8
512
0.617
0.0185
0.7019
0.076
0.927
27.80
SD2.1-UnCLIP Backbone
TABLE II: Attention Runtime and Quality Comparison of Dense Attention and GLPA Variants on FLUX and SD Backbones.
s
R
Attention runtime/step (s)
CD ↓
IoU ↑
LPIPS ↓
SSIM ↑
PSNR ↑
2
1
0.390
0.0246
0.6156
0.101
0.914
26.75
2
2
0.617
0.0185
0.7019
0.076
0.927
27.80
2
3
0.799
0.0262
0.6210
0.084
0.921
27.41
4
2
0.370
0.0308
0.5578
0.096
0.917
27.47
8
2
0.324
0.0404
0.4496
0.125
0.908
25.75
TABLE III: Effect of Pooling Stride and Interaction Rounds on Attention Efficiency, Novel View Synthesis, and 3D Reconstruction.
Methods
Latency (ms)
Peak memory (G)
LPIPS ↓
SSIM ↑
PSNR ↑
Dense
26.61
0.828
0.101
0.918
25.84
PT-DiT [ 38 ]
35.41
1.913
0.145
0.901
24.39
Agent Attention [ 36 ]
21.74
1.986
0.137
0.909
24.84
GLPA
22.07
0.601
0.111
0.910
25.86
TABLE IV: Efficiency and Quality Comparison of Alternative Attention Designs on the FLUX Backbone.
Methods
N-view
Resolution
Latency (ms)
Peak memory (G)
Dense
8
512
14.92
0.244
Era3D [ 10 ]
8
512
1.67
0.322
CTR3D [ 11 ]
8
512
9.20
0.325
GLPA
8
512
5.02
0.156
TABLE V: Efficiency Comparison of Attention Modules on the SD Backbone.