LiveMoments: Reselected Key Photo Restoration in Live Photos via Reference-guided Diffusion

Clara Xue; Zizheng Yan; Zhenning Shi; Yuhang Yu; Jingyu Zhuang; Qi Zhang; Jinwei Chen; Qingnan Fan

cs.CVApr 14, 2026

LiveMoments: Reselected Key Photo Restoration in Live Photos via Reference-guided Diffusion

Authors: Clara Xue, Zizheng Yan, Zhenning Shi, Yuhang Yu, Jingyu Zhuang, Qi Zhang, Jinwei Chen, Qingnan Fan

Organizations: vivo BlueImage Lab · College of Computer Science, Nankai University

Abstract

Live Photo captures both a high-quality key photo and a short video clip to preserve the precious dynamics around the captured moment. While users may choose alternative frames as the key photo to capture better expressions or timing, these frames often exhibit noticeable quality degradation, as the photo capture ISP pipeline delivers significantly higher image quality than the video pipeline. This quality gap highlights the need for dedicated restoration techniques to enhance the reselected key photo. To this end, we propose LiveMoments, a reference-guided image restoration framework tailored for the reselected key photo in Live Photos. Our method employs a two-branch neural network: a reference branch that extracts structural and textural information from the original high-quality key photo, and a main branch that restores the reselected frame using the guidance provided by the reference branch. Furthermore, we introduce a unified Motion Alignment module that incorporates motion guidance for spatial alignment at both the latent and image levels. Experiments on real and synthetic Live Photos demonstrate that LiveMoments significantly improves perceptual quality and fidelity over existing solutions, especially in scenes with fast motion or complex structures. Our code is available at https://github.com/OpenVeraTeam/LiveMoments.

Figures & tables

Figure 2: Overall architecture of LiveMoments. After the fixed VAE encoder, the original key photo and the reselected LQ frame are fed into the ReferenceNet and RestorationNet, respectively, and fused via cross-attention. For latent-level motion alignment, the optical flow is estimated with a fixed RAFT model and encoded with a Motion Encoder, which is further injected into the cross-attention as an additive bias. For image-level alignment, Patch Correspondence Retrieval (PCR) strategy is adopted during the inference to ensure spatial consistency when using the tiling strategy.

Figure 3: Illustration of the proposed Patch Correspondence Retrieval (PCR) strategy. The average displacement within a patch (the dense displacement field contained in $P_{Ls}^{i}$ ) is used to shift the top-left corner $(x_{i},y_{i})$ to $(\hat{x}_{i},\hat{y}_{i})$ , then we crop the aligned reference patch $\hat{P}_{Ho}^{i}$ from $I_{Ho}$ . On the right, we compare the reselected patch $P_{Ls}^{i}$ , unaligned Ref patch $P_{Ho}^{i}$ and the aligned Ref patch $\hat{P}_{Ho}^{i}$ .

Method	vivoLive144						iPhoneLive90
Method	$\text{NI}_{re}\downarrow$	$\text{MU}_{re}\downarrow$	$\text{CA}_{re}\downarrow$	$\text{MA}_{re}\downarrow$	CLIP-Q $\uparrow$	DINO-Q $\uparrow$	$\text{NI}_{re}\downarrow$	$\text{MU}_{re}\downarrow$	$\text{CA}_{re}\downarrow$	$\text{MA}_{re}\downarrow$	CLIP-Q $\uparrow$	DINO-Q $\uparrow$
TTSR	0.2050	0.3545	0.1928	0.1400	0.9626	0.8304	0.3069	0.2858	0.2593	0.1974	0.9505	0.8689
C2-Matching	0.2047	0.3512	0.1929	0.1390	0.9623	0.8298	0.3147	0.2800	0.2791	0.1978	0.9505	0.8685
DATSR	0.2212	0.3573	0.1951	0.1386	0.9621	0.8261	0.3324	0.2859	0.2853	0.1963	0.9500	0.8657
MRefSR	0.2276	0.3537	0.1916	0.1371	0.9616	0.8216	0.3408	0.2845	0.2900	0.1978	0.9499	0.8652
CoSeR	0.1953	0.1865	0.2752	0.1080	0.9658	0.9197	0.1774	0.2492	0.1564	0.0784	0.9608	0.8618

Table 1: Quantitative comparison with RefSR and SISR methods on real-world Live Photo datasets. The best results are highlighted in bold . Here, NI re denotes

\text{NIQE}_{re}

, MU re denotes

\text{MUSIQ}_{re}

, CA re denotes CLIPIQA and MA re denotes

\text{MANIQA}_{re}

Panel of Figure 4 — Figure 4: Visual comparison on the two real-world Live Photo datasets: vivoLive144 (top) and iPhoneLive90 (bottom). The aligned original key photo is cropped manually for better comparison.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	FID $\downarrow$	$\text{NI}_{re}\downarrow$	$\text{MU}_{re}\downarrow$	$\text{CA}_{re}\downarrow$	$\text{MA}_{re}\downarrow$	CLIP-Q $\uparrow$	DINO-Q $\uparrow$
TTSR	31.74	0.8778	0.2416	0.1259	18.53	0.4254	0.3885	0.1996	0.2209	0.9621	0.8373
C2-Matching	31.85	0.8782	0.2419	0.1250	18.51	0.4354	0.3873	0.2051	0.2199	0.9619	0.8391
DATSR	31.84	0.8783	0.2417	0.1251	18.50	0.4380	0.3876	0.2034	0.2186	0.9618	0.8388
MRefSR	31.84	0.8782	0.2420	0.1252	18.73	0.4342	0.3886	0.2058	0.2195	0.9612	0.8374
CoSeR	27.60	0.8136	0.2436	0.1135	13.92	0.2603	0.2548	0.6380	0.1028	0.9699	0.8924
ReFIR (SeeSR)	27.44	0.7942	0.2873	0.1530	28.26	0.3378	0.4028	1.2814	0.1837	0.9558	0.7934

Table 2: Quantitative comparison with RefSR and SISR methods on the synthetic dataset SynLive260. The best results are highlighted in bold . Here, NI re denotes

\text{NIQE}_{re}

, MU re denotes

\text{MUSIQ}_{re}

, CA re denotes CLIPIQA and MA re denotes

\text{MANIQA}_{re}

Method	$\text{NIQE}_{re}\downarrow$	$\text{CLIPIQA}_{re}\downarrow$	$\text{MANIQA}_{re}\downarrow$	CLIP-Q $\uparrow$	DINO-Q $\uparrow$
RestorationNet	0.1677	0.2348	0.1105	0.9690	0.9081
[2pt/4pt] RestorationNet + ReferenceNet	0.1097	0.0823	0.0631	0.9792	0.9539
RestorationNet + ReferenceNet + warp RefImage	0.1034	0.0873	0.0573	0.9774	0.9480
RestorationNet + ReferenceNet + warp RefLatent	0.1130	0.0850	0.0622	0.9774	0.9437
RestorationNet + ReferenceNet + warp RefKV	0.1183	0.0853	0.0657	0.9776	0.9456
[2pt/4pt] LiveMoments (full model)	0.0990	0.0809	0.0556	0.9805	0.9629

Table 3: Ablation study of the network design on vivoLive144. All warp operations are applied on the original key photo, denoted as the reference image (Ref). The best results are highlighted in bold .

Figure 5: (a) Comparison between no-reference metrics, the proposed relative no-reference metrics, and human visual preference in evaluating the proposed task. The aligned key photo is cropped manually for better comparison. (b) Comparison between LiveMoments with (w) and without (w/o) the proposed Patch Correspondence Retrieval (PCR) strategy when processing 4K images. Patches are cropped for clearer visualization.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Parameter Counts (M)	Parameter-only VRAM (MB)	Inference-time Peak VRAM (MB)
VAE	83.82	159.87	12368.33
ReferenceNet	2028.33	3868.73	10090.78
RestorationNet	2084.95	3976.73	10091.28
RAFT	5.26	10.03	13180.38
Motion Encoder	66.09	126.06	9940.00

Appendix

Table 4: Comparisons on branch-wise parameter counts and VRAM, measured when processing a 4K Live Photo.

Type	Method	Parameter Counts (M)	TFLOPs	Peak Memory (GB)	Inference Time (s)
Time-Step Distilled Methods	OSEDiff	1294.38 + 470.93 (DAPE)	20.53	8.66	0.43
Time-Step Distilled Methods	TSD-SR	2207.33	21.91	8.60	0.24
Non-Distilled Methods	CoSeR	2655.52	–	32.74	48.83
	ReFIR (SeeSR)	2039.83 + 470.93 (DAPE)	1560.70	17.83	27.62
	ReFIR (SUPIR)	4801.18	2672.70	59.82	31.67
[2pt/4pt]	StableSR	1554.64	2236.31	34.53	164.67

Appendix

Table 5: Comparisons with RefISR and SISR methods on parameter counts, FLOPs, peak memory usage and inference time at a resolution of

1024\times 1024

on a single NVIDIA H20 GPU.

Metric	vivoLive144		iPhoneLive90
Metric	Spearman $\uparrow$	Pearson $\uparrow$	Spearman $\uparrow$	Pearson $\uparrow$
$\text{NIQE}_{re}$	0.493	0.518	0.604	0.560
$\text{MUSIQ}_{re}$	0.585	0.543	0.414	0.440
$\text{CLIPIQA}_{re}$	0.558	0.542	0.486	0.495
$\text{MANIQA}_{re}$	0.535	0.548	0.370	0.413

Appendix

Table 6: Spearman and Pearson correlations between relative no-reference metrics and human perception on real-world Live Photo datasets.

Panel of Figure 7 — Figure 7: Comparison between no-reference metrics, the proposed relative no-reference metrics, and human visual preference in evaluating the key photo reselection task. The aligned key photo is cropped manually for better comparison.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	$\text{CA}_{re}\downarrow$	$\text{MA}_{re}\downarrow$	CLIP-Q $\uparrow$	DINO-Q $\uparrow$
RestorationNet	30.42	0.8610	0.1685	0.0793	0.1721	0.0927	0.9841	0.9195
[2pt/4pt] RestorationNet + ReferenceNet	31.47	0.8980	0.0837	0.0388	0.1246	0.0586	0.9949	0.9643
RestorationNet + ReferenceNet + warp RefImage	31.49	0.8927	0.0968	0.0475	0.1569	0.0546	0.9934	0.9556
RestorationNet + ReferenceNet + warp RefLatent	31.84	0.8986	0.0924	0.0441	0.1305	0.0618	0.9936	0.9566
RestorationNet + ReferenceNet + warp RefKV	31.82	0.8991	0.0895	0.0435	0.1306	0.0645	0.9940	0.9580
[2pt/4pt] LiveMoments (full model)	31.65	0.8990	0.0828	0.0365	0.1155	0.0495	0.9950	0.9740

Appendix

Table 7: Ablation study of the network design on SynLive260. All warp operations are applied on the original key photo, denoted as the reference image (Ref). The best results are highlighted in bold . Here, CA re denotes CLIPIQA and MA re denotes

\text{MANIQA}_{re}

Degradation Type	$\text{NIQE}_{re}\downarrow$	$\text{MUSIQ}_{re}\downarrow$	$\text{CLIPIQA}_{re}\downarrow$	$\text{MANIQA}_{re}\downarrow$	CLIP-Q $\uparrow$	DINO-Q $\uparrow$
Moderate-StableSR	0.0925	0.1086	0.0883	0.0673	0.9786	0.9550
StableSR	0.0933	0.1165	0.1056	0.0734	0.9791	0.9581
SeeSR	0.0904	0.1030	0.0812	0.0647	0.9788	0.9505
[2pt/4pt] Customized Live Photo Degradation	0.0990	0.0893	0.0809	0.0556	0.9805	0.9629

Appendix

Table 8: Ablation study of different training degradation settings on vivoLive144. “Customized Live Photo degradation” is the setting used in the main experiments. Best results are highlighted in bold .

Panel of Figure 8 — Figure 8: Comparison between LiveMoments with (w) and without (w/o) the proposed Patch Correspondence Retrieval (PCR) strategy when processing 4K images. Patches are cropped for clearer visualization. Please zoom in for better view.

Method	$\text{NIQE}_{re}\downarrow$	$\text{MUSIQ}_{re}\downarrow$	$\text{CLIPIQA}_{re}\downarrow$	$\text{MANIQA}_{re}\downarrow$	CLIP-Q $\uparrow$	DINO-Q $\uparrow$
LiveMoments + 10% noise	0.0999	0.0921	0.0808	0.0561	0.9802	0.9622
LiveMoments + 20% noise	0.1035	0.0935	0.0802	0.0551	0.9799	0.9613
LiveMoments + 40% noise	0.1082	0.0993	0.0811	0.0546	0.9792	0.9591
LiveMoments + 80% noise	0.1108	0.1072	0.0845	0.0548	0.9785	0.9555
LiveMoments + 100% noise	0.1116	0.1099	0.0862	0.0566	0.9782	0.9540
LiveMoments + 200% noise	0.1159	0.1222	0.0915	0.0578	0.9771	0.9497

Appendix

Table 9: Ablation study of robustness analysis under flow noise injection and flow estimator replacement on vivoLive144. Best results are highlighted in bold .

Panel of Figure 9 — Figure 9: Comparison between different degradation settings. The aligned LQ frame is cropped manually for better comparison. Please zoom in for better view.

Panel of Figure 10 — Figure 10: Robust cases with inaccurate motion alignment. The aligned original key photo is cropped manually for better comparison. Please zoom in for better view.

Panel of Figure 11 — Figure 11: Failure cases with inaccurate motion alignment. The aligned original key photo is cropped manually for better comparison. Please zoom in for better view.

Panel of Figure 12 — Figure 12: More visual comparisons of RefISR, RefVSR and SISR methods on vivoLive144 dataset. The aligned original key photo is cropped manually for comparison. Please zoom in for a better view.

Figure 13: High-resolution visual comparisons of diffusion-based RefISR methods on vivoLive144 dataset. The aligned original key photo is cropped manually for better comparison.

Figure 14: High-resolution visual comparisons of diffusion-based RefISR methods on vivoLive144 dataset. The aligned original key photo is cropped manually for better comparison.

Panel of Figure 15 — Figure 15: More visual comparisons of other RefISR, RefVSR and SISR methods on vivoLive144 dataset. The aligned original key photo is cropped manually for better comparison. Please zoom in for a better view.

Figure 16: High-resolution visual comparisons of diffusion-based RefISR methods on vivoLive144 dataset. The aligned original key photo is cropped manually for better comparison.

Panel of Figure 17 — Figure 17: More visual comparisons of RefISR, RefVSR and SISR methods on iPhoneLive90 dataset. The aligned original key photo is cropped manually for comparison. Please zoom in for a better view.

Figure 18: High-resolution visual comparisons of diffusion-based RefISR methods on iPhoneLive90 dataset. The aligned original key photo is cropped manually for better comparison.

Panel of Figure 19 — Figure 19: More visual comparisons of other RefISR, RefVSR and SISR methods on iPhoneLive90 dataset. The aligned original key photo is cropped manually for better comparison. Please zoom in for a better view.

Figure 20: High-resolution visual comparisons of diffusion-based RefISR methods on SynLive260 dataset.

Panel of Figure 21 — Figure 21: More visual comparisons of RefISR, RefVSR and SISR methods on SynLive260 dataset. Please zoom in for a better view.

Geometry-Guided Image Restoration

See all 15 papers

Jul 5, 2026

Perceiving Better Moments: Cover Frame Reselection and Enhancement for Live Photos with the Live2K Dataset

Modern smartphones capture Live Photos, short video bursts surrounding a still image, offering a dynamic and engaging photographic experience. However, the cover photo and video components are generated by two distinct imaging pipelines: the photo stream undergoes full computational photography processing, while the video stream is constrained by real-time efficiency and heavy compression. This intrinsic separation produces a substantial quality gap in resolution, color fidelity, and dynamic range between the cover photo and video frames. When users reselect an alternative frame from the video to replace an imperfect cover, the chosen frame often suffers from severe degradation, making direct replacement visually unsatisfactory. Restoring such frames requires simultaneous enhancement of spatial detail and color appearance, a task considerably more challenging than ordinary super-resolution or color enhancement. To address this, we define the Live Photo Cover Frame Reselection and Enhancement (LPRE) task, which leverages the intrinsic cues available within each Live Photo: the high-quality cover image as a structural and color reference, the user-reselected low-quality frame as the reconstruction target and several adjacent video frames providing temporal cues. Building upon this formulation, we construct Live2K, a real-world dataset of 2,042 Live Photos, and develop a unified one-stage baseline that integrates multi-frame fusion, guided color enhancement and super-resolution, establishing the first benchmark for Live Photo enhancement research.

Junyu Lou, Kai Chen, Weiyi You +3

University of Electronic Science and Technology of China · OPPO Research Institute · The Hong Kong Polytechnic University

Image Enhancement

Ask PDF

Mar 23, 2026

Dynamic Exposure Burst Image Restoration

Burst image restoration aims to reconstruct a high-quality image from burst images, which are typically captured using manually designed exposure settings. Although these exposure settings significantly influence the final restoration performance, the problem of finding optimal exposure settings has been overlooked. In this paper, we present Dynamic Exposure Burst Image Restoration (DEBIR), a novel burst image restoration pipeline that enhances restoration quality by dynamically predicting exposure times tailored to the shooting environment. In our pipeline, Burst Auto-Exposure Network (BAENet) estimates the optimal exposure time for each burst image based on a preview image, as well as motion magnitude and gain. Subsequently, a burst image restoration network reconstructs a high-quality image from burst images captured using these optimal exposure times. For training, we introduce a differentiable burst simulator and a three-stage training strategy. Our experiments demonstrate that our pipeline achieves state-of-the-art restoration quality. Furthermore, we validate the effectiveness of our approach on a real-world camera system, demonstrating its practicality.

Woohyeok Kim, Jaesung Rim, Daeyeon Kim +1

POSTECH

Image Restoration Computational Imaging

Ask PDF

Aug 27, 2026

Zero-Shot Video Restoration and Enhancement with Text-to-Image Latent Diffusion Models and Multi-Modal References

Zero-shot image restoration methods with text-to-image latent diffusion models have achieved great success in universal image restoration tasks without training. However, applying them to video restoration will result in severe temporal flickering. In this paper, we propose a novel framework for zero-shot video restoration and enhancement which uses a text-to-image latent diffusion model and multi-modal references. Through the proposed dual prompt tuning inversion and sampling, the inference time can be reduced to nearly 1/3 of the original. The performance and temporal consistency can be also significantly stregthened. By using the proposed texture-aware video token merging, the temporal correlation between frames can be further utilized to improve the temporal consistency. We futher propose the referenced self-attention and referenced token merging to support image reference. Experimental results demonstrate the superiority of the proposed method in restoring and enhancing temporally consistent videos.

Cong Cao, Huanjing Yue, Xin Liu +1

School of Electrical and Information Engineering, Tianjin University, Tianjin, China · Computer Vision and Pattern Recognition Laboratory, School of Engineering Science, Lappeenranta-Lahti University of Technology LUT, Lappeenranta, Finland

Video Restoration T2I Diffusion Models

Ask PDF

Abstract

Geometry-Guided Image Restoration

Explore similar work

Perceiving Better Moments: Cover Frame Reselection and Enhancement for Live Photos with the Live2K Dataset

Dynamic Exposure Burst Image Restoration

Zero-Shot Video Restoration and Enhancement with Text-to-Image Latent Diffusion Models and Multi-Modal References