LiveMoments: Reselected Key Photo Restoration in Live Photos via Reference-guided Diffusion

Clara Xue; Zizheng Yan; Zhenning Shi; Yuhang Yu; Jingyu Zhuang; Qi Zhang; Jinwei Chen; Qingnan Fan

cs.CVApr 14, 2026

LiveMoments: Reselected Key Photo Restoration in Live Photos via Reference-guided Diffusion

Authors: Clara Xue, Zizheng Yan, Zhenning Shi, Yuhang Yu, Jingyu Zhuang, Qi Zhang, Jinwei Chen, Qingnan Fan

Organizations: vivo BlueImage Lab · College of Computer Science, Nankai University

Abstract

Live Photo captures both a high-quality key photo and a short video clip to preserve the precious dynamics around the captured moment. While users may choose alternative frames as the key photo to capture better expressions or timing, these frames often exhibit noticeable quality degradation, as the photo capture ISP pipeline delivers significantly higher image quality than the video pipeline. This quality gap highlights the need for dedicated restoration techniques to enhance the reselected key photo. To this end, we propose LiveMoments, a reference-guided image restoration framework tailored for the reselected key photo in Live Photos. Our method employs a two-branch neural network: a reference branch that extracts structural and textural information from the original high-quality key photo, and a main branch that restores the reselected frame using the guidance provided by the reference branch. Furthermore, we introduce a unified Motion Alignment module that incorporates motion guidance for spatial alignment at both the latent and image levels. Experiments on real and synthetic Live Photos demonstrate that LiveMoments significantly improves perceptual quality and fidelity over existing solutions, especially in scenes with fast motion or complex structures. Our code is available at https://github.com/OpenVeraTeam/LiveMoments.

Figures & tables

Figure 2: Overall architecture of LiveMoments. After the fixed VAE encoder, the original key photo and the reselected LQ frame are fed into the ReferenceNet and RestorationNet, respectively, and fused via cross-attention. For latent-level motion alignment, the optical flow is estimated with a fixed RAFT model and encoded with a Motion Encoder, which is further injected into the cross-attention as an additive bias. For image-level alignment, Patch Correspondence Retrieval (PCR) strategy is adopted during the inference to ensure spatial consistency when using the tiling strategy.

Figure 3: Illustration of the proposed Patch Correspondence Retrieval (PCR) strategy. The average displacement within a patch (the dense displacement field contained in $P_{Ls}^{i}$ ) is used to shift the top-left corner $(x_{i},y_{i})$ to $(\hat{x}_{i},\hat{y}_{i})$ , then we crop the aligned reference patch $\hat{P}_{Ho}^{i}$ from $I_{Ho}$ . On the right, we compare the reselected patch $P_{Ls}^{i}$ , unaligned Ref patch $P_{Ho}^{i}$ and the aligned Ref patch $\hat{P}_{Ho}^{i}$ .

Method	vivoLive144						iPhoneLive90
Method	$\text{NI}_{re}\downarrow$	$\text{MU}_{re}\downarrow$	$\text{CA}_{re}\downarrow$	$\text{MA}_{re}\downarrow$	CLIP-Q $\uparrow$	DINO-Q $\uparrow$	$\text{NI}_{re}\downarrow$	$\text{MU}_{re}\downarrow$	$\text{CA}_{re}\downarrow$	$\text{MA}_{re}\downarrow$	CLIP-Q $\uparrow$	DINO-Q $\uparrow$
TTSR	0.2050	0.3545	0.1928	0.1400	0.9626	0.8304	0.3069	0.2858	0.2593	0.1974	0.9505	0.8689
C2-Matching	0.2047	0.3512	0.1929	0.1390	0.9623	0.8298	0.3147	0.2800	0.2791	0.1978	0.9505	0.8685
DATSR	0.2212	0.3573	0.1951	0.1386	0.9621	0.8261	0.3324	0.2859	0.2853	0.1963	0.9500	0.8657
MRefSR	0.2276	0.3537	0.1916	0.1371	0.9616	0.8216	0.3408	0.2845	0.2900	0.1978	0.9499	0.8652
CoSeR	0.1953	0.1865	0.2752	0.1080	0.9658	0.9197	0.1774	0.2492	0.1564	0.0784	0.9608	0.8618

Table 1: Quantitative comparison with RefSR and SISR methods on real-world Live Photo datasets. The best results are highlighted in bold . Here, NI re denotes

\text{NIQE}_{re}

, MU re denotes

\text{MUSIQ}_{re}

, CA re denotes CLIPIQA and MA re denotes

\text{MANIQA}_{re}

Panel of Figure 4 — Figure 4: Visual comparison on the two real-world Live Photo datasets: vivoLive144 (top) and iPhoneLive90 (bottom). The aligned original key photo is cropped manually for better comparison.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	FID $\downarrow$	$\text{NI}_{re}\downarrow$	$\text{MU}_{re}\downarrow$	$\text{CA}_{re}\downarrow$	$\text{MA}_{re}\downarrow$	CLIP-Q $\uparrow$	DINO-Q $\uparrow$
TTSR	31.74	0.8778	0.2416	0.1259	18.53	0.4254	0.3885	0.1996	0.2209	0.9621	0.8373
C2-Matching	31.85	0.8782	0.2419	0.1250	18.51	0.4354	0.3873	0.2051	0.2199	0.9619	0.8391
DATSR	31.84	0.8783	0.2417	0.1251	18.50	0.4380	0.3876	0.2034	0.2186	0.9618	0.8388
MRefSR	31.84	0.8782	0.2420	0.1252	18.73	0.4342	0.3886	0.2058	0.2195	0.9612	0.8374
CoSeR	27.60	0.8136	0.2436	0.1135	13.92	0.2603	0.2548	0.6380	0.1028	0.9699	0.8924
ReFIR (SeeSR)	27.44	0.7942	0.2873	0.1530	28.26	0.3378	0.4028	1.2814	0.1837	0.9558	0.7934

Table 2: Quantitative comparison with RefSR and SISR methods on the synthetic dataset SynLive260. The best results are highlighted in bold . Here, NI re denotes

\text{NIQE}_{re}

, MU re denotes

\text{MUSIQ}_{re}

, CA re denotes CLIPIQA and MA re denotes

\text{MANIQA}_{re}

Method	$\text{NIQE}_{re}\downarrow$	$\text{CLIPIQA}_{re}\downarrow$	$\text{MANIQA}_{re}\downarrow$	CLIP-Q $\uparrow$	DINO-Q $\uparrow$
RestorationNet	0.1677	0.2348	0.1105	0.9690	0.9081
[2pt/4pt] RestorationNet + ReferenceNet	0.1097	0.0823	0.0631	0.9792	0.9539
RestorationNet + ReferenceNet + warp RefImage	0.1034	0.0873	0.0573	0.9774	0.9480
RestorationNet + ReferenceNet + warp RefLatent	0.1130	0.0850	0.0622	0.9774	0.9437
RestorationNet + ReferenceNet + warp RefKV	0.1183	0.0853	0.0657	0.9776	0.9456
[2pt/4pt] LiveMoments (full model)	0.0990	0.0809	0.0556	0.9805	0.9629

Table 3: Ablation study of the network design on vivoLive144. All warp operations are applied on the original key photo, denoted as the reference image (Ref). The best results are highlighted in bold .

Figure 5: (a) Comparison between no-reference metrics, the proposed relative no-reference metrics, and human visual preference in evaluating the proposed task. The aligned key photo is cropped manually for better comparison. (b) Comparison between LiveMoments with (w) and without (w/o) the proposed Patch Correspondence Retrieval (PCR) strategy when processing 4K images. Patches are cropped for clearer visualization.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Parameter Counts (M)	Parameter-only VRAM (MB)	Inference-time Peak VRAM (MB)
VAE	83.82	159.87	12368.33
ReferenceNet	2028.33	3868.73	10090.78
RestorationNet	2084.95	3976.73	10091.28
RAFT	5.26	10.03	13180.38
Motion Encoder	66.09	126.06	9940.00

Appendix

Table 4: Comparisons on branch-wise parameter counts and VRAM, measured when processing a 4K Live Photo.

Type	Method	Parameter Counts (M)	TFLOPs	Peak Memory (GB)	Inference Time (s)
Time-Step Distilled Methods	OSEDiff	1294.38 + 470.93 (DAPE)	20.53	8.66	0.43
Time-Step Distilled Methods	TSD-SR	2207.33	21.91	8.60	0.24
Non-Distilled Methods	CoSeR	2655.52	–	32.74	48.83
	ReFIR (SeeSR)	2039.83 + 470.93 (DAPE)	1560.70	17.83	27.62
	ReFIR (SUPIR)	4801.18	2672.70	59.82	31.67
[2pt/4pt]	StableSR	1554.64	2236.31	34.53	164.67

Appendix

Table 5: Comparisons with RefISR and SISR methods on parameter counts, FLOPs, peak memory usage and inference time at a resolution of

1024\times 1024

on a single NVIDIA H20 GPU.

Metric	vivoLive144		iPhoneLive90
Metric	Spearman $\uparrow$	Pearson $\uparrow$	Spearman $\uparrow$	Pearson $\uparrow$
$\text{NIQE}_{re}$	0.493	0.518	0.604	0.560
$\text{MUSIQ}_{re}$	0.585	0.543	0.414	0.440
$\text{CLIPIQA}_{re}$	0.558	0.542	0.486	0.495
$\text{MANIQA}_{re}$	0.535	0.548	0.370	0.413

Appendix

Table 6: Spearman and Pearson correlations between relative no-reference metrics and human perception on real-world Live Photo datasets.

Panel of Figure 7 — Figure 7: Comparison between no-reference metrics, the proposed relative no-reference metrics, and human visual preference in evaluating the key photo reselection task. The aligned key photo is cropped manually for better comparison.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DISTS $\downarrow$	$\text{CA}_{re}\downarrow$	$\text{MA}_{re}\downarrow$	CLIP-Q $\uparrow$	DINO-Q $\uparrow$
RestorationNet	30.42	0.8610	0.1685	0.0793	0.1721	0.0927	0.9841	0.9195
[2pt/4pt] RestorationNet + ReferenceNet	31.47	0.8980	0.0837	0.0388	0.1246	0.0586	0.9949	0.9643
RestorationNet + ReferenceNet + warp RefImage	31.49	0.8927	0.0968	0.0475	0.1569	0.0546	0.9934	0.9556
RestorationNet + ReferenceNet + warp RefLatent	31.84	0.8986	0.0924	0.0441	0.1305	0.0618	0.9936	0.9566
RestorationNet + ReferenceNet + warp RefKV	31.82	0.8991	0.0895	0.0435	0.1306	0.0645	0.9940	0.9580
[2pt/4pt] LiveMoments (full model)	31.65	0.8990	0.0828	0.0365	0.1155	0.0495	0.9950	0.9740

Appendix

Table 7: Ablation study of the network design on SynLive260. All warp operations are applied on the original key photo, denoted as the reference image (Ref). The best results are highlighted in bold . Here, CA re denotes CLIPIQA and MA re denotes

\text{MANIQA}_{re}

Degradation Type	$\text{NIQE}_{re}\downarrow$	$\text{MUSIQ}_{re}\downarrow$	$\text{CLIPIQA}_{re}\downarrow$	$\text{MANIQA}_{re}\downarrow$	CLIP-Q $\uparrow$	DINO-Q $\uparrow$
Moderate-StableSR	0.0925	0.1086	0.0883	0.0673	0.9786	0.9550
StableSR	0.0933	0.1165	0.1056	0.0734	0.9791	0.9581
SeeSR	0.0904	0.1030	0.0812	0.0647	0.9788	0.9505
[2pt/4pt] Customized Live Photo Degradation	0.0990	0.0893	0.0809	0.0556	0.9805	0.9629

Appendix

Table 8: Ablation study of different training degradation settings on vivoLive144. “Customized Live Photo degradation” is the setting used in the main experiments. Best results are highlighted in bold .

Panel of Figure 8 — Figure 8: Comparison between LiveMoments with (w) and without (w/o) the proposed Patch Correspondence Retrieval (PCR) strategy when processing 4K images. Patches are cropped for clearer visualization. Please zoom in for better view.

Method	$\text{NIQE}_{re}\downarrow$	$\text{MUSIQ}_{re}\downarrow$	$\text{CLIPIQA}_{re}\downarrow$	$\text{MANIQA}_{re}\downarrow$	CLIP-Q $\uparrow$	DINO-Q $\uparrow$
LiveMoments + 10% noise	0.0999	0.0921	0.0808	0.0561	0.9802	0.9622
LiveMoments + 20% noise	0.1035	0.0935	0.0802	0.0551	0.9799	0.9613
LiveMoments + 40% noise	0.1082	0.0993	0.0811	0.0546	0.9792	0.9591
LiveMoments + 80% noise	0.1108	0.1072	0.0845	0.0548	0.9785	0.9555
LiveMoments + 100% noise	0.1116	0.1099	0.0862	0.0566	0.9782	0.9540
LiveMoments + 200% noise	0.1159	0.1222	0.0915	0.0578	0.9771	0.9497

Appendix

Table 9: Ablation study of robustness analysis under flow noise injection and flow estimator replacement on vivoLive144. Best results are highlighted in bold .

Panel of Figure 9 — Figure 9: Comparison between different degradation settings. The aligned LQ frame is cropped manually for better comparison. Please zoom in for better view.

Panel of Figure 10 — Figure 10: Robust cases with inaccurate motion alignment. The aligned original key photo is cropped manually for better comparison. Please zoom in for better view.

Panel of Figure 11 — Figure 11: Failure cases with inaccurate motion alignment. The aligned original key photo is cropped manually for better comparison. Please zoom in for better view.

Panel of Figure 12 — Figure 12: More visual comparisons of RefISR, RefVSR and SISR methods on vivoLive144 dataset. The aligned original key photo is cropped manually for comparison. Please zoom in for a better view.

Figure 13: High-resolution visual comparisons of diffusion-based RefISR methods on vivoLive144 dataset. The aligned original key photo is cropped manually for better comparison.

Figure 14: High-resolution visual comparisons of diffusion-based RefISR methods on vivoLive144 dataset. The aligned original key photo is cropped manually for better comparison.

Panel of Figure 15 — Figure 15: More visual comparisons of other RefISR, RefVSR and SISR methods on vivoLive144 dataset. The aligned original key photo is cropped manually for better comparison. Please zoom in for a better view.

Figure 16: High-resolution visual comparisons of diffusion-based RefISR methods on vivoLive144 dataset. The aligned original key photo is cropped manually for better comparison.

Panel of Figure 17 — Figure 17: More visual comparisons of RefISR, RefVSR and SISR methods on iPhoneLive90 dataset. The aligned original key photo is cropped manually for comparison. Please zoom in for a better view.

Figure 18: High-resolution visual comparisons of diffusion-based RefISR methods on iPhoneLive90 dataset. The aligned original key photo is cropped manually for better comparison.

Panel of Figure 19 — Figure 19: More visual comparisons of other RefISR, RefVSR and SISR methods on iPhoneLive90 dataset. The aligned original key photo is cropped manually for better comparison. Please zoom in for a better view.

Figure 20: High-resolution visual comparisons of diffusion-based RefISR methods on SynLive260 dataset.

Panel of Figure 21 — Figure 21: More visual comparisons of RefISR, RefVSR and SISR methods on SynLive260 dataset. Please zoom in for a better view.

Geometry-Guided Image Restoration

See all 16 papers

Perceiving Better Moments: Cover Frame Reselection and Enhancement for Live Photos with the Live2K Dataset
Jul 5, 2026Junyu Lou, Kai Chen, Weiyi You +3Image Enhancement
University of Electronic Science and Technology of China · OPPO Research Institute · The Hong Kong Polytechnic University
Dynamic Exposure Burst Image Restoration
Mar 23, 2026Woohyeok Kim, Jaesung Rim, Daeyeon Kim +1Image Restoration Computational Imaging
POSTECH
Zero-Shot Video Restoration and Enhancement with Text-to-Image Latent Diffusion Models and Multi-Modal References
Aug 27, 2026Cong Cao, Huanjing Yue, Xin Liu +1Video Restoration T2I Diffusion Models
School of Electrical and Information Engineering, Tianjin University, Tianjin, China · Computer Vision and Pattern Recognition Laboratory, School of Engineering Science, Lappeenranta-Lahti University of Technology LUT, Lappeenranta, Finland

Jul 5, 2026J/K move · Enter open · S save

Perceiving Better Moments: Cover Frame Reselection and Enhancement for Live Photos with the Live2K Dataset

Junyu Lou, Kai Chen, Weiyi You, Hui Zeng+2

University of Electronic Science and Technology of China · OPPO Research Institute · The Hong Kong Polytechnic University

Image Enhancement

Open paper AskPDF arXiv

Modern smartphones capture Live Photos, short video bursts surrounding a still image, offering a dynamic and engaging photographic experience. However, the cover photo and video components are generated by two distinct imaging pipelines: the photo stream undergoes full computational photography processing, while the video stream is constrained by real-time efficiency and heavy compression. This intrinsic separation produces a substantial quality gap in resolution, color fidelity, and dynamic range between the cover photo and video frames. When users reselect an alternative frame from the video to replace an imperfect cover, the chosen frame often suffers from severe degradation, making direct replacement visually unsatisfactory. Restoring such frames requires simultaneous enhancement of spatial detail and color appearance, a task considerably more challenging than ordinary super-resolution or color enhancement. To address this, we define the Live Photo Cover Frame Reselection and Enhancement (LPRE) task, which leverages the intrinsic cues available within each Live Photo: the high-quality cover image as a structural and color reference, the user-reselected low-quality frame as the reconstruction target and several adjacent video frames providing temporal cues. Building upon this formulation, we construct Live2K, a real-world dataset of 2,042 Live Photos, and develop a unified one-stage baseline that integrates multi-frame fusion, guided color enhancement and super-resolution, establishing the first benchmark for Live Photo enhancement research.

Abstract

Geometry-Guided Image Restoration

Explore similar work

Perceiving Better Moments: Cover Frame Reselection and Enhancement for Live Photos with the Live2K Dataset

Dynamic Exposure Burst Image Restoration

Zero-Shot Video Restoration and Enhancement with Text-to-Image Latent Diffusion Models and Multi-Modal References