Generative Phomosaic with Structure-Aligned and Personalized Diffusion

Jaeyoung Chung; Hyunjin Son; Kyoung Mu Lee

cs.CVApr 8, 2026

Generative Phomosaic with Structure-Aligned and Personalized Diffusion

Authors: Jaeyoung Chung, Hyunjin Son, Kyoung Mu Lee

Organizations: Seoul National University, Seoul, South Korea

Abstract

We present the first generative approach to photomosaic creation. Traditional photomosaic methods rely on a large number of tile images and color-based matching, which limits both diversity and structural consistency. Our generative photomosaic framework synthesizes tile images using diffusion-based generation conditioned on reference images. A low-frequency conditioned diffusion mechanism aligns global structure while preserving prompt-driven details. This generative formulation enables photomosaic composition that is both semantically expressive and structurally coherent, effectively overcoming the fundamental limitations of matching-based approaches. By leveraging few-shot personalized diffusion, our model is able to produce user-specific or stylistically consistent tiles without requiring an extensive collection of images.

Figures & tables

Figure 2: Method Overview. We generate photomosaic images using a diffusion-based framework. During the noise initialization stage, each partial reference block (e.g. $96\times 96$ ) is expanded to proper resolution (e.g. $768\times 768$ ) using an integral-noise subsampling. At every denoising step, we align the color distribution of the evolving tile image with its corresponding reference block and apply a low-frequency guidance term to preserve coarse structural layout. As a result, the model produces tile images that both follow the structure of the reference image and comply with the target prompt.

Figure 3: Qualitative Results across Different Mosaic Levels.

	Ref. Image (Global)					Tile Image (Local)
	PSNR ↑	SSIM ↑	LPIPS ↓	HPSv2 ↑	IR ↑	BLIP ↑	CLIP ↑	IQA ↑	HPSv2 ↑	IR ↑
Match & Tone [ 16 ]	16.719	0.366	0.206	0.129	-1.783	0.625	0.612	0.596	0.174	-1.145
Color ControlNet [ 21 ]	10.145	0.080	0.435	0.123	-2.272	0.639	0.613	0.811	0.266	0.472
AdaIN [ 11 ]	12.983	0.204	0.400	0.130	-2.279	0.577	0.614	0.834	0.244	-0.106
NoiseBlend [ 17 ]	11.590	0.205	0.479	0.117	-2.275	0.239	0.597	0.385	0.195	-1.851
StreamDiffI2I [ 13 ]	15.212	0.414	0.272	0.127	-1.377	0.166	0.602	0.259	0.173	-1.073

Table 1: Quantitative results

Figure 6: Qualitative Results of Generative Photomosaic.

	Ref. Image (Global)			Tile Image (Local)
	PSNR ↑	SSIM ↑	LPIPS ↓	BLIP ↑	CLIP ↑	IQA ↑
Ours	17.598	0.551	0.121	0.569	0.619	0.812
w/o sec 3.1	16.224	0.510	0.186	0.562	0.611	0.806
w/o sec 3.2	16.431	0.495	0.161	0.566	0.612	0.564
w/o sec 3.3	13.142	0.314	0.398	0.567	0.614	0.810

Table 2: Ablation Results.

Panel of Figure 7 — Figure 7: Visualization of Ablation Results.

	Ref. Image (Global)			Tile Image (Local)
	PSNR ↑	SSIM ↑	LPIPS ↓	BLIP ↑	CLIP ↑	IQA ↑
Ours	17.598	0.551	0.121	0.569	0.619	0.812
LPIPS-guided	12.242	0.216	0.456	0.373	0.603	0.656
YCbCr-guided	14.107	0.361	0.337	0.548	0.611	0.801

Table 3: Guidance Ablation Results.

Panel of Figure 8 — Figure 8: Visualization of Guidance Ablation Results.

Figure 9: Generation process of local tile and its corresponding reference block.

Panel of Figure 10 — Figure 10: Ablation Results on the Guidance Weight Magnitude.

Panel of Figure 11 — Figure 11: Application : Personalized Generative Photomosaic.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 12: Global and Local Prompt Sets for Quantitative Evaluation.

	Ref. Image (Global)					Tile Image (Local)
	PSNR ↑	SSIM ↑	LPIPS ↓	HPSv2 ↑	IR ↑	BLIP ↑	CLIP ↑	IQA ↑	HPSv2 ↑	IR ↑
FD	27.007	0.937	0.010	0.134	-2.174	0.444	0.602	0.383	0.174	-1.574
Ours	17.598	0.551	0.121	0.132	-2.225	0.569	0.619	0.812	0.239	-0.317

Appendix

Table 4: Comparison to Factorized Diffusion.

Figure 13: Qualitative Result of Factorized Diffusion.

	$Level$ -1	$Level$ -2	$Level$ -3	$Level$ -4
Match & Tone [15]	0.026	0.016	0.012	0.008
Color ControlNet [20]	0.107	0.088	0.072	0.054
AdaIN [10]	0.044	0.025	0.018	0.012
NoiseBlend [16]	0.067	0.014	0.008	0.005
StreamDiffI2I [12]	0.025	0.016	0.013	0.011
Ours	0.025	0.009	0.005	0.003

Appendix

Table 5: Low-Frequency Alignment Evaluation.

	M&T	ControlNet	AdaIN	NoiseBlend	StreamDiffI2I	FD	Ours
runtime(s)	1.8	127	189	187	3.9	436	1088

Appendix

Table 6: Runtime Comparison.

Diffusion-Based Image Generation

Diffusion-based image generation creates images by progressively denoising noise, with controls for content, style, and structure.

See all 327 papers

PhotoQuilt: Training-Free Arbitrary-Resolution Photomosaics via Bootstrapped Tiled Denoising
Jun 29, 2026Koorosh Roohi, Javad Rajabi, Andrew Fleet +1Image Generation Compositional Image Generation
University of Toronto · Vector Institute · KITE Research Institute +2
Efficient and Training-Free Single-Image Diffusion Models
Jun 3, 2026Haojun Qiu, Kiriakos N. Kutulakos, David B. LindellImage Generation Diffusion-Based Image Generation
Department of Computer Science, University of Toronto · Vector Institute
Controllable Image Generation with Composed Parallel Token Prediction
Apr 7, 2026Jamie Stirling, Noura Al-Moubayed, Chris G. Willcocks +1Controllable Image Generation Compositional Image Generation
Durham University, United Kingdom

Jun 29, 2026J/K move · Enter open · S save

PhotoQuilt: Training-Free Arbitrary-Resolution Photomosaics via Bootstrapped Tiled Denoising

Koorosh Roohi, Javad Rajabi, Andrew Fleet, Babak Taati

University of Toronto · Vector Institute · KITE Research Institute+2

Image Generation Compositional Image Generation Diffusion-Based Image Generation Diffusion Models

Open paper AskPDF arXiv

Photomosaics are large images whose local regions are seen as independent tiles while their overall arrangement forms a coherent scene. Generating them at high resolution, with every tile convincing in its own right, is computationally expensive, since the canvas must hold many detailed tiles at once. We present PhotoQuilt, a training-free framework that generates photomosaics at arbitrary resolution. Diffusion models struggle to satisfy both scales at once, as direct high-resolution generation is costly and tends toward one smooth image rather than a mosaic, while patch-based tiling keeps local detail but loses global structure. PhotoQuilt resolves this with a bootstrapped tiled denoising procedure. We first produce a global composition at low resolution to fix the layout, then upscale it in latent space and re-inject noise to restore generative capacity. Denoising proceeds within fixed tiles, so each forms its own image while the shared global structure holds them in one layout. Because tile generation is handled separately, PhotoQuilt scales to large canvases without quadratic attention cost. Experiments show that PhotoQuilt outperforms current baselines on both global structure and local realism.

Abstract

Diffusion-Based Image Generation

Explore similar work

PhotoQuilt: Training-Free Arbitrary-Resolution Photomosaics via Bootstrapped Tiled Denoising

Efficient and Training-Free Single-Image Diffusion Models

Controllable Image Generation with Composed Parallel Token Prediction