Kingfisher: Fast Personalized Visual Autoregressive Generation via Scale-Aware Prompt-Conditioned Scale Decomposition

Dinh-Khoi Vo; Akihiro Sugimoto; Minh-Triet Tran; Trung-Nghia Le

cs.CVOct 16, 2026

Kingfisher: Fast Personalized Visual Autoregressive Generation via Scale-Aware Prompt-Conditioned Scale Decomposition

Authors: Dinh-Khoi Vo, Akihiro Sugimoto, Minh-Triet Tran, Trung-Nghia Le

Organizations: University of Science, Ho Chi Minh City, Vietnam · Vietnam National University, Ho Chi Minh City, Vietnam · National Institute of Informatics, Tokyo, Japan

Abstract

Personalized image generation has been extensively studied in diffusion models, often relying on subject-specific optimization of model weights, textual embeddings, LoRA modules, or adapters. In this work, we explore a different direction for fast personalization in visual autoregressive models. Our key observation is that a short prefix of early autoregressive scale codes is sufficient to strongly determine both object structure and global appearance, while later scales primarily refine the established generation trajectory. Based on this insight, we introduce Kingfisher, a model-tuning-free framework that decomposes early residual scale codes into content- and style-related components through prompt-conditioned scale-code subspaces. Given two real images, Kingfisher first decomposes their early scale codes, then recomposes the content components of one image with the style components of the other into a compact semantic prefix. The frozen generator subsequently predicts the remaining scales through bitwise self-correction for coherent refinement. Experiments on CSD-100 show that Kingfisher achieves strong content preservation and the highest style and text alignment among the evaluated methods, while reducing pair-specific preparation time by approximately 54x compared with diffusion-based methods and 7.5x compared with VAR-based alternatives. These results demonstrate that effective personalization can be achieved by directly manipulating the native scale-space representation of a frozen visual autoregressive generator.

Figures & tables

Figure 1: Kingfisher combines the content of one real image with the appearance of another by decomposing and recomposing early autoregressive scale codes. The remaining scales are predicted by the frozen VAR generator via bit-wise self-correction, enabling fast, coherent personalization without model tuning, LoRA, adapters, or learned subject embeddings.

Figure 2: Scale-wise analysis of visual autoregressive generation. We analyze residual scales from two complementary views: prefix reconstruction and prompt routing. Prefix reconstruction shows that a short early ground-truth prefix is nearly sufficient to recover input semantics, while prompt routing shows that early scales dominate object identity, shape, color, and global appearance. Together, these results indicate that early consecutive scales form compact yet entangled semantic anchors, whereas later scales mainly refine details.

Figure 3: Overview of the Kingfisher pipeline. Given a content image $A$ and a style image $B$ , both are encoded into multi-scale residual codes using the frozen tokenizer. Kingfisher constructs a neutral-style affine space from neutral prompts and an object-content subspace from object-specific prompts. The selected early residual codes are decomposed into content, style, and residual factors through prompt-space constraints and recomposed into a compact semantic prefix. The frozen Infinity generator then predicts the remaining scales through bitwise self-correction, producing an output that preserves the object structure of $A$ while adopting the visual appearance of $B$ .

	Content Align.		Style Align.		Text Align.
Method	CSD-C $\uparrow$	CLIP-I $\uparrow$	CSD-S $\uparrow$	DINO $\uparrow$	CLIP-T $\uparrow$
DreamBooth-LoRA ruiz2023dreambooth	0.5519	0.7362	0.4207	0.3813	0.2492
B-LoRA frenkel2024implicit	0.4739	0.7182	0.4990	0.4220	0.2683
UnZipLoRA liu2025unziplora	0.5782	0.7322	0.3673	0.3560	0.2472
Inspiration Tree vinker2023concept	0.5413	0.7591	0.2897	0.2601	0.2565
CSD-VAR nguyen2025csd	0.4731	0.7184	0.2812	0.2503	0.2648

Table 1: Quantitative comparison on the cross-image CSD-100 benchmark. Best results are shown in bold and second-best results are underlined .

Figure 4: Qualitative comparison on the cross-image CSD-100 benchmark nguyen2025csd . Given a content reference $I^{A}$ and a style reference $I^{B}$ , each method aims to preserve the content of $I^{A}$ while transferring the appearance of $I^{B}$ . Kingfisher achieves a favorable balance between content preservation and style transfer across diverse content–style combinations.

Method	Prep./pair (s) $\downarrow$	Infer./image (s) $\downarrow$	Votes $\uparrow$	Pref. $\uparrow$
	Efficiency		Human Evaluation
DreamBooth-LoRA ruiz2023dreambooth	$1542.21\pm 38.47$	$11.98\pm 0.91$	54	9.0%
B-LoRA frenkel2024implicit	$1309.00\pm 27.63$	$12.88\pm 1.02$	168	28.0%
UnZipLoRA liu2025unziplora	$3463.70\pm 40.28$	$11.83\pm 0.95$	45	7.5%
Inspiration Tree vinker2023concept	$7465.36\pm 216.84$	$6.56\pm 0.54$	39	6.5%
CSD-VAR nguyen2025csd	$\underline{171.65\pm 11.82}$	$\mathbf{1.39\pm 0.12}$	40	6.7%

Table 2: Efficiency and human evaluation. Efficiency is measured on a single A100 GPU. Preparation denotes the pair-specific optimization time for one content–style pair, while inference denotes the time required to generate one image after preparation. Human preference is computed from 600 forced-choice selections.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Experiment	Range	CLIP-I $\uparrow$	DINO-I $\uparrow$	LPIPS $\downarrow$	CLIP-T $\uparrow$	DINO-P1 $\uparrow$
Prefix recon.	$R_{1}$	0.42	0.36	0.58	–	–
Prefix recon.	$R_{1:2}$	0.61	0.54	0.41	–	–
Prefix recon.	$R_{1:3}$	0.78	0.72	0.26	–	–
Prefix recon.	$R_{1:4}$	0.88	0.84	0.16	–	–
Prefix recon.	$R_{1:7}$	0.93	0.89	0.11	–	–
Prompt routing	$R_{10:13}$	–	–	–	0.28	0.74

Appendix

Table 3: Quantitative scale-wise analysis. Results are averaged over 100 real images and 200 prompt-routing pairs. For prefix reconstruction, CLIP-I and DINO-I measure similarity to the input image, while LPIPS measures perceptual reconstruction distance. For prompt routing, CLIP-T measures alignment with Prompt 2 , while DINO-P1 measures similarity to the all-Prompt 1 reference.

Type	Prompt template
Neutral	“a plain realistic photograph with natural colors and ordinary daylight”
	“a standard documentary photograph with neutral balanced lighting”
	“a clean product photograph with a simple background and natural texture”
	“a realistic snapshot with balanced colors and soft natural illumination”
	“a straightforward studio photograph with ordinary materials and neutral tones”
Object	“a clearly recognizable <object> ”

Appendix

Table 4: Neutral and object-conditioned prompt templates used to construct the prompt-conditioned scale subspaces.

Prefix configuration	CSD-C $\uparrow$	CLIP-I $\uparrow$	CSD-S $\uparrow$	DINO $\uparrow$	CLIP-T $\uparrow$
$[A_{1},A_{2},A_{3},A_{4}]$	0.7774	0.9370	0.2922	0.2720	0.3017
$[R_{1}^{\ast},A_{2},A_{3},R_{4}^{\ast}]$	0.6281	0.8392	0.4342	0.3751	0.3119
$\mathbf{[R_{1}^{\ast},R_{2}^{\ast},A_{3},R_{4}^{\ast}]}$	0.5504	0.7922	0.5210	0.4367	0.3127
$[R_{1}^{\ast},R_{2}^{\ast},R_{3}^{\ast},R_{4}^{\ast}]$	0.4979	0.7416	0.6156	0.5416	0.2959

Appendix

Table 5: Ablation of early-scale prefix composition. Best results are shown in bold and second-best results are underlined.

$\beta$	CSD-C $\uparrow$	CLIP-I $\uparrow$	CSD-S $\uparrow$	DINO $\uparrow$	CLIP-T $\uparrow$
0.00	0.5436	0.7784	0.5353	0.4597	0.3068
0.25	0.5471	0.7866	0.5333	0.4521	0.3104
0.50	0.5504	0.7922	0.5210	0.4367	0.3127
0.75	0.5551	0.7968	0.5161	0.4239	0.3130
1.00	0.5613	0.8027	0.5063	0.4096	0.3144

Appendix

Table 6: Sensitivity to the structural correction coefficient

\beta

. Best results are shown in bold and second-best results are underlined.

Configuration	CSD-C $\uparrow$	CLIP-I $\uparrow$	CSD-S $\uparrow$	DINO $\uparrow$	CLIP-T $\uparrow$
w/o $\mathcal{L}_{\mathrm{sty}}$	0.5478	0.7895	0.4726	0.3914	0.3048
w/o $\mathcal{L}_{\mathrm{align}}$	0.5127	0.7568	0.5073	0.4218	0.3035
w/o $\mathcal{L}_{\mathrm{remove}}$	0.5019	0.7446	0.4981	0.4127	0.3009
w/o $\mathcal{L}_{\mathrm{rec}}$	0.4742	0.7165	0.4386	0.3597	0.2863
w/o $\mathcal{L}_{\mathrm{orth}}$	0.5264	0.7698	0.4935	0.4062	0.3074
Full objective	0.5504	0.7922	0.5210	0.4367	0.3127

Appendix

Table 7: Ablation of the decomposition objective. Best results are shown in bold and second-best results are underlined.

Figure 5: Additional qualitative ablation of prefix composition. From left to right: content reference $I^{A}$ , style reference $I^{B}$ , $[R_{1}^{\ast},A_{2},A_{3},R_{4}^{\ast}]$ with $\beta=0.5$ , $[R_{1}^{\ast},R_{2}^{\ast},A_{3},R_{4}^{\ast}]$ with $\beta=0.5$ , $[R_{1}^{\ast},R_{2}^{\ast},R_{3}^{\ast},R_{4}^{\ast}]$ with $\beta=0.5$ , $[R_{1}^{\ast},A_{2},A_{3},R_{4}^{\ast}]$ with $\beta=1.0$ , $[R_{1}^{\ast},R_{2}^{\ast},A_{3},R_{4}^{\ast}]$ with $\beta=1.0$ , and $[R_{1}^{\ast},R_{2}^{\ast},R_{3}^{\ast},R_{4}^{\ast}]$ with $\beta=1.0$ .

Figure 6: Visualization of style removal. Each pair shows an input image and the corresponding style-removed result $R-S$ . The operation suppresses appearance-specific information while retaining the underlying object content and structure.

Figure 7: Additional qualitative comparison with prior methods. Given a content reference $I^{A}$ and a style reference $I^{B}$ , we compare Kingfisher with the evaluated baselines across additional content–style combinations.

Figure 8: Qualitative results of Kingfisher on CSD-100. Each example shows the content reference $I^{A}$ , the style reference $I^{B}$ , and the corresponding Kingfisher output.

Figure 9: Challenging cases with abstract or ambiguous style references. When the style reference contains weak, highly abstract, or insufficiently distinctive visual cues, its appearance is difficult to isolate from a single image. Consequently, both existing baselines and Kingfisher may exhibit weaker or less consistent style transfer.

Image Generation

Image generation studies models that create or edit images from text, examples, layouts, and other structured controls.

See all 474 papers

Logit Refiner: Improving Visual Autoregressive Models via Intra-Scale Dependency Modeling
Sep 10, 2026Meimingwei Li, Stefan Andreas Baumann, Felix Krause +1Visual Autoregressive Models T2I Generation
Munich Center for Machine Learning (MCML)
PersonalQ: Select, Quantize, and Serve Personalized Diffusion Models for Efficient Inference
Mar 24, 2026Qirui Wang, Qi Guo, Yiding Sun +4Diffusion Model Serving T2I Diffusion Models
School of Software Engineering, Xi’an Jiaotong University, China · Nankai University, China
FlashAR: Efficient Post-Training Acceleration for Autoregressive Image Generation
May 10, 2026Junkang Zhou, Yefei He, Feng Chen +2Parallel Decoding Autoregressive Image Generation
Zhejiang University, China · University of Adelaide, Australia

Sep 10, 2026J/K move · Enter open · S save

Logit Refiner: Improving Visual Autoregressive Models via Intra-Scale Dependency Modeling

Meimingwei Li, Stefan Andreas Baumann, Felix Krause, Björn Ommer

Munich Center for Machine Learning (MCML)

Visual Autoregressive Models T2I Generation Autoregressive Image Generation

Open paper AskPDF arXiv

Visual Autoregressive Models (VAR) generate images through next-scale prediction, producing all tokens within each scale in parallel. We show that this parallel decoding constitutes a mean-field-style approximation that discards spatial dependencies among same-scale tokens, causing locally incoherent samples regardless of backbone capacity -- a limitation of the decoding rule. Addressing this limitation, we introduce the Logit Refiner, a lightweight autoregressive module that restores intra-scale dependencies by sequentially sampling tokens conditioned on frozen backbone features. Adding only ~10% parameters and less than 5% of the base model's training compute, it plugs into any pretrained VAR checkpoint without retraining. Controlled ablations isolate joint intra-scale sampling -- rather than additional capacity or training -- as the critical ingredient. Across backbones from 310M to 2B parameters on class-conditional ImageNet 256x256, the refiner consistently improves generation quality, enabling a 1.1B-parameter model to surpass one twice its size. The approach further generalizes to text-to-image generation, confirming that the mean-field bottleneck persists across VAR variants and is effectively alleviated by our method. Project page: https://compvis.github.io/logit-refiner/

Abstract

Image Generation

Explore similar work

Logit Refiner: Improving Visual Autoregressive Models via Intra-Scale Dependency Modeling

PersonalQ: Select, Quantize, and Serve Personalized Diffusion Models for Efficient Inference

FlashAR: Efficient Post-Training Acceleration for Autoregressive Image Generation