DRG-Font: Dynamic Reference-Guided Few-shot Font Generation via Contrastive Style-Content Disentanglement

Rejoy Chakraborty; Prasun Roy; Saumik Bhattacharya; Umapada Pal

cs.CVApr 15, 2026

DRG-Font: Dynamic Reference-Guided Few-shot Font Generation via Contrastive Style-Content Disentanglement

Authors: Rejoy Chakraborty, Prasun Roy, Saumik Bhattacharya, Umapada Pal

Organizations: Indian Statistical Institute Kolkata · Indian Institute of Technology Kharagpur

Abstract

Few-shot Font Generation aims to generate stylistically consistent glyphs from a few reference glyphs. However, capturing complex font styles from a few exemplars remains challenging, and the existing methods often struggle to retain discernible local characteristics in generated samples. This paper introduces DRG-Font, a contrastive font generation strategy that learns complex glyph attributes by decomposing style and content embedding spaces. For optimal style supervision, the proposed architecture incorporates a Reference Selection (RS) Module to dynamically select the best style reference from an available pool of candidates. The network learns to decompose glyph attributes into style and shape priors through a Multi-scale Style Head Block (MSHB) and a Multi-scale Content Head Block (MCHB). For style adaptation, a Multi-Fusion Upsampling Block (MFUB) produces the target glyph by combining the reference style prior and target content prior. The proposed method demonstrates significant improvements over state-of-the-art approaches across multiple visual and analytical benchmarks.

Figures & tables

Figure 2: An overview of the proposed DRG-Font. The initial reference selection is performed by the RS Module, which finds the optimal style reference $x_{s}$ from an available set of candidates, such that $x_{s}$ is structurally closest to the given content reference $x_{c}$ . The Generator network independently encodes $x_{s}$ and $x_{c}$ by using $E_{sc}$ , producing the style and content embeddings from both references. The target glyph $\hat{y}$ is generated by $D_{sc}$ through a cross-fusion of the embeddings. The discriminator network enforces adversarial guidance by auxiliary style and content classification. The Stable Diffusion Encoder (SDv2 Enc) [ 30 ] is used to ensure high-quality latent reconstruction.

Figure 3: Architecture of the proposed Style-Content Encoder

E_{sc}

. The input image

x_{\alpha}

passes through four consecutive Down2x Blocks. The resulting feature maps then pass through the Multiscale Style Head Block (MSHB) and the Multiscale Content Head Block (MCHB) in parallel. Both MSHB and MCHB encode three latent embeddings using three different heads, which are finally concatenated to produce the style embedding

e^{style}_{\alpha}

(from MSHB) and the content embedding

e^{content}_{\alpha}

(from MCHB).

Figure 4: Architecture of the proposed Style-Content Decoder

D_{sc}

. The network uses a Multi-Fusion Upsampling Block (MFUB) to generate the target image

\hat{y}

by combining the multiscale style embeddings

e^{style}_{s}=\langle e^{style}_{s,1},e^{style}_{s,2},e^{style}_{s,3}\rangle

from MSHB and content embedding

e^{content}_{c}

from MCHB.

Figure 5: Qualitative comparison results on Unseen English and Seen English fonts. Boxes marked in red highlight issues like failure of style adaptation, structural deformation, and missing strokes. Boxes marked in blue highlight issues like minor appearance of artifacts and minor structural inconsistencies.

Figure 6: Qualitative comparison results on Unseen Chinese and Seen Chinese fonts. Boxes marked in red highlight issues like failure of style adaptation, structural deformation, and missing strokes. Boxes marked in blue highlight issues like minor appearance of artifacts and minor structural inconsistencies.

Method	L1 $\downarrow$	RMSE $\downarrow$	SSIM $\uparrow$	LPIPS $\downarrow$	L1 $\downarrow$	RMSE $\downarrow$	SSIM $\uparrow$	LPIPS $\downarrow$	User Study $\uparrow$
	Unseen English				Seen English
FANNET [ 31 ]	0.077	0.239	0.731	0.185	0.078	0.238	0.779	0.116	10.789
MA-Font [ 28 ]	0.089	0.272	0.694	0.143	0.087	0.269	0.712	0.138	6.710
PatchFont [ 22 ]	0.098	0.266	0.675	0.143	0.100	0.281	0.680	0.143	5.657
FASTER [ 8 ]	0.075	0.246	0.723	0.139	0.077	0.239	0.731	0.138	7.894
DA-Font [ 4 ]	0.074	0.243	0.713	0.111	0.069	0.223	0.775	0.084	15.526

Table 1: Quantitative comparison of the proposed method with the existing state-of-the-art techniques on Unseen English and Seen English fonts. The best scores are shown in bold and the second best scores are underlined .

Method	L1 $\downarrow$	RMSE $\downarrow$	SSIM $\uparrow$	LPIPS $\downarrow$	L1 $\downarrow$	RMSE $\downarrow$	SSIM $\uparrow$	LPIPS $\downarrow$	User Study $\uparrow$
	Unseen Chinese				Seen Chinese
FANNET [ 31 ]	0.161	0.347	0.449	0.289	0.147	0.334	0.502	0.291	2.105
MA-Font [ 28 ]	0.176	0.374	0.430	0.150	0.172	0.372	0.456	0.160	9.605
PatchFont [ 22 ]	0.251	0.442	0.288	0.224	0.225	0.420	0.353	0.204	3.158
FASTER [ 8 ]	0.197	0.339	0.458	0.160	0.093	0.297	0.484	0.166	8.421
DA-Font [ 4 ]	0.166	0.357	0.469	0.143	0.125	0.303	0.603	0.107	21.053

Table 2: Quantitative comparison of the proposed method with the existing state-of-the-art techniques on Unseen Chinese and Seen Chinese fonts. The best scores are shown in bold and the second best scores are underlined .

Figure 7: Qualitative comparison of the generation quality with and without (in both train and test phases) incorporating the RS Module into the proposed pipeline.

RS Module		Metrics
Training	Testing	L1 $\downarrow$	RMSE $\downarrow$	SSIM $\uparrow$	LPIPS $\downarrow$
✗	✗	0.098	0.287	0.676	0.167
✗	✓	0.085	0.266	0.699	0.141
✓	✗	0.081	0.256	0.713	0.129
✓	✓	0.072	0.237	0.739	0.108

Table 3: Quantitative analysis of the impact of the RS Module on the generation quality.

Loss Function		Metrics
$\mathcal{L}_{\text{cls}}$	$\mathcal{L}_{\text{latent}}$	L1 $\downarrow$	RMSE $\downarrow$	SSIM $\uparrow$	LPIPS $\downarrow$
✗	✗	0.074	0.243	0.730	0.113
✗	✓	0.073	0.240	0.736	0.108
✓	✗	0.075	0.244	0.729	0.114
✓	✓	0.072	0.237	0.739	0.108

Table 4: Quantitative analysis of the effectiveness of

\mathcal{L}_{\text{cls}}

and

\mathcal{L}_{\text{latent}}

on the generation quality.

Head Dimension	L1 $\downarrow$	RMSE $\downarrow$	SSIM $\uparrow$	LPIPS $\downarrow$
128	0.074	0.241	0.734	0.108
256	0.072	0.237	0.739	0.108
512	0.074	0.241	0.734	0.111

Table 5: Quantitative analysis of the effect of embedding dimension of each head (

\in\mathbb{R}^{\frac{d}{3}}

) on the generation quality.

Explore similar work

SmartFont: Dynamic Condition Allocation for Few-Shot Font Generation
Jun 11, 2026Zian Yang, Zixin WangFew-Shot Learning Diffusion-Based Image Generation
Fudan University
Active Reference Acquisition in Few-Shot Font Generation
Jun 15, 2026Shinnosuke MatsuoActive Learning Few-Shot Learning
NTT, Inc., Japan · Kyushu University
Structure-Level Disentangled Diffusion for Few-Shot Chinese Font Generation
Feb 21, 2026Jie Li, Suorong Yang, Jian Zhao +1Few-Shot Learning Conditional Diffusion Models
State Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China · Department of Computer Science and Technology, Nanjing University, China +1

Jun 11, 2026J/K move · Enter open · S save

SmartFont: Dynamic Condition Allocation for Few-Shot Font Generation

Zian Yang, Zixin Wang

Fudan University

Few-Shot Learning Diffusion-Based Image Generation Conditional Diffusion Models

Open paper AskPDF arXiv

Few-shot font generation simultaneously requires global structural completeness and fine-grained local style fidelity. Existing methods usually either rely on global content-style modeling, which is robust but imperfectly disentangled, or emphasize component/local modeling, which captures fine details but relies heavily on local priors and reference coverage. We argue that the key challenge is not merely to learn purer conditions, but to organize complementary yet biased global and local conditions through multi-level allocation during generation. To this end, we propose SmartFont, a diffusion-based few-shot font generation framework that combines global content-style generation with weakly supervised local corrective experts. The local branch performs semantic-spatial allocation by learning expert-wise local concepts and semantically meaningful spatial maps under weak component supervision, enabling fine-grained correction without requiring explicit component-conditioned inference. On top of this, a denoising-state condition allocation module adaptively weights global content, global style, and local corrective feature across timesteps and injection blocks. Extensive experiments show that SmartFont achieves better global-local balance, improves glyph quality and local detail fidelity.

Abstract

Explore similar work

SmartFont: Dynamic Condition Allocation for Few-Shot Font Generation

Active Reference Acquisition in Few-Shot Font Generation

Structure-Level Disentangled Diffusion for Few-Shot Chinese Font Generation