CONSTANT: Towards High-Quality One-Shot Handwriting Generation with Patch Contrastive Enhancement and Style-Aware Quantization

Anh-Duy Le; Van-Linh Pham; Thanh-Nam Vo; Xuan Toan Mai; Tuan-Anh Tran

cs.CVMar 8, 2026

CONSTANT: Towards High-Quality One-Shot Handwriting Generation with Patch Contrastive Enhancement and Style-Aware Quantization

Authors: Anh-Duy Le, Van-Linh Pham, Thanh-Nam Vo, Xuan Toan Mai, Tuan-Anh Tran

Organizations: Viettel Artificial Intelligence and Data Services Center, Vietnam · Ho Chi Minh City University of Technology, Vietnam

Abstract

One-shot styled handwriting image generation, despite achieving impressive results in recent years, remains challenging due to the difficulty in capturing the intricate and diverse characteristics of human handwriting by using solely a single reference image. Existing methods still struggle to generate visually appealing and realistic handwritten images and adapt to complex, unseen writer styles, struggling to isolate invariant style features (e.g., slant, stroke width, curvature) while ignoring irrelevant noise. To tackle this problem, we introduce Patch Contrastive Enhancement and Style-Aware Quantization via Denoising Diffusion (CONSTANT), a novel one-shot handwriting generation via diffusion model. CONSTANT leverages three key innovations: 1) a Style-Aware Quantization (SAQ) module that models style as discrete visual tokens capturing distinct concepts; 2) a contrastive objective to ensure these tokens are well-separated and meaningful in the embedding style space; 3) a latent patch-based contrastive (LLatentPCE) objective help improving quality and local structures by aligning multiscale spatial patches of generated and real features in latent space. Extensive experiments and analysis on benchmark datasets from multiple languages, including English, Chinese, and our proposed ViHTGen dataset for Vietnamese, demonstrate the superiority of adapting to new reference styles and producing highly detailed images of our method over state-of-the-art approaches. Code is available at GitHub

Figures & tables

Panel of Figure 1 — Figure 1: Our method leverages the idea of vector quantization to learn the embedding space of diverse style concepts, enabling better adaptation to new styles than other approaches.

Figure 2: Overall architecture of our method . a) The training pipeline, the model is optimized using objectives: $L_{denoising}$ , $L_{SCE}$ and $L_{LatentPCE}$ and $L_{SAQ}$ , b) Architecture of our SAQ module, with Inception-V3 backbone, codebook embedding and a Attention Pool module, c) Sampling pipeline of our method.

Figure 3: Visualization of our $L_{LatentPCE}$ objective in bidirectional format including two sub contrastive loss. First loss receives ( anchor , positive , negatives ), second loss receives ( anchor , positive , negative ).

Method	Reference	HWD $\downarrow$	FID $\downarrow$	WER $\downarrow$	$Acc_{Wid}$ $\uparrow$
HWT [ 2 ]	Few-shot	1.23	19.82	0.62	9.08
VATr [ 36 ]		1.13	16.30	0.51	49.81
DiffusionPen [ 33 ]		1.04	18.94	0.23	38.86
HiGAN+ [ 11 ]	One-shot	0.89	13.90	0.56	55.20
HiGAN [ 12 ]		1.55	27.13	0.55	29.79
One-DM [ 4 ]		1.05	15.97	0.36	4.5

Table 1: Comparison between SOTA methods on IAM test dataset. Bold values indicate best and Underline values indicate second-best.

	IV-S		OOV-S		IV-U		OOV-U
Method	HWD $\downarrow$	FID $\downarrow$	HWD $\downarrow$	FID $\downarrow$	HWD $\downarrow$	FID $\downarrow$	HWD $\downarrow$	FID $\downarrow$
HWT [ 2 ]	2.30	135.51	2.30	146.16	2.35	138.39	2.36	148.75
VATr [ 36 ]	2.50	132.87	2.51	140.56	2.58	137.34	2.60	144.02
DiffusionPen [ 33 ]	1.14	91.20	1.16	97.65	1.52	112.87	1.65	122.52
HiGAN [ 12 ]	2.29	118.70	2.31	128.49	2.36	119.56	2.37	128.60
HiGAN+ [ 11 ]	1.76	117.76	1.79	122.56	1.78	117.63	1.81	124.38

Table 2: Evaluation in terms of FID and HWD scores on four scenarios: IV-S (In-vocab, Seen style), OOV-S (Out-of-vocab, Seen style), IV-U (In-vocab, Unseen style), OOV-U (Out-of-vocab, Unseen style). Bold values indicate best and Underline values indicate second-best.

Table 6

Figure 4: Comparison between ours with other methods using random style from IAM test set and two different textual contents.

Figure 5: Visualization of handwritten text generation on IMGUR5K test set.

				Style images	FID $\downarrow$	HWD $\downarrow$
Base	$SAQ$	$L_{SCE}$	$L_{PCE}$
$\checkmark$					16.73	0.87
$\checkmark$	$\checkmark$				12.47	0.85
$\checkmark$	$\checkmark$	$\checkmark$			12.55	0.84
$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$		10.20	0.74

Table 5: Contribution of our SAQ,

L_{SCE}

, and

L_{LatentPCE}

to the baseline model in terms of FID and HWD score.

	Chinese		Vietnamese
Method	HWD $\downarrow$	FID $\downarrow$	HWD $\downarrow$	FID $\downarrow$
One-DM [ 4 ]	0.48	22.97	1.08	22.53
Our	0.37	22.74	0.83	18.81

Table 6: Quantitative comparisons with One-DM on Chinese and Vietnamese scripts in terms of FID and HWD.

Figure 6: Comparisons between CONSTANT and One-DM [ 4 ] on Chinese and Vietnamese scripts.

Figure 7: Style cross-attention map visualization between our CONSTANT and baseline model.

Figure 8: Real images versus generated images using IAM-trained model on IMGUR5K references.

Panel of Figure 9 — Figure 9: Performing domain adaptation on IMGUR5K model using IAM reference images.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: User plausibility study results

Figure 3: An overview of our ViHTGen dataset. a) The diversity of handwritten style between different writers. b) The complexity of our dataset in both background and stroke shape.

Statistic	ViHTGen (ours)	IAM
Language	Vietnamese	English
# Writers	223	500
# Word Instances	50000+	62857
# Unique words	4644	3332
Style Complexity (Slant, Ink color, Stroke width, character shape)	High	Medium
Background complexity	High	Low

Appendix

Table 1: Comparison between our ViHTGen and IAM dataset.

Panel of Figure 4 — Figure 4: Statistical analysis for the ViHTGen dataset. (a) The percentage of images with different text lengths. (b) Frequency distribution of top-10 characters in the dataset

Method	HWD $\downarrow$	FID $\downarrow$
One-DM [ 4 ]	1.22	17.74
Ours	0.73	10.22

Appendix

Table 2: Quantitative results on IIIT-English-Word test set.

Figure 5: Comparisons between our method and One-DM [ 4 ] on IIIT-English-Word.

	Chinese		Vietnamese
Method	HWD $\downarrow$	FID $\downarrow$	HWD $\downarrow$	FID $\downarrow$
DiffusionPen [ 33 ]	0.57	54.07	1.05	23.59
Our	0.37	22.74	0.83	18.81

Appendix

Table 3: Quantitative comparisons with One-DM on Chinese and Vietnamese scripts in terms of FID and HWD.

Figure 6: Qualitative results between CONSTANT and DiffusionPen [ 33 ] in Chinese script.

Data Source	Accuracy
Real data	81.76
Real data + 100K	82.13
Real data + 200K	82.96

Appendix

Table 4: Generated data helps improve OCR performance on IAM test set

	IAM		IMGUR5K		IIIT-English-Word
Codebook length	HWD $\downarrow$	FID $\downarrow$	HWD $\downarrow$	FID $\downarrow$	HWD $\downarrow$	FID $\downarrow$
512	0.86	13.80	1.02	13.37	0.75	14.02
1024	0.84	12.46	1.03	12.9	0.77	12.61
2048	0.86	14.10	0.99	11.48	0.73	10.22

Appendix

Table 5: Ablation Study About The Codebook Embedding Length in SAQ Module

Criteria	CONSTANT	One-DM
Model parameters	124 $\times 10^{6}$	185 $\times 10^{6}$
Training VRAM	6.82GB	18.66GB
Inference time 7 7 7 Sampling taken under 50 steps	1.25 s/sample	1.85 s/sample
Inference VRAM	2.1GB	2.9GB

Appendix

Table 6: Comparison training and inference cost between CONSTANT and One-DM.

Figure 7: Visualization of some failure cases.

Figure 8: Visualization of arbitrary textual content between different methods on IAM test dataset.

Figure 9: Visualization of our CONSTANT with other competitors, where RED box shows the style adaptability between different methods to the reference image.

Figure 10: Other examples on IMGUR5K dataset.

Figure 11: Other examples on ViHTGen dataset.

Image Generation

Image generation studies models that create or edit images from text, examples, layouts, and other structured controls.

See all 463 papers

Zero-Shot Paragraph-level Handwriting Imitation with Latent Diffusion Models
Sep 1, 2024Martin Mayr, Marcel Dreier, Florian Kordon +5Latent Diffusion Models Image Generation
Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany. · Planet AI GmbH, Rostock, Germany.
Diffusion-Based Ukrainian Handwritten Text Generation with Cross-Domain Style Transfer
May 26, 2026Andrii Ahitoliev, Pavlo BerezinLatent Diffusion Models Image Style Transfer
Ukrainian Catholic University, Lviv, Ukraine · National University of “Kyiv-Mohyla Academy”, Kyiv, Ukraine
InkShield: Writing Style Protection Against Unauthorized Handwriting Mimicry
Jul 29, 2026Jian Xiong, Wenbo Jiang, Zihan Wang +4Adversarial Examples
University of Electronic Science and Technology of China

Sep 1, 2024J/K move · Enter open · S save

Zero-Shot Paragraph-level Handwriting Imitation with Latent Diffusion Models

Martin Mayr, Marcel Dreier, Florian Kordon, Mathias Seuret+4

Pattern Recognition Lab, Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany. · Planet AI GmbH, Rostock, Germany.

Latent Diffusion Models Image Generation Multimodal Diffusion Models

Open paper AskPDF arXiv

The imitation of cursive handwriting is mainly limited to generating handwritten words or lines. Multiple synthetic outputs must be stitched together to create paragraphs or whole pages, whereby consistency and layout information are lost. To close this gap, we propose a method for imitating handwriting at the paragraph level that also works for unseen writing styles. Therefore, we introduce a modified latent diffusion model that enriches the encoder-decoder mechanism with specialized loss functions that explicitly preserve the style and content. We enhance the attention mechanism of the diffusion model with adaptive 2D positional encoding and the conditioning mechanism to work with two modalities simultaneously: a style image and the target text. This significantly improves the realism of the generated handwriting. We set a new benchmark in our comprehensive evaluation, achieving 61 % mAP and 56 % top-1 accuracy in style preservation, significantly outperforming the previous best method (37 % mAP, 30 % top-1). We are making our code publicly available for reproducibility, supporting research in this area and research into potential countermeasures: https://github.com/M4rt1nM4yr/paragraph_handwriting_imitation_ldm

Abstract

Image Generation

Explore similar work

Zero-Shot Paragraph-level Handwriting Imitation with Latent Diffusion Models

Diffusion-Based Ukrainian Handwritten Text Generation with Cross-Domain Style Transfer

InkShield: Writing Style Protection Against Unauthorized Handwriting Mimicry