Organizations: The Hong Kong University of Science and Technology (Guangzhou) · Faculty of Engineering and IT, University of Technology Sydney · Xiamen University · The Hong Kong University of Science and Technology
Page-level calligraphy synthesis requires balancing glyph precision with layout composition. Existing character models lack spatial context, while page-level methods often compromise brushwork detail. In this paper, we present CalliMaster, a unified framework for controllable generation and editing that resolves this conflict by decoupling spatial planning from content synthesis. Inspired by the human cognitive process of ``planning before writing'', we introduce a coarse-to-fine pipeline \textbf{(Text → Layout → Image)} to tackle the combinatorial complexity of page-scale synthesis. Operating within a single Multimodal Diffusion Transformer, a spatial planning stage first predicts character bounding boxes to establish the global spatial arrangement. This intermediate layout then serves as a geometric prompt for the content synthesis stage, where the same network utilizes flow-matching to render high-fidelity brushwork. Beyond achieving state-of-the-art generation quality, this disentanglement supports versatile downstream capabilities. By treating the layout as a modifiable constraint, CalliMaster enables controllable semantic re-planning: users can resize or reposition characters while the model automatically harmonizes the surrounding void space and brush momentum. Furthermore, we demonstrate the framework's extensibility to artifact restoration and forensic analysis, providing a comprehensive tool for digital cultural heritage.
Figures & tables
Figure 2: Overview of CalliMaster. The framework decouples calligraphy generation into two core stages: spatial planning and content writing. Inference executes these sequentially: planning the layout before writing the content. Training randomly samples between these two primary stages and two auxiliary states to jointly optimize the model. To enable this unified strategy, the core MF-DiT block employs modality-aware AdaLN driven by independent timesteps ( tc , tb , ti ), structural attention masking, and modulate embeddings.
Figure 3: Qualitative comparison of CalliMaster against state-of-the-art models. A red cross × marks incorrectly rendered or hallucinated characters, and a red circle ◯ indicates omitted characters. Yellow boxes □ highlight continuous strokes.
Figure 4: Semantic Layout Editing via Geometric Prompts. CalliMaster supports diverse interactive operations (replacement, scaling, deletion, and insertion). By manipulating bounding boxes, the model re-harmonizes continuous strokes and adjusts the spatial composition to maintain visual rhythm.
Method
CLIP-Score ↑
GPT-Score ↑
CharsAcc ↑
FontDiffuser
0.8029
2.4
75.52%
Gemini-Nano2
0.7620
2.2
69.23%
Hunyuan 3.0
0.7547
2.3
38.47%
Seedream 4.0
0.7864
2.6
62.54%
GPT-Image-1
0.7897
2.7
50.18%
UniCalli
0.9051
3.6
42.03%
Table 1: Quantitative comparison of calligraphy generation methods. CLIP/GPT scores measure stylistic similarity to the reference. CharsAcc is the F1 score from PP-OCR [ 8 ] .
Method
Style Fidelity ↑
Glyph Accuracy ↑
Naturalness ↑
Layout Similarity ↑
Overall ↑
FontDiffuser
1.720, 1.385
4.920, 0.396
2.080, 1.512
1.493, 1.287
1.840, 1.346
GPT-Image-1
3.147, 1.178
3.720, 1.215
2.587, 1.302
2.360, 1.194
2.680, 1.253
Hunyuan 3.0
2.653, 1.268
4.107, 0.984
3.187, 1.245
2.573, 1.136
3.013, 1.108
Gemini-Nano2
2.373, 1.312
3.547, 1.278
2.840, 1.356
2.187, 1.224
2.453, 1.285
UniCalli
4.187, 0.968
4.760, 0.478
4.413, 0.782
4.080, 0.914
4.467, 0.812
CalliMaster (Ours)
4.427, 0.856
4.853, 0.384
4.613, 0.724
4.520, 0.738
4.680, 0.674
Table 2: User study results (5-point Likert scale, mean ± std). Best scores in bold .
Method
GT
GT+Noise(L)
GT+Noise(M)
CalliMaster
FontDiffuser
Gemini-Nano2
GPT-Image-1
Seedream 4.0
DRS ↓
0.186
0.237
0.472
0.299
0.561
0.826
0.962
0.997
Table 3: Diffusion Reconstruction Score (DRS) across methods. “GT+Noise” denotes GT with blob artifacts at increasing intensities (L/M: 8/20 blobs).
Figure 5: Per-timestep DRS at each noise scale. Low t probes stroke-level fidelity; high t probes global layout plausibility.
Method
CLIP-Score ↑
LPIPS ↓
CalliMaster (Ours)
0.9234
0.1768
w/o Box Prediction
0.8698
0.2149
w/o Modulate Embedding
0.7400
0.3782
Table 4: Ablation Study on Model Components. Quantitative evaluation of key components in our framework.
Figure 6: Effect of Synthetic Data Proportion. We analyze the impact of the synthetic data sampling probability ( psynth ) on generation quality. The bar charts illustrate the CLIP Score (left) and LPIPS (right) across different training ratios, evaluated under both predicted and given bounding box settings.
Appendix figures & tables13 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10
Figure 8: Sample user study questionnaire for calligrapher Huang Tingjian. This image displays one specific evaluation case. It presents 6 randomly ordered outputs from the compared methods. Evaluators rate each on Style Fidelity, Glyph Accuracy, Naturalness, Layout Similarity, and Overall Quality using a 5-point Likert scale.
Figure 9: Sample user study questionnaire for calligrapher Sun Guoting. The same 10-case, 6-method, 5-criterion protocol as in Fig. 8 is applied, targeting a distinct calligraphic style.
Figure 10: Semantic editing cases. The two left images display the initially generated calligraphy and its corresponding bounding box layout. Red outlines □ and cross × marks denote user edits including box enlargement, repositioning, and deletion. Following these modifications, the system regenerates the image with adapted character structures and overall layout. The model successfully creates natural stroke connections between characters. Red circles ◯ indicate the directly edited characters. Yellow markers highlight the surrounding text layouts that dynamically adjust to the new spatial composition.
Figure 11: Style-conditioned results for the first group. Top: spatial bounding-box layouts generated for calligraphers’ styles, showing the inferred character sizes, positions, and column arrangement. Bottom: calligraphy images generated from the layout above, exhibiting the brushwork and stroke character distinctive to these calligraphers.
Figure 12: Style-conditioned results for the second group. Top: spatial layout adapted to the target styles under the exact same text condition. Note the distinct spatial rhythm and character proportions compared with group 1. Bottom: generated calligraphy reflecting these calligraphers’ individual aesthetics.
Figure 13: Style-conditioned results for the third group. Top: spatial layout for calligraphers’ styles, demonstrating yet another spatial composition strategy induced by calligraphic style conditioning. Bottom: the corresponding generated images, illustrating how CalliMaster faithfully renders style-specific stroke dynamics and page-level composition.
Figure 14: Additional generated samples (page 1 of 6). This collection shows various page-level calligraphy works generated in the style of Wang Xizhi.
Figure 15: Additional generated samples (page 2 of 6). This collection shows various page-level calligraphy works generated in the style of Sun Guoting.
Figure 16: Additional generated samples (page 3 of 6). This collection shows various page-level calligraphy works generated in the style of Qian Long.
Figure 17: Additional generated samples (page 4 of 6). This collection shows a comparison of different calligraphers and script styles for the same poem.
Figure 18: Additional generated samples (page 5 of 6). This collection shows a comparison of various calligraphers and script styles for the same poem.
Figure 19: Additional generated samples (page 6 of 6). This final collection shows more examples of different calligraphers and script styles for the same poem.