Cross-Domain, Multi-Task Data-to-Text Generation without In-Domain Training Data

Yifei Song; Kun Efimov-Zhang; Claire Gardent

cs.CLAug 24, 2026

Cross-Domain, Multi-Task Data-to-Text Generation without In-Domain Training Data

Authors: Yifei Song, Kun Efimov-Zhang, Claire Gardent

Organizations: CNRS/LORIA and Université de Lorraine

Abstract

Structured data exists in many forms (tables, knowledge graphs, charts, and time series), and converting it into text may involve different generation tasks. However, most prior work on data-to-text (D2T) generation has focused on specific tasks and datasets, relying either on task-specific training data or on the zero-shot capabilities of large language models. We study cross-domain D2T generation in a setting where neither in-domain training text nor test references are available, and where domains, generation goals, and input structures vary substantially. We compare data-driven knowledge distillation (DDKD) against zero-shot inference and fine-tuning on out-of-domain D2T data, and introduce structure-preserving augmentation via structural subsampling and perturbation. Experiments on five benchmarks show that, at constant model size (1.7B parameters), DDKD consistently outperforms both fine-tuning and zero-shot inference. Moreover, the resulting small models outperform a much larger finetuned model on two of the five domains, achieving comparable performance on the remaining three. We further construct QUINTD-5, a fivefold extension of QUINTD-1, and show that simply scaling real target-domain inputs yields only modest gains, whereas our augmentation strategy remains more effective and more cost-efficient for cross-domain distillation.

Figures & tables

Figure 1: Per-domain error rate ( lower is better ) across the five QUINTD-1 target domains, averaged over two independent LLM judges (GPT-5.1 and Gemini-2.5-Pro). Results compare three small models of identical scale: zero-shot inference, fine-tuning on WebNLG, and our DDKD-distilled model. Across both Qwen3 and Gemma3 backbones, DDKD consistently yields substantially lower error rates, indicating improved cross-domain transfer without target-domain text supervision.

Figure 2: Overview of our cross-domain Data-Driven Knowledge Distillation framework. Large teacher models are first trained on source-domain WebNLG using multiple structured input linearizations. Given limited target-domain structured inputs (QUINTD-1), the teacher generates synthetic texts for distillation. We compare two ways of improving target-domain supervision: collecting more real structured inputs as a naive baseline (QUINTD-5), and our proposed structural data augmentation , which increases diversity and yields progressively simpler training examples. A smaller student model is then distilled on the resulting data and evaluated with LLM judges and human annotation for faithfulness and content coverage, together with inter-judge agreement analysis.

Dataset	Format	Generation Goal	Structure Complexity	# Input Tokens †
Wikidata	MD	Structured entity description	#Props = 101, #Vals = 418	122 (84-190)
Ice Hockey	JSON	Event-based game summary	#Props = 20, #Vals = 975	335 (315-366)
OpenWeather	JSON	Time-series weather forecast reporting	#Props = 28, #Vals = 3,782	3,294 (3,163-3,487)
GSM Arena	JSON	Entity-centric specification description	#Props = 9, #Vals = 2,361	1,280 (886-3,164)
OWID	CSV	Chart Captioning	#Rows = 217, #Cols = 2	3,742 (178-8,076)

Table 1: QUINTD-1 Benchmark and Generation Tasks . #Props, #Vals: Number of distinct properties and Values present in each dataset. # Input Tokens: Average (Min-Max) number of input tokens from Qwen3 tokenizer.

Representation	Example
WebNLG Triples	{"subject": "Abilene_Regional_Airport", "property": "cityServed", "object": "Abilene,_Texas"}
	{"subject": "Abilene,_Texas", "property": "isPartOf", "object": "Texas"}
CSV	# category: Airport
	subject,property,object
	Abilene Regional Airport,cityServed,Abilene; Texas
	Abilene; Texas,isPartOf,Texas

Table 2: Example of deterministic WebNLG linearisation into the three target-domain input formats used in our experiments.

Domain	Base	Sub./Pert.	Mixed	Len. ( $\mu\pm\sigma$ )
Wikidata	100	32,628	65,256	$173\pm 52$
Ice Hockey	100	25,500	51,000	$421\pm 28$
OpenWeather	100	1,100	2,200	$6{,}983\pm 857$
GSM Arena	100	1,381	2,762	$1{,}601\pm 643$
OWID	100	1,000	2,000	$5{,}067\pm 3{,}355$

Table 3: Statistics of the synthetic training data generated from the QUINTD-1 development sets under different augmentation strategies. Base: the number of QUINTD-1 instances. Sub./Pert.: the number of instances after subsampling or structural perturbation. Mixed: the number of instances for the mixed strategy. Len: mean and standard deviation of input-output sequence lengths (in Qwen3 tokenizer tokens).

Zero-shot (ZS)
System	Wikidata	Ice Hockey	OpenWeather	GSM Arena	OWID	NormAvg $\downarrow$
GPT-4.1-ZS	0.12	0.37	0.76	1.05	0.53	0.13
Qwen3-1.7B-ZS	0.74	1.38	6.10	1.22	2.11	0.64
Qwen3-8B-ZS	0.27	0.59	3.37	1.17	1.20	0.30
Qwen3-32B-ZS	0.45	0.63	2.53	0.49	0.99	0.25
WebNLG-SFT

Table 4: LLM-as-a-judge evaluation (all categories): average number of errors per output on the five QUINTD-1 target domains ( lower is better ). Results are based on GPT-5.1 as the judge. For DDKD, unless otherwise specified, results use 100 real seed instances (QUINTD-1 setting); rows marked with (500) use 500 real seed instances (QUINTD-5 setting). Bold indicates the best small model (1.7B) per domain and for NormAvg; underline indicates the best overall system. NormAvg denotes the mean of per-domain min-max normalised error counts, where normalisation is performed independently for each domain across all systems.

Appendix figures & tables66 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	WebNLG-SFT	Target-Domain SFT
Train. type	LoRA	LoRA
LoRA rank	8	8
LoRA $\alpha$	32	32
Target mods	all-linear	all-linear
LR	$1\!\times\!10^{-4}$	$5\!\times\!10^{-5}$ / $1\!\times\!10^{-4}$
Train BS	8	4 / 8

Appendix

Table 5: Training hyperparameters used for source-domain (WebNLG) and target-domain fine-tuning. For target-domain SFT, values are reported as Base / Augmented when they differ. For consistency, long maximum sequence lengths are used across training stages to match downstream inference conditions.

Qwen3
System	Peak Mem.	Train Time
Zero-shot (ZS)
Qwen3-1.7B-ZS	–	–
Qwen3-8B-ZS	–	–
Qwen3-32B-ZS	–	–
WebNLG-SFT

Appendix

Table 6: Training memory and time cost on a single NVIDIA A100 (80 GB). DDKD costs are reported on OWID to reflect the maximum input-length setting. Zero-shot models incur no training cost.

Domain	In-instance	Cross-instance
Wikidata		✓
Ice Hockey		✓
OpenWeather		✓
GSM Arena		✓
OWID	✓

Appendix

Table 7: Structural perturbation strategies adopted for each target domain.

Domain	QUINTD-1			QUINTD-5
Domain	#Props / Rows	#Vals / Cols	Input Len	#Props / Rows	#Vals / Cols	Input Len
Wikidata	95	416	122 (83–196)	176	1,762	123 (82–206)
Ice Hockey	20	971	333 (314–373)	20	3,047	332 (314–382)
OpenWeather	28	4,833	6,415 (6,214–6,749)	28	7,986	6,401 (6,159–6,886)
GSM Arena	9	2,432	1,307 (811–3,164)	9	8,140	1,273 (773–3,164)
OWID	236	2	4,004 (648–8,111)	273	2	4,465 (135–8,088)

Appendix

Table 8: Comparison of QUINTD-1 and QUINTD-5 development inputs. #Props / Rows denotes the number of distinct properties (JSON/Markdown) or rows (CSV), and #Vals denotes the number of distinct values. Input Len shows average input tokens (min–max, Qwen3 tokenizer). QUINTD-5 preserves similar input length distributions while substantially increasing knowledge diversity.

Figure 3: Kernel density estimates (KDE) of prompt token length distributions for QUINTD-1 and QUINTD-5 dev sets across five domains. Density normalization enables direct comparison despite differing sample sizes (100 vs. 500).

Level	Incorrect Fact	Not Checkable	Misleading	Other	All (Total Errors)
Token	0.618	0.490	0.169	0.580	0.582
Example	0.680	0.256	0.334	0.459	0.666
System	0.941	0.831	0.781	0.949	0.955

Appendix

Table 9: Cross-domain inter-judge agreement between GPT-5.1 and Gemini-2.5-Pro, macro-averaged across all target domains. All (Total Errors) is computed by correlating summed error counts across categories.

Domain	Level	Incorrect Fact	Not Checkable	Misleading	Other	All (Total Errors)
Wikidata	Token	0.563	0.663	0.147	0.255	0.523
	Example	0.521	0.748	0.088	0.280	0.562
	System	0.591	0.930	0.716	0.395	0.932
OpenWeather	Token	0.566	0.183	0.173	0.588	0.531
	Example	0.557	0.172	0.291	0.361	0.550
	System	0.915	0.444	0.075	0.826	0.913

Appendix

Table 10: Inter-judge Pearson correlation (

r

) between GPT-5.1 and Gemini-2.5-Pro at different aggregation levels, computed separately for each domain and error category. All (Total Errors) is computed by correlating the summed error counts across all error categories. System-level agreement is consistently higher than Token- and Example-level agreement, indicating robust consistency in system ranking across judges.

Target Domain	Selected DDKD Model	Teacher
Wikidata	Qwen3-1.7B-DDKD-Mixed	Qwen3-32B-SFT
Ice Hockey	Qwen3-1.7B-DDKD-Mixed	Qwen3-32B-SFT
OpenWeather	Qwen3-1.7B-DDKD-Sub	Qwen3-32B-SFT
GSM Arena	Qwen3-1.7B-DDKD-Mixed	Qwen3-32B-ZS
OWID	Qwen3-1.7B-DDKD-Mixed	Qwen3-32B-SFT

Appendix

Table 11: Final selection of distilled models per target domain. Model selection is primarily based on the average number of errors per output under GPT-5.1, and further informed by error rate and consistency with an independent judge (Gemini-2.5-Pro). Teacher models are selected on a per-domain basis according to their domain-level performance.

Metric: Nominal $\alpha$ (Span-level detection)
Granularity	Overall	Breakdown by Error Type ( $\alpha$ )
Granularity	Agreement	Incorrect Fact	Not Checkable	Misleading	Other
Token-level	0.369	0.370	0.399	0.020	0.342
Metric: Interval $\alpha$ (Error counts)
Instance-level	0.578	0.549	0.530	0.033	0.710
System-level	0.902	0.810	0.498	-0.072	0.871

Appendix

Table 12: Inter-Annotator Agreement (Krippendorff’s

\alpha

) Analysis. We report the agreement scores for the overall dataset and a breakdown by specific error types. The Overall column represents the aggregate agreement calculated over all error types. The breakdown columns show the agreement when isolating each error type (binary distinction for token-level, counts for instance/system-level). Note that Misleading errors show near-zero or negative agreement, indicating high subjectivity, whereas Incorrect Fact and Other show high reliability at the system level.

Domain	Error Type	GPT-5.1		Gemini-2.5-Pro
		Span #	Example #	Span #	Example #
OWID	Incorrect Fact	62	23	69	28
	Not Checkable	13	8	2	2
	Misleading	10	7	9	9
	Other	12	11	28	25
GSM Arena	Incorrect Fact	23	16	31	20

Appendix

Table 13: Distribution of error annotations in the human evaluation set, aggregated over four systems. For each target domain and error category, we report the total number of error spans ( Span # ) and the number of examples containing at least one such error ( Example # ), as identified by GPT-5.1 and Gemini-2.5-Pro. All domains contain 48 annotated outputs (12 inputs

\times

4 systems).

Figure 4: Human Annotation Toolkit Interface - Screenshot 1

Figure 5: Human Annotation Toolkit Interface - Screenshot 2

Figure 6: Human Annotation Toolkit Interface - Screenshot 3

System	GSM-Arena	Ice Hockey	OWID	Weather	Wikidata	NormAvg $\downarrow$
Large Model (32B)
Teacher Model	1.06	0.42	1.25	0.78	0.44	0.15
Small Models (1.7B)
Zero-Shot	1.47	1.50	1.92	3.89	1.11	0.77
WebNLG-SFT	2.03	4.47	1.94	3.42	0.92	0.91
DDKD (Ours)	0.69	0.22	0.72	1.67	0.56	0.09

Appendix

Table 14: Human evaluation results (average error count per output; lower is better). For each domain, we report the mean number of annotated errors per output, averaged over the three human annotators on the 12-example diagnostic set. NormAvg is the mean of per-domain min–max normalized scores computed across the four evaluated systems (lower is better), to reduce scale differences between domains.

Domain	Overall	Breakdown by Error Type ( $r$ )
Domain	All Errors	Fact	Unchk	Mis.	Oth.
GSM-Arena	0.941	0.997	0.502	-0.316	0.998
OWID	0.904	0.950	0.668	0.823	0.952
Weather	0.941	0.730	0.960	-0.422	0.884
Wikidata	0.829	-0.180	0.400	0.667	0.781
Ice Hockey	0.732	0.829	0.990	0.943	0.947

Appendix

Table 15: System-Level Human–LLM Agreement (Pearson

r

). Correlation between human and GPT-5.1 evaluations across 4 systems. All Errors is computed on the total error count. Breakdown columns correspond to: Fact (Incorrect Fact), Unchk (Not Checkable), Mis. (Misleading), and Oth. (Other). High correlations (

>0.9

) in All Errors indicate that GPT-5.1 largely reproduces the human system ranking.

Figure 9: System-level content coverage sanity check summarized by NormAvg (domain-wise min–max normalized coverage scores averaged across domains). Each row shows the mean NormAvg across two LLM judges (dot) and the judge-specific values (markers). DDKD-Best consistently outperforms same-size baselines and remains close to the selected best teacher, indicating gains are not achieved via content omission.

Domain	$\alpha_{\text{ord}}$	$\alpha_{\text{int}}$	#Pairs	#Skip
OWID	0.525	0.504	396	4
GSM Arena	0.617	0.670	396	4
Weather	0.373	0.472	391	9
Ice Hockey	0.283	0.524	395	5
Wikidata	0.744	0.835	398	2
ALL	0.732	0.794	1,976	20

Appendix

Table 16: Instance-level inter-judge agreement between GPT-5.1 and Gemini-2.5-Pro .

\alpha_{\text{ord}}

is computed on Coverage Score (1–5, ordinal) and

\alpha_{\text{int}}

on Estimated Coverage Ratio (0–1, interval). #Skip counts instances excluded due to parsing errors or missing values.

Domain	$r_{s}$	$\rho_{s}$	$r_{r}$	$\rho_{r}$	#Sys
OWID	0.924	1.000	0.916	1.000	4
GSM Arena	0.974	0.800	0.987	1.000	4
Weather	0.963	0.800	0.975	0.800	4
Ice Hockey	1.000	1.000	1.000	1.000	4
Wikidata	0.000	0.000	0.789	0.800	4
ALL	0.959	0.951	0.961	0.959	20

Appendix

Table 17: System-level inter-judge agreement between GPT-5.1 and Gemini-2.5-Pro .

r_{s}/\rho_{s}

are correlations on per-system mean scores and

r_{r}/\rho_{r}

on per-system mean ratios .

Dataset	#Inst.	Domain Scope	Reference
WebNLG	39,890	16 DBpedia categories	Human
E2E	46,733	Restaurant only	Human
KELM-Q1	18,723	Wikidata/Wikipedia	Automatic

Appendix

Table 18: Comparison of source-domain initialization datasets used in our ablation study. Statistics correspond to the actual training set used in our experiments.

Source Init.	Wikidata		IceHockey		OpenWeather		GSMArena		OWID		NE $\uparrow$	NC $\uparrow$
Source Init.	Err $\downarrow$	Cov $\uparrow$	Err $\downarrow$	Cov $\uparrow$	Err $\downarrow$	Cov $\uparrow$	Err $\downarrow$	Cov $\uparrow$	Err $\downarrow$	Cov $\uparrow$	NE $\uparrow$	NC $\uparrow$
WebNLG	0.09	4.61	0.02	3.99	1.86	2.76	0.65	3.23	0.43	3.41	0.954	1.000
E2E	0.09	4.38	0.00	2.56	3.84	2.07	0.39	2.43	0.90	2.18	0.828	0.066
KELM	0.31	4.40	0.39	2.93	5.06	2.08	1.86	2.21	2.39	2.02	0.000	0.072

Appendix

Table 19: Sensitivity to the source-domain initialization dataset. We fine-tune the same teacher model ( Qwen3-32B ) on three source datasets and directly evaluate transfer to the five QUINTD-1 target domains. Err denotes the average error number from our faithfulness evaluation (lower is better), and Cov denotes the content coverage score assigned by GPT-5.1 (higher is better). NE and NC denote domain-wise min–max normalized averages for Err and Cov, respectively; for Err, reversed min–max normalization is used so that higher is better. WebNLG provides the strongest overall initialization, achieving the most favorable trade-off between factual correctness and coverage across heterogeneous target domains.

Zero-shot (ZS)
System	Incorrect Fact	Not Checkable	Misleading	Other	All Categories	NormAll $\downarrow$
GPT-4.1-ZS	0.14	0.47	0.07	0.00	0.68	0.16
Qwen3-1.7B-ZS	1.54	0.53	0.21	0.04	2.31	0.64
Qwen3-8B-ZS	0.75	0.38	0.16	0.03	1.32	0.30
Qwen3-32B-ZS	0.50	0.35	0.13	0.04	1.02	0.25
WebNLG-SFT

Appendix

Table 20: Qwen3 Error-Type Result Summary by GPT-5.1 . LLM-as-a-judge evaluation (error-type breakdown): macro-averaged number of errors per output across all target domains ( lower is better ). Results are based on GPT-5.1 as the judge. Bold marks the best 1.7B system per column (ties allowed); underline marks the best overall system per column (ties allowed). NormAll is the mean of per-domain min–max normalized All Categories error counts, where normalization is performed independently within each domain across all systems to ensure equal domain contribution ( lower is better ).

Zero-shot (ZS)
System	Wikidata	Ice Hockey	OpenWeather	GSM Arena	OWID
Qwen3-1.7B-ZS	0.98	1.71	7.19	1.34	2.57
Qwen3-8B-ZS	0.59	0.62	4.22	0.69	1.67
Qwen3-32B-ZS	0.53	0.49	3.69	0.10	1.60
WebNLG-SFT
Qwen3-1.7B-SFT	0.70	2.42	10.83	2.67	2.52

Appendix

Table 21: Qwen3 Result Summary by Gemini-2.5-Pro . LLM-as-a-judge evaluation ( all categories ): average number of errors per output on the five QUINTD-1 target domains ( lower is better ). Bold indicates the best small model (1.7B) per domain, while underlined values indicate the best overall system. All error categories (Incorrect Fact, Not Checkable, Misleading, Other) are counted and summed. Results are based on Gemini-2.5-Pro as the judge.

Zero-shot (ZS)
System	Wikidata	Ice Hockey	OpenWeather	GSM Arena	OWID
Gemma3-1B-IT-ZS	1.71	2.80	4.66	4.03	3.14
Gemma3-27B-IT-ZS	0.56	0.28	4.98	0.44	0.90
WebNLG-SFT
Gemma3-1B-IT-SFT	5.49	5.34	5.48	3.60	3.86
Gemma3-27B-IT-SFT	0.12	0.20	2.48	0.37	0.77

Appendix

Table 22: Gemma3 Result Summary by GPT-5.1 . LLM-as-a-judge evaluation ( all categories ): average number of errors per output on the five QUINTD-1 target domains ( lower is better ). Bold indicates the best small model per domain, while underlined values indicate the best overall system. All error categories (Incorrect Fact, Not Checkable, Misleading, Other) are counted and summed. Results are based on GPT-5.1 as the judge.

Zero-shot (ZS)
System	Wikidata	Ice Hockey	OpenWeather	GSM Arena	OWID
Qwen3-1.7B-ZS	33.17 $\pm$ 18.76	63.40 $\pm$ 12.40	125.86 $\pm$ 21.35	121.59 $\pm$ 19.23	52.61 $\pm$ 12.91
Qwen3-8B-ZS	35.57 $\pm$ 18.33	72.35 $\pm$ 8.22	120.54 $\pm$ 13.61	140.67 $\pm$ 15.46	52.32 $\pm$ 13.51
Qwen3-32B-ZS	34.79 $\pm$ 18.72	75.44 $\pm$ 11.90	127.22 $\pm$ 22.52	183.45 $\pm$ 34.34	71.73 $\pm$ 24.26
WebNLG-SFT
Qwen3-1.7B-SFT	29.64 $\pm$ 17.50	116.73 $\pm$ 304.27	767.44 $\pm$ 749.55	776.32 $\pm$ 916.36	189.57 $\pm$ 370.39

Appendix

Table 23: Qwen3 Outputs Length Analysis . Output length statistics ( tokens ): mean

\pm

standard deviation over 100 outputs per system on each QUINTD-1 target domain. Tokenization uses the same tokenizer as the corresponding backbone (Qwen3) in our evaluation pipeline.

Zero-Shot (ZS)
Model	# Real	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
GPT-4.1-ZS	–	0.00	0.12	0.00	0.00	0.12
Qwen3-1.7B-ZS	–	0.18	0.50	0.02	0.04	0.74
Qwen3-8B-ZS	–	0.01	0.23	0.00	0.03	0.27
Qwen3-32B-ZS	–	0.04	0.39	0.01	0.01	0.45
WebNLG-SFT

Appendix

Table 24: Error counts on Wikidata evaluated by GPT-5.1 (Qwen3). We report the average number of errors per output across different error categories (lower is better). # Real denotes the number of real seed instances used for distillation. "All Categories" represents the total error count. Bold indicates the best performing model (lowest total errors).

Model	# Real	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Zero-Shot (ZS)
GPT-4.1-ZS	–	0%	11%	0%	0%	11%
Qwen3-1.7B-ZS	–	16%	31%	2%	4%	46%
Qwen3-8B-ZS	–	1%	19%	0%	3%	22%
Qwen3-32B-ZS	–	4%	29%	1%	1%	33%
WebNLG-SFT

Appendix

Table 25: Error rates on Wikidata evaluated by GPT-5.1 (Qwen3). Values represent the percentage of generated outputs containing at least one error of the specific type. # Real denotes the number of real seed instances used for distillation. The "All Categories" column reports the percentage of outputs containing any type of error. Lower percentages indicate better performance.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Qwen3-1.7B-ZS	0.30	0.28	0.06	0.34	0.98
Qwen3-8B-ZS	0.05	0.20	0.08	0.26	0.59
Qwen3-32B-ZS	0.11	0.28	0.04	0.10	0.53
WebNLG-SFT
Qwen3-1.7B-SFT	0.19	0.08	0.20	0.23	0.70

Appendix

Table 26: Error counts on Wikidata evaluated by Gemini-2.5-Pro (Qwen3). We report the average number of errors per output across different error categories (lower is better). "All Categories" represents the total error count. Bold indicates the best performing model (lowest total errors).

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Qwen-1.7B-ZS	26%	20%	4%	29%	62%
Qwen3-8B-ZS	5%	18%	8%	20%	44%
Qwen3-32B-ZS	11%	24%	4%	9%	40%
WebNLG-SFT
Qwen3-1.7B-SFT	18%	8%	14%	20%	44%

Appendix

Table 27: Error rates on Wikidata evaluated by Gemini-2.5-Pro (Qwen3). Values represent the percentage of generated outputs containing at least one error of the specific type. The "All Categories" column reports the percentage of outputs containing any type of error. Lower percentages indicate better performance.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	0.17	1.28	0.06	0.20	1.71
Gemma3-27B-IT-ZS	0.04	0.48	0.00	0.04	0.56
WebNLG-SFT
Gemma3-1B-IT-SFT	2.65	1.69	0.16	0.98	5.49
Gemma3-27B-IT-SFT	0.04	0.06	0.02	0.00	0.12

Appendix

Table 28: Error counts on Wikidata evaluated by GPT-5.1 (Gemma3). We report the average number of errors per output across different error categories (lower is better). "All Categories" represents the total error count.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	15%	67%	6%	18%	75%
Gemma3-27B-IT-ZS	4%	37%	0%	4%	39%
WebNLG-SFT
Gemma3-1B-IT-SFT	80%	76%	14%	66%	99%
Gemma3-27B-IT-SFT	4%	6%	2%	0%	11%

Appendix

Table 29: Error rates on Wikidata evaluated by GPT-5.1 (Gemma3). Values represent the percentage of generated outputs containing at least one error of the specific type. The "All Categories" column reports the percentage of outputs containing any type of error. Lower percentages indicate better performance.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	0.38	0.96	0.16	0.53	2.03
Gemma3-27B-IT-ZS	0.05	0.38	0.04	0.70	1.17
WebNLG-SFT
Gemma3-1B-IT-SFT	5.64	0.35	0.37	1.08	7.44
Gemma3-27B-IT-SFT	0.08	0.05	0.06	0.06	0.25

Appendix

Table 30: Error counts on Wikidata evaluated by Gemini-2.5-Pro (Gemma3). We report the average number of errors per output across different error categories (lower is better). "All Categories" represents the total error count.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	34%	58%	15%	50%	93%
Gemma3-27B-IT-ZS	5%	34%	4%	63%	77%
WebNLG-SFT
Gemma3-1B-IT-SFT	98%	14%	29%	84%	100%
Gemma3-27B-IT-SFT	7%	5%	6%	6%	22%

Appendix

Table 31: Error rates on Wikidata evaluated by Gemini-2.5-Pro (Gemma3). Values represent the percentage of generated outputs containing at least one error of the specific type. The "All Categories" column reports the percentage of outputs containing any type of error. Lower percentages indicate better performance.

Model	# Real	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Zero-Shot (ZS)
GPT-4.1-ZS	–	0.01	0.36	0.00	0.00	0.37
Qwen3-1.7B-ZS	–	0.73	0.58	0.06	0.01	1.38
Qwen3-8B-ZS	–	0.30	0.27	0.02	0.00	0.59
Qwen3-32B-ZS	–	0.07	0.54	0.02	0.00	0.63
WebNLG-SFT

Appendix

Table 32: Error counts on Ice Hockey evaluated by GPT-5.1 (Qwen3). We report the average number of errors per output across different error categories (lower is better). # Real denotes the number of real seed instances used for distillation. "All Categories" represents the total error count. Bold indicates the best performing model (lowest total errors).

Model	# Real	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Zero-Shot (ZS)
GPT-4.1-ZS	–	1%	25%	0%	0%	26%
Qwen3-1.7B-ZS	–	44%	47%	6%	1%	73%
Qwen3-8B-ZS	–	24%	20%	2%	0%	40%
Qwen3-32B-ZS	–	5%	45%	2%	0%	47%
WebNLG-SFT

Appendix

Table 33: Error rates on Ice Hockey evaluated by GPT-5.1 (Qwen3). Values represent the percentage of generated outputs containing at least one error of the specific type. # Real denotes the number of real seed instances used for distillation. The "All Categories" column reports the percentage of outputs containing any type of error. Lower percentages indicate better performance.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Qwen3-1.7B-ZS	0.88	0.27	0.19	0.37	1.71
Qwen3-8B-ZS	0.39	0.07	0.06	0.10	0.62
Qwen3-32B-ZS	0.10	0.19	0.05	0.15	0.49
WebNLG-SFT
Qwen3-1.7B-SFT	1.59	0.12	0.27	0.44	2.42

Appendix

Table 34: Error counts on Ice Hockey evaluated by Gemini-2.5-Pro (Qwen3). We report the average number of errors per output across different error categories (lower is better). "All Categories" represents the total error count. Bold indicates the best performing model (lowest total errors).

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Qwen3-1.7B-ZS	53%	25%	17%	35%	91%
Qwen3-8B-ZS	33%	7%	4%	10%	46%
Qwen3-32B-ZS	8%	17%	5%	12%	33%
WebNLG-SFT
Qwen3-1.7B-SFT	79%	11%	26%	39%	97%

Appendix

Table 35: Error rates on Ice Hockey evaluated by Gemini-2.5-Pro (Qwen3). Values represent the percentage of generated outputs containing at least one error of the specific type. The "All Categories" column reports the percentage of outputs containing any type of error. Lower percentages indicate better performance.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	2.33	0.34	0.01	0.12	2.80
Gemma3-27B-IT-ZS	0.13	0.14	0.01	0.00	0.28
WebNLG-SFT
Gemma3-1B-IT-SFT	2.69	1.14	0.31	1.20	5.34
Gemma3-27B-IT-SFT	0.11	0.09	0.00	0.00	0.20

Appendix

Table 36: Error counts on Ice Hockey evaluated by GPT-5.1 (Gemma3). We report the average number of errors per output across different error categories (lower is better). "All Categories" represents the total error count.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	92%	26%	1%	12%	94%
Gemma3-27B-IT-ZS	12%	13%	1%	0%	24%
WebNLG-SFT
Gemma3-1B-IT-SFT	88%	62%	24%	78%	100%
Gemma3-27B-IT-SFT	10%	9%	0%	0%	19%

Appendix

Table 37: Error rates on Ice Hockey evaluated by GPT-5.1 (Gemma3). Values represent the percentage of generated outputs containing at least one error of the specific type. The "All Categories" column reports the percentage of outputs containing any type of error. Lower percentages indicate better performance.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	2.46	0.13	0.17	0.33	3.09
Gemma3-27B-IT-ZS	0.21	0.10	0.05	0.19	0.55
WebNLG-SFT
Gemma3-1B-IT-SFT	3.09	0.51	0.33	1.96	5.89
Gemma3-27B-IT-SFT	0.17	0.05	0.06	0.32	0.60

Appendix

Table 38: Error counts on Ice Hockey evaluated by Gemini-2.5-Pro (Gemma3).

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	92%	10%	14%	31%	98%
Gemma3-27B-IT-ZS	17%	10%	5%	15%	40%
WebNLG-SFT
Gemma3-1B-IT-SFT	93%	32%	26%	96%	99%
Gemma3-27B-IT-SFT	16%	5%	6%	29%	50%

Appendix

Table 39: Error rates on Ice Hockey evaluated by Gemini-2.5-Pro (Gemma3).

Model	# Real	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Zero-Shot (ZS)
GPT-4.1-ZS	–	0.34	0.32	0.10	0.00	0.76
Qwen3-1.7B-ZS	–	5.06	0.50	0.52	0.02	6.10
Qwen3-8B-ZS	–	2.52	0.34	0.46	0.05	3.37
Qwen3-32B-ZS	–	1.70	0.35	0.31	0.17	2.53
WebNLG-SFT

Appendix

Table 40: Error counts on OpenWeather evaluated by GPT-5.1 (Qwen3). We report the average number of errors per output across different error categories (lower is better). # Real denotes the number of real seed instances used for distillation. "All Categories" represents the total error count. Bold indicates the best performing open-source model (lowest total errors).

Model	# Real	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Zero-Shot (ZS)
GPT-4.1-ZS	–	29%	23%	10%	5%	49%
Qwen3-1.7B-ZS	–	98%	23%	36%	2%	99%
Qwen3-8B-ZS	–	94%	26%	37%	5%	98%
Qwen3-32B-ZS	–	83%	20%	25%	17%	92%
WebNLG-SFT

Appendix

Table 41: Error rates on OpenWeather evaluated by GPT-5.1 (Qwen3). Values represent the percentage of generated outputs containing at least one error of the specific type. # Real denotes the number of real seed instances used for distillation. The "All Categories" column reports the percentage of outputs containing any type of error. Lower percentages indicate better performance.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Qwen3-1.7B-ZS	6.30	0.10	0.74	0.05	7.19
Qwen3-8B-ZS	3.61	0.06	0.52	0.03	4.22
Qwen3-32B-ZS	2.84	0.07	0.75	0.03	3.69
WebNLG-SFT
Qwen3-1.7B-SFT	9.54	0.90	0.61	0.59	10.83

Appendix

Table 42: Error counts on OpenWeather evaluated by Gemini-2.5-Pro (Qwen3). We report the average number of errors per output across different error categories (lower is better). "All Categories" represents the total error count. Bold indicates the best performing model (lowest total errors).

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Qwen3-1.7B-ZS	100%	8%	42%	5%	100%
Qwen3-8B-ZS	100%	6%	42%	3%	100%
Qwen3-32B-ZS	96%	6%	47%	3%	97%
WebNLG-SFT
Qwen3-1.7B-SFT	95%	7%	31%	46%	99%

Appendix

Table 43: Error rates on OpenWeather evaluated by Gemini-2.5-Pro (Qwen3). Values represent the percentage of generated outputs containing at least one error of the specific type. The "All Categories" column reports the percentage of outputs containing any type of error. Lower percentages indicate better performance.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	3.02	1.05	0.57	0.02	4.66
Gemma3-27B-IT-ZS	3.57	1.00	0.34	0.07	4.98
WebNLG-SFT
Gemma3-1B-IT-SFT	2.65	1.69	0.16	0.98	5.48
Gemma3-27B-IT-SFT	2.26	0.06	0.12	0.04	2.48

Appendix

Table 44: Error counts on OpenWeather evaluated by GPT-5.1 (Gemma3). We report the average number of errors per output across different error categories (lower is better). "All Categories" represents the total error count. Bold indicates the best performing model (lowest total errors).

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	94%	64%	45%	2%	100%
Gemma3-27B-IT-ZS	93%	27%	25%	7%	97%
WebNLG-SFT
Gemma3-1B-IT-SFT	80%	76%	14%	66%	99%
Gemma3-27B-IT-SFT	31%	6%	4%	4%	43%

Appendix

Table 45: Error rates on OpenWeather evaluated by GPT-5.1 (Gemma3). Values represent the percentage of generated outputs containing at least one error of the specific type. The "All Categories" column reports the percentage of outputs containing any type of error. Lower percentages indicate better performance.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	4.06	0.65	0.57	0.09	5.37
Gemma3-27B-IT-ZS	5.72	0.10	0.69	0.10	6.52
WebNLG-SFT
Gemma3-1B-IT-SFT	5.64	0.35	0.37	1.08	7.44
Gemma3-27B-IT-SFT	2.86	0.05	1.61	0.33	4.85

Appendix

Table 46: Error counts on OpenWeather evaluated by Gemini-2.5-Pro (Gemma3).

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	100%	41%	46%	9%	100%
Gemma3-27B-IT-ZS	100%	9%	46%	1%	100%
WebNLG-SFT
Gemma3-1B-IT-SFT	98%	14%	29%	84%	100%
Gemma3-27B-IT-SFT	52%	1%	45%	26%	89%

Appendix

Table 47: Error rates on OpenWeather evaluated by Gemini-2.5-Pro (Gemma3).

Model	# Real	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Zero-Shot (ZS)
GPT-4.1-ZS	–	0.01	1.04	0.00	0.00	1.05
Qwen3-1.7B-ZS	–	0.42	0.61	0.08	0.11	1.22
Qwen3-8B-ZS	–	0.21	0.79	0.10	0.70	1.17
Qwen3-32B-ZS	–	0.03	0.42	0.04	0.00	0.49
WebNLG-SFT

Appendix

Table 48: Error counts on GSM Arena evaluated by GPT-5.1 (Qwen3). We report the average number of errors per output across different error categories (lower is better). # Real denotes the number of real seed instances used for distillation. "All Categories" represents the total error count. Bold indicates the best performing model (lowest total errors).

Model	# Real	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Zero-Shot (ZS)
GPT-4.1-ZS	–	1%	57%	0%	0%	57%
Qwen3-1.7B-ZS	–	33%	45%	8%	11%	71%
Qwen3-8B-ZS	–	19%	53%	9%	6%	69%
Qwen3-32B-ZS	–	3%	29%	4%	0%	34%
WebNLG-SFT

Appendix

Table 49: Error rates on GSM Arena evaluated by GPT-5.1 (Qwen3). Values represent the percentage of generated outputs containing at least one error of the specific type. # Real denotes the number of real seed instances used for distillation. The "All Categories" column reports the percentage of outputs containing any type of error. Lower percentages indicate better performance.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Qwen3-1.7B-ZS	0.65	0.25	0.29	0.15	1.34
Qwen3-8B-ZS	0.33	0.24	0.06	0.06	0.69
Qwen3-32B-ZS	0.04	0.03	0.03	0.00	0.10
WebNLG-SFT
Qwen3-1.7B-SFT	1.12	0.20	0.20	1.15	2.67

Appendix

Table 50: Error counts on GSM Arena evaluated by Gemini-2.5-Pro (Qwen3). We report the average number of errors per output across different error categories (lower is better). "All Categories" represents the total error count. Bold indicates the best performing model (lowest total errors).

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Qwen3-1.7B-ZS	46%	20%	25%	13%	72%
Qwen3-8B-ZS	30%	21%	6%	6%	47%
Qwen3-32B-ZS	4%	3%	3%	0%	10%
WebNLG-SFT
Qwen3-1.7B-SFT	65%	12%	17%	76%	94%

Appendix

Table 51: Error rates on GSM Arena evaluated by Gemini-2.5-Pro (Qwen3). Values represent the percentage of generated outputs containing at least one error of the specific type. The "All Categories" column reports the percentage of outputs containing any type of error. Lower percentages indicate better performance.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	1.25	2.64	0.07	0.07	4.03
Gemma3-27B-IT-ZS	0.07	0.34	0.03	0.00	0.44
WebNLG-SFT
Gemma3-1B-IT-SFT	1.90	0.75	0.13	0.82	3.60
Gemma3-27B-IT-SFT	0.15	0.13	0.07	0.02	0.37

Appendix

Table 52: Error counts on GSM Arena evaluated by GPT-5.1 (Gemma3). We report the average number of errors per output across different error categories (lower is better). "All Categories" represents the total error count. Bold indicates the best performing model (lowest total errors).

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	83%	89%	7%	7%	98%
Gemma3-27B-IT-ZS	6%	30%	3%	0%	44%
WebNLG-SFT
Gemma3-1B-IT-SFT	86%	47%	13%	59%	96%
Gemma3-27B-IT-SFT	12%	12%	7%	2%	29%

Appendix

Table 53: Error rates on GSM Arena evaluated by GPT-5.1 (Gemma3). Values represent the percentage of generated outputs containing at least one error of the specific type. The "All Categories" column reports the percentage of outputs containing any type of error. Lower percentages indicate better performance.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	1.48	1.47	0.18	0.08	3.21
Gemma3-27B-IT-ZS	0.10	0.07	0.01	0.00	0.18
WebNLG-SFT
Gemma3-1B-IT-SFT	2.29	0.45	0.27	1.13	4.14
Gemma3-27B-IT-SFT	0.23	0.08	0.05	0.03	0.39

Appendix

Table 54: Error counts on GSM Arena evaluated by Gemini-2.5-Pro (Gemma3).

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	84%	67%	14%	8%	99%
Gemma3-27B-IT-ZS	10%	7%	1%	0%	15%
WebNLG-SFT
Gemma3-1B-IT-SFT	90%	31%	23%	69%	96%
Gemma3-27B-IT-SFT	19%	8%	5%	2%	32%

Appendix

Table 55: Error rates on GSM Arena evaluated by Gemini-2.5-Pro (Gemma3).

Model	# Real	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Zero-Shot (ZS)
GPT-4.1-ZS	–	0.21	0.16	0.16	0.00	0.53
Qwen3-1.7B-ZS	–	1.29	0.44	0.36	0.02	2.11
Qwen3-8B-ZS	–	0.71	0.25	0.24	0.00	1.20
Qwen3-32B-ZS	–	0.68	0.05	0.25	0.01	0.99
WebNLG-SFT

Appendix

Table 56: Error counts on OWID evaluated by GPT-5.1 (Qwen3). We report the average number of errors per output across different error categories (lower is better). # Real denotes the number of real seed instances used for distillation. "All Categories" represents the total error count. Bold indicates the best performing model (lowest total errors).

Model	# Real	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Zero-Shot (ZS)
GPT-4.1-ZS	–	21%	12%	15%	0%	38%
Qwen3-1.7B-ZS	–	68%	42%	35%	2%	87%
Qwen3-8B-ZS	–	51%	22%	22%	0%	73%
Qwen3-32B-ZS	–	48%	5%	24%	1%	61%
WebNLG-SFT

Appendix

Table 57: Error rates on OWID evaluated by GPT-5.1 (Qwen3). Values represent the percentage of generated outputs containing at least one error of the specific type. # Real denotes the number of real seed instances used for distillation. The "All Categories" column reports the percentage of outputs containing any type of error. Lower percentages indicate better performance.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Qwen3-1.7B-ZS	1.75	0.33	0.43	0.06	2.57
Qwen3-8B-ZS	1.12	0.13	0.42	0.00	1.67
Qwen3-32B-ZS	1.18	0.03	0.37	0.02	1.60
WebNLG-SFT
Qwen3-1.7B-SFT	2.00	0.08	0.13	0.31	2.52

Appendix

Table 58: Error counts on OWID evaluated by Gemini-2.5-Pro (Qwen3). We report the average number of errors per output across different error categories (lower is better). "All Categories" represents the total error count. Bold indicates the best performing model (lowest total errors).

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Qwen3-1.7B-ZS	84%	32%	40%	6%	96%
Qwen3-8B-ZS	65%	13%	37%	0%	85%
Qwen3-32B-ZS	67%	3%	28%	2%	79%
WebNLG-SFT
Qwen3-1.7B-SFT	77%	6%	12%	30%	86%

Appendix

Table 59: Error rates on OWID evaluated by Gemini-2.5-Pro (Qwen3). Values represent the percentage of generated outputs containing at least one error of the specific type. The "All Categories" column reports the percentage of outputs containing any type of error. Lower percentages indicate better performance.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	2.40	0.35	0.39	0.00	3.14
Gemma3-27B-IT-ZS	0.74	0.04	0.12	0.00	0.90
WebNLG-SFT
Gemma3-1B-IT-SFT	2.76	0.28	0.32	0.50	3.86
Gemma3-27B-IT-SFT	0.60	0.00	0.03	0.14	0.77

Appendix

Table 60: Error counts on OWID evaluated by GPT-5.1 (Gemma3). We report the average number of errors per output across different error categories (lower is better). "All Categories" represents the total error count. Bold indicates the best performing model (lowest total errors).

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	93%	28%	30%	0%	97%
Gemma3-27B-IT-ZS	53%	3%	12%	0%	59%
WebNLG-SFT
Gemma3-1B-IT-SFT	50%	23%	32%	34%	92%
Gemma3-27B-IT-SFT	10%	0%	3%	14%	23%

Appendix

Table 61: Error rates on OWID evaluated by GPT-5.1 (Gemma3). Values represent the percentage of generated outputs containing at least one error of the specific type. The "All Categories" column reports the percentage of outputs containing any type of error. Lower percentages indicate better performance.

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	2.55	0.22	0.49	0.05	3.31
Gemma3-27B-IT-ZS	1.28	0.10	0.29	0.05	1.63
WebNLG-SFT
Gemma3-1B-IT-SFT	1.90	0.11	0.39	0.56	2.96
Gemma3-27B-IT-SFT	0.51	0.00	0.16	0.56	1.23

Appendix

Table 62: Error counts on OWID evaluated by Gemini-2.5-Pro (Gemma3).

Zero-Shot (ZS)
Model	Incorrect Fact	Not Checkable	Misleading	Other	All Categories
Gemma3-1B-IT-ZS	97%	20%	44%	5%	100%
Gemma3-27B-IT-ZS	72%	1%	26%	5%	83%
WebNLG-SFT
Gemma3-1B-IT-SFT	72%	10%	37%	48%	94%
Gemma3-27B-IT-SFT	28%	0%	16%	42%	68%

Appendix

Table 63: Error rates on OWID evaluated by Gemini-2.5-Pro (Gemma3).

System	Wikidata		IceHockey		GSM		Weather		OWID		NormAvg †
System	Score	Ratio	Score	Ratio	Score	Ratio	Score	Ratio	Score	Ratio	NormAvg †
Qwen3-1.7B-ZS	4.61	0.912	3.82	0.736	3.20	0.601	2.78	0.463	2.93	0.501	0.5795
Qwen3-1.7B-SFT	4.56	0.899	3.42	0.628	3.00	0.551	2.20	0.294	2.52	0.385	0.0000
Qwen3-32B-SFT / ZS	4.61	0.916	3.99	0.776	3.84	0.761	2.76	0.456	3.41	0.609	0.8908
Qwen3-1.7B-DDKD-Best	4.66	0.926	3.97	0.766	3.85	0.754	2.57	0.404	2.94	0.493	0.8150

Appendix

Table 64: Content coverage sanity check using GPT-5.1 as the judge. Each cell reports mean Coverage Score (1–5) and mean Estimated Coverage Ratio (0–1). For Wikidata (entity description), the judge evaluates completeness over all input facts; for other domains, it evaluates coverage over task-relevant important content (salient entities/attributes and numeric patterns) without exhaustive row-level matching for long tables. Qwen3-32B-SFT/ZS denotes the teacher variant selected per domain (Table 11 ). † NormAvg is the average across domains of min–max normalized coverage score within each domain (normalization computed over the systems in this table; higher is better).

System	Wikidata		IceHockey		GSM		Weather		OWID		NormAvg †
System	Score	Ratio	Score	Ratio	Score	Ratio	Score	Ratio	Score	Ratio	NormAvg †
Qwen3-1.7B-ZS	4.48	0.900	3.27	0.650	3.24	0.673	2.55	0.439	2.80	0.524	0.4833
Qwen3-1.7B-SFT	4.51	0.897	2.74	0.512	3.14	0.642	1.99	0.228	2.43	0.421	0.0857
Qwen3-32B-SFT / ZS	4.55	0.913	3.48	0.699	3.85	0.801	2.64	0.448	2.96	0.537	1.0000
Qwen3-1.7B-DDKD-Best	4.51	0.909	3.46	0.690	3.68	0.767	2.52	0.415	2.85	0.498	0.7540

Appendix

Table 65: Content coverage sanity check using Gemini-2.5-Pro as the judge. Each cell reports mean Coverage Score (1–5) and mean Estimated Coverage Ratio (0–1). For Wikidata (entity description), the judge evaluates completeness over all input facts; for other domains, it evaluates coverage over task-relevant important content (salient entities/attributes and numeric patterns) without exhaustive row-level matching for long tables. Qwen3-32B-SFT/ZS denotes the teacher variant selected per domain (Table 11 ). † NormAvg is the average across domains of min–max normalized coverage scores within each domain (normalization computed over the systems in this table; higher is better).

Text Generation

Text generation studies methods for producing coherent, useful language from prompts, data, and dialogue context.

See all 424 papers

DS $^2$ -Instruct: Domain-Specific Data Synthesis for Large Language Models Instruction Tuning
Mar 13, 2026Ruiyao Xu, Noelle I. Samia, Han LiuSynthetic Data Generation for Language Models LLM Fine-Tuning
Northwestern University
EmbGen: Teaching with Reassembled Corpora
May 19, 2026Arun K Lenin, Kai Rouse, Andrea Nicastro +1Synthetic Instruction Generation Domain Adaptation for LLMs
Commonwealth Bank of Australia Sydney, Australia
TAKE: Trajectory-Aware Knowledge Estimation for Text Dataset Distillation
Jun 13, 2026Tri-Nhan Vo, Dang Nguyen, Sunil GuptaDataset Distillation
Deakin Applied AI Initiative, Deakin University, Australia

Mar 13, 2026J/K move · Enter open · S save

DS $^2$ -Instruct: Domain-Specific Data Synthesis for Large Language Models Instruction Tuning

Ruiyao Xu, Noelle I. Samia, Han Liu

Northwestern University

Synthetic Data Generation for Language Models LLM Fine-Tuning Instruction Tuning

Open paper AskPDF arXiv

Adapting Large Language Models (LLMs) to specialized domains requires high-quality instruction tuning datasets, which are expensive to create through human annotation. Existing data synthesis methods focus on general-purpose tasks and fail to capture domain-specific terminology and reasoning patterns. To address this, we introduce DS

^2

-Instruct, a zero-shot framework that generates domain-specific instruction datasets without human supervision. Our approach first generates task-informed keywords to ensure comprehensive domain coverage. It then creates diverse instructions by pairing these keywords with different cognitive levels from Bloom's Taxonomy. Finally, it uses self-consistency validation to ensure data quality. We apply this framework to generate datasets across seven challenging domains, such as mathematics, finance, and logical reasoning. Comprehensive evaluation demonstrates that models fine-tuned on our generated data achieve substantial improvements over existing data generation methods.

Abstract

Text Generation

Explore similar work

DS2^22-Instruct: Domain-Specific Data Synthesis for Large Language Models Instruction Tuning

EmbGen: Teaching with Reassembled Corpora

TAKE: Trajectory-Aware Knowledge Estimation for Text Dataset Distillation

DS $^2$ -Instruct: Domain-Specific Data Synthesis for Large Language Models Instruction Tuning