Synthetic Data Generation for Training Diversified Commonsense Reasoning Models

Tianhui Zhang; Bei Peng; Danushka Bollegala

cs.CLMar 18, 2026

Synthetic Data Generation for Training Diversified Commonsense Reasoning Models

Authors: Tianhui Zhang, Bei Peng, Danushka Bollegala

Organizations: University of Liverpool · University of Sheffield · Amazon

Abstract

Conversational agents are required to respond to their users not only with high quality (i.e. commonsense bearing) responses, but also considering multiple plausible alternative scenarios, reflecting the diversity in their responses. Despite the growing need to train diverse commonsense generators, the progress of this line of work has been significantly hindered by the lack of large-scale high-quality diverse commonsense training datasets. Due to the high annotation costs, existing Generative Commonsense Reasoning (GCR) datasets are created using a small number of human annotators, covering only a narrow set of commonsense scenarios. To address this training resource gap, we propose a two-stage method to create the first-ever synthetic dataset CommonSyn for diversified (GCR). The model fine-tuned on our synthetic data jointly increase both generation diversity and quality compared with vanilla models and the model fine-tuned on human-crafted dataset across different size Large Language Models (LLMs)

Figures & tables

Figure 1: Quality–Diversity trade-off for representative models. The x-axis represents the semantic diversity (Self-CosSim,

\uparrow

) and the y-axis represents the generation quality (Overall,

\uparrow

). While vanilla models (hollow markers) often suffer from either low quality or limited diversity, its fine-tuned version on our synthetic data, CommonSyn (solid markers), consistently pushes the performance frontier towards the top-right quadrant, achieving a superior Pareto improvement across diverse model families.

Statistics	CommonSyn	CommonGen Train
# Concept-Sets	20, 742	32,651
– Size = 3	9,847	25,020
– Size = 4	6,857	4,240
– Size = 5	4,038	3,391
# Sentences	83,184	67,389
per Concept-Set	4.01	2.06

Table 1: Comparison between our synthetic dataset CommonSyn and the CommonGen training set in terms of scale and structure.

Model	Dataset	Quality Metrics ( $\uparrow$ )			Diversity Metrics ( $\uparrow$ )
Model	Dataset	Win-Tie	Cov.	Overall	S-BLEU3 †	S-BLEU4 †	Vendi	S-Cos †
Llama-3.1-8B-Inst	Vanilla	19.0	84.7	16.1	73.4	80.3	22.8	35.1
	CommonGen	31.7	85.8	27.2	76.4	84.7	21.2	30.7
	CommonSyn	47.3 **	94.5	44.7	75.5	82.9	22.7	34.2
Qwen-2.5-7B-Inst.	Vanilla	42.9	92.3	39.6	39.2	44.1	15.8	16.3
	CommonGen	32.5	88.6	28.8	73.2	81.8	21.4	31.2

Table 2: Main results for the top-6 performing models. When fine-tuned on our proposed dataset ( CommonSyn ), we observe consistent improvements achieving the best balance between quality and diversity, surpassing both the vanilla model (prior to fine-tuning) and human-annotated CommonGen data ( ** :

p<0.01

). The complete results are shown in Table 14 . Diversity metrics marked with

\dagger

are inverted by subtracting from 1.

Baselines
Method	Quality ( $\uparrow$ )			Diversity ( $\uparrow$ )
Method	Win-Tie	Cov.	Overall	S-B4 †	Vendi	S-Cos †
CommonGen	31.7	85.8	27.2	84.7	21.2	30.7
PersonaHub ( Ge et al., 2024 )	20.8	94.0	19.6	78.1	23.8	35.6
Prismatic ( Jung et al., 2025 )	44.2	91.6	40.4	82.9	21.5	31.1
Sentence Generation Methods

Table 3: Comparison of different data sources and selection strategies on Llama-3.1-8B. CommonSyn effectively balances quality and diversity, maintaining high Coverage. Metrics marked with

\dagger

are inverted (by subtracting from 1) such that higher values indicate better performance.

Role	Prompt
System	Given several keywords, generate one coherent sentence that contains all the required keywords using background commonsense knowledge:
User	{Concept Set}
Assistant	{Sentence}

Table 4: Prompts used for training samples for CommonSyn.

Task	Model	Quality ( $\uparrow$ )	Diversity ( $\uparrow$ )
Task	Model	Win-Tie	S-B4 †	Vendi	S-Cos †
ComVE	Vanilla	75.3	77.3	17.7	29.8
	CommonGen	41.7	86.3	21.6	44.9
	CommonSyn	78.9	85.9	18.1	30.0
$\alpha$ -NLG	Vanilla	54.6	83.5	18.8	24.2
	CommonGen	25.9	81.3	22.6	35.3

Table 5: Cross-task generalization on unseen generative commonsense tasks. Models fine-tuned on human data ( CommonGen ) suffer from severe quality drop . In contrast, model fine-tuned on CommonSyn generalises well, outperforming vanilla baselines while maintaining diversity. Metrics marked with

\dagger

are inverted.

Dataset	CSQA	CSQA2	PIQA	Avg.
Vanilla	70.3	51.8	69.2	63.8
CommonGen	53.6	60.4	67.0	60.1
CommonSyn	70.8	49.0	71.3	63.7

Table 6: Zero-shot accuracy (%) on discriminative commonsense QA tasks. Training on human-annotated data causes catastrophic forgetting (avg.

-3.7

pp), whereas CommonSyn preserves general reasoning capabilities (avg.

-0.1

pp).

Method	Unique	% Unseen	% Unseen
	Concepts	Concepts	Triples
Human (CommonGen)	3,677	75.36	100
Direct Generation	805	52.55	96.65
Augment (Add- $C$ )	3,791	72.80	99.65
2-Seed Expansion	4,260	75.28	99.62

Table 7: Comparison of concept set construction strategies (

N=20k

). 2-Seed Expansion gets the highest number of unique concepts and maintains high novelty relative to the test set, indicating no data leakage.

Figure 2: Trade-off between Quality and Diversity. The dashed line indicates the Pareto frontier. CommonSyn (Red Star) strictly outperforms the gradient-hybrid method Q&G-based (Brown Cross) by achieving higher quality at the same diversity level.

Strategy	Quality ( $\uparrow$ )			Diversity ( $\uparrow$ )
Strategy	Win-Tie	Cov.	Overall	S-B4 †	Vendi	S-Cos †
Q-based	49.0	94.1	46.1	78.5	21.4	30.8
D-based	45.6	94.3	43.0	80.6	22.2	32.9
G-based	43.9	94.4	41.4	83.8	23.0	35.1
Q&G-based	45.7	94.7	43.3	81.8	22.7	34.2
CommonSyn	47.3	94.5	44.7	82.9	22.7	34.2

Table 8: Comparison of selection strategies. CommonSyn outperforms the gradient-based hybrid ( Q&G-based ) in quality while matching its diversity, validating the effectiveness of our 2-stage selection.

Input Concepts	Human Reference	Vanilla	CommonGen	CommonSyn (Ours)
dog, throw, frisbee, catch	My dog likes to catch frisbees after I throw them.	As the sun was setting… a energetic dog awaited its owner’s signal to throw the frisbee…	A golden colored dog throws a frisbee and catches it. ✗(Logic)	I throw the frisbee, and my dog leaps to catch it mid-air.
ride, dog, run, shoot, horse	A dog runs by while the man rides a horse and shoots.	As the rodeo rider rode his horse at full speed, he expertly steered it… (Misses: dog, shoot)	…shoots a watermelon with a water pistol on a dog . ✗(Nonsense)	She rode the horse past a field where her dog ran and someone shot at a deer.
instruction, tie, neck, give	I had to give instructions to tie the tie around her son’s neck.	To properly tie a neck tie, follow the instructions carefully…	tied up in knots how to give instructions for tying a tie. ✗(Grammar)	In class, the teacher gives detailed instruction on how to properly tie a neck tie .

Table 9: A comparison of sentences generated by different methods. Although CommonSyn generates concise, logically sound, and grammatically correct sentences that fully cover the input concepts, other approaches often suffer from verbosity (Vanilla), commonsense violations (CommonGen), or incorrect grammar.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Num of Concept Sets	Quality ( $\uparrow$ )			Diversity ( $\uparrow$ )
Num of Concept Sets	Win-Tie	Cov.	Overall	S-B4 †	Vendi	S-Cos †
5k	46.0	94.4	43.4	82.2	22.4	33.5
10k	46.2	94.3	43.6	82.7	17.3	34.3
15k	45.0	94.4	42.5	82.6	22.5	33.9
20k	46.6	94.0	43.8	82.8	22.8	34.6

Appendix

Table 10: Performance obtained with numbers of unique concept sets (fixed total sentence sample size = 40k). Consistently high performance is seen across concept set sizes, indicating high data efficiency of our proposed method.

Input Concepts	Vanilla	CommonGen	CommonSyn (Ours)
take, shot, goal, player	1. The player took a shot with the intention of scoring the goal in the upcoming match. 2. The player takes a shot with the intention of scoring the goal in the ongoing soccer match. 3. The goal keeper took a daring shot at goal, hoping to catch the player off guard. 4. The player takes a shot with the intention of scoring the goal in the ongoing soccer match.	1. soccer player takes a shot on goal during their match . 2. football player takes a shot on goal during a friendly match . 3. soccer player taking a shot for the goal during the match . 4. football player takes a shot on goal during a training session .	1. The player takes a shot to score the final goal in the championship. 2. During the intense match, the player took a bold shot toward the goal. 3. The soccer player took a powerful shot to score the goal. 4. The soccer player takes a fierce shot towards the goal, hoping to score.
pan, stove, cook, food	1. She can pan fry the chicken on the stove to cook the delicious food. 2. In the morning, I like to cook delicious food on a stove using a pan. 3. She uses a stove to cook food in a pan. 4. She uses a pan on the stove to cook food for dinner.	1. A young boy cooking food with a pan on the stove. 2. A man is cooking food in a pan on a stove. 3. A man is cooking food in a pan on a stove. 4. a man cooks with a frying pan on the stove.	1. In the morning, he stands at the stove to cook breakfast in a pan. 2. I use a pan on my stove to cook food every day. 3. Every Sunday, I cook a hearty meal on my pan over the stove. 4. He carefully cooked the pasta on the stovetop pan for his family.
dog, throw, frisbee, catch	1. The dog happily catches the frisbee that was thrown by its owner. 2. The dog threw the frisbee and then eagerly waited to catch it. 3. The dog threw the frisbee and then eagerly waited to catch it. 4. The dog threw the frisbee and then patiently waited to catch it again.	1. A young boy throwing a frisbee to his dog as he catches it. 2. A woman is throwing a frisbee for her dog to catch. 3. Two dogs play tug with a frisbee. 4. Dog throws the Frisbee for another dog to catch.	1. My dog stands excitedly as I throw the frisbee; it always loves a good catch. 2. She throws a frisbee for her dog to catch in the park. 3. My dog always catches the frisbee I throw during our evening walks. 4. The owner throws the frisbee for his dog to catch, enjoying the sunny day.
coffee, drink, newspaper, read	1. He likes to drink coffee while reading the newspaper in the morning. 2. He likes to read the newspaper while drinking coffee in the morning. 3. He likes to read the newspaper while drinking coffee in the morning. 4. He likes to drink coffee while reading the newspaper in the morning.	1. young woman reading a newspaper while drinking coffee and smoking. 2. young women drinking a coffee and reading the newspaper at home. 3. A woman is reading a newspaper while drinking coffee. 4. The man is drinking coffee while reading a newspaper.	1. She enjoys a quiet morning with a cup of coffee and the local newspaper, sipping it as she reads. 2. He likes to drink his coffee while reading the morning newspaper. 3. Every morning, he drinks a cup of coffee and reads the newspaper. 4. After work, I sit down to read the newspaper and drink coffee.
ball, throw, pitcher, batter	1. The pitcher threw the ball to the batter during the baseball game. 2. The pitcher threw the ball to the batter during the baseball game. 3. The pitcher threw the ball to the batter during the baseball game. 4. The pitcher throws the ball to the batter during the baseball game.	1. The pitcher has just thrown a pitch to home plate and the batter is swinging. 2. A batter is waiting for the ball to be thrown by the pitcher. 3. a ball thrown by a pitcher and the batter. 4. pitcher throws a ball at the batter.	1. The pitcher throws the ball, but the batter swings and misses, watching it sail past. 2. The batter waited for the ball to be thrown by the pitcher. 3. During practice, a young batter focuses on how to react to a thrown ball by the pitcher. 4. The pitcher throws a curve ball past the batter.

Appendix

Table 11: Extended case study showing four sampled generations per model for each concept set. CommonGen often ignores or distorts input concepts, while Vanilla exhibits high redundancy. CommonSyn produces diverse, fluent, and concept-consistent outputs that respect real-world roles and relationships.

Task Input	Vanilla (Zero-shot)	CommonGen (Human)	CommonSyn (Ours)
Task 1: ComVE (Generate explanation for why the statement is false)
Input: sugar is used to make coffee sour	That’s a counterfactual statement. (Refusal/Vague)	“The image shows sugar is added to make coffee sour.” ✗ (Hallucination: no image exists)	Sugar is typically added to make coffee sweeter, not sourer. ✔ (Correct commonsense reasoning)
Input: The inverter was able to power the continent.	It defies the fundamental laws of physics and engineering.	The inverter in the car wasn’t able to power the entire continent of Europe.	The inverter, no matter how powerful, cannot power an entire continent due to its size and energy requirements.
Task 2: $\alpha$ -NLG (Generate a hypothesis connecting Observation 1 and 2)
O1: Timmy was running fast to class. O2: Timmy later found out that he failed.	Timmy likely ran to class because he forgot that the exam was scheduled for a specific time and he didn’t want to be late.	Because Timmy was running fast to class, he made it to his exam on time and therefore he passed. ✗ (Contradicts O2: he failed)	Timmy’s need to be on time for the exam, combined with his earlier desire to be punctual, suggests he likely rushed to class, possibly missing key information. ✔ (Plausible inference consistent with failure)
O1: Music was Kat’s life. O2: She completed her studies with honors and is now a renowned pianist.	Kat enjoyed it so much that she played all of the time.	Music was Kat’s life and she is now a renowned pianist. ✗ (Mere repetition, no causal link)	Kat’s dedication to music fueled her academic success, leading to her distinguished career as a pianist.

Appendix

Table 12: Qualitative comparison on downstream generative tasks. Models fine-tuned on human-authored CommonGen data generalise poorly: they hallucinate non-existent images (ComVE), contradict input facts (

\alpha

-NLG), or generate off-topic descriptions (ROCStories). In contrast, CommonSyn shows robust commonsense reasoning and task adherence across unseen domains.

Task	Inference Prompt
ComVE	Given an implausible or counterfactual statement, generate one short explanation ( $\leq 22$ words) that explains why it is implausible or counterfactual using background commonsense knowledge.
$\alpha$ -NLG	Given an initial observation and a later observation, generate a short hypothesis ( $\leq 22$ words) that bridges Observation 1 and Observation 2 using background commonsense knowledge.
ROCStories	Read the following 4-sentence story context and write a short and plausible ending ( $\leq 22$ words) to this story using background commonsense knowledge.

Appendix

Table 13: Inference prompts used for zero-shot evaluation on downstream generative tasks. All outputs are constrained to

\leq 22

words.

Model	Dataset	Quality Metrics ( $\uparrow$ )			Diversity Metrics ( $\uparrow$ )
Model	Dataset	Win-Tie	Cov.	Overall	S-BLEU3	S-BLEU4	Vendi	S-Cos
Llama-3.1-8B-Inst	Vanilla	19.0	84.7	16.1	73.4	80.3	22.8	35.1
	CommonGen	31.7	85.8	27.2	76.4	84.70	21.2	30.7
	CommonSyn	47.3	94.5	44.7	75.5	82.9	22.7	34.2
Llama-3.2-1B-Inst	Vanilla	6.0	64.3	3.86	79.5	84.8	27.2	49.7
	CommonGen	20.9	84.0	17.56	79.2	87.0	21.2	34.1

Appendix

Table 14: Full experimental results across all 11 evaluated models. For S-BLEU and S-Cos, higher scores indicate greater diversity.

Synthetic Data Generation for Language Models

Synthetic data generation for language models creates artificial text, tasks, and interaction traces to train or evaluate models.

See all 284 papers

CHIMERA: Compact Synthetic Data for Generalizable LLM Reasoning
Mar 1, 2026Xinyu Zhu, Yihao Feng, Yanchao Sun +5LLM Reasoning Synthetic Data Generation for Language Models
Department of Computer Science, University of Virginia, Charlottesville, VA, USA · Apple, Cupertino, CA, USA · Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
Learning from Synthetic Data Improves Multi-hop Reasoning
Mar 2, 2026Anmol Kabra, Yilun Yin, Albert Gong +6RL for Language Model Reasoning Multi-Hop Reasoning
Cornell University · University of Cambridge · Stanford University
It's All Training: A Fully Synthetic Single-Stage Recipe for LLMs
Sep 29, 2026Pierre-Carl Langlais, Pieter Delobelle, Yannick Detrois +7Language Model Pretraining Synthetic Data Pretraining
PleIAs · Sorbonne Center for Artificial Intelligence · Sciences Po Médialab +6

Mar 1, 2026J/K move · Enter open · S save

CHIMERA: Compact Synthetic Data for Generalizable LLM Reasoning

Xinyu Zhu, Yihao Feng, Yanchao Sun, Xianzhi Du+4

Department of Computer Science, University of Virginia, Charlottesville, VA, USA · Apple, Cupertino, CA, USA · Department of Computer Science, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

LLM Reasoning Synthetic Data Generation for Language Models Scientific Reasoning in Language Models

Open paper AskPDF arXiv

Large Language Models (LLMs) have recently exhibited remarkable reasoning capabilities, largely enabled by supervised fine-tuning (SFT)- and reinforcement learning (RL)-based post-training on high-quality reasoning data. However, reproducing and extending these capabilities in open and scalable settings is hindered by three fundamental data-centric challenges: (1) the cold-start problem, arising from the lack of seed datasets with detailed, long Chain-of-Thought (CoT) trajectories needed to initialize reasoning policies; (2) limited domain coverage, as most existing open-source reasoning datasets are concentrated in mathematics, with limited coverage of broader scientific disciplines; and (3) the annotation bottleneck, where the difficulty of frontier-level reasoning tasks makes reliable human annotation prohibitively expensive or infeasible. To address these challenges, we introduce CHIMERA, a compact synthetic reasoning dataset comprising 9K samples for generalizable cross-domain reasoning. CHIMERA is constructed with three key properties: (1) it provides rich, long CoT reasoning trajectories synthesized by state-of-the-art reasoning models; (2) it has broad and structured coverage, spanning 8 major scientific disciplines and over 1K fine-grained topics organized via a model-generated hierarchical taxonomy; and (3) it employs a fully automated, scalable evaluation pipeline that uses strong reasoning models to cross-validate both problem validity and answer correctness. We use CHIMERA to post-train a 4B Qwen3 model. Despite the dataset's modest size, the resulting model achieves strong performance on a suite of challenging reasoning benchmarks, including GPQA-Diamond, AIME 24/25/26, HMMT 25, and Humanity's Last Exam, approaching or matching the reasoning performance of substantially larger models such as DeepSeek-R1 and Qwen3-235B.

Abstract

Synthetic Data Generation for Language Models

Explore similar work

CHIMERA: Compact Synthetic Data for Generalizable LLM Reasoning

Learning from Synthetic Data Improves Multi-hop Reasoning

It's All Training: A Fully Synthetic Single-Stage Recipe for LLMs