A widely adopted strategy for model enhancement is to use synthetic data generated by a stronger model for supervised fine-tuning (SFT). However, for emerging reasoning models like Qwen3-8B, this approach often fails to improve reasoning capabilities and can even lead to a substantial drop in performance. In this work, we identify substantial stylistic divergence between teacher generated data and the distribution of student as a major factor impacting SFT. To bridge this gap, we propose a Teacher-Student Cooperation Data Synthesis framework (TESSY), which interleaves teacher and student models to alternately generate style and non-style tokens. Consequently, TESSY produces synthetic sequences that inherit the advanced reasoning capabilities of the teacher while maintaining stylistic consistency with the distribution of the student. In experiments on code generation using GPT-OSS-120B as the teacher, fine-tuning Qwen3-8B on teacher-generated data leads to performance drops of 3.25% on LiveCodeBench-Pro and 10.02% on OJBench, whereas TESSY achieves improvements of 11.25% and 6.68%.
Figures & tables
Figure 1: Illustration of the Teacher–Student Cooperation Data Synthesis Framework. Blue and Pink denote the capability-related text spans generated in the responses of GPT-OSS-120B and Qwen3-8B, respectively. Green and Purple indicate the stylistic text segments produced by GPT-OSS-120B and Qwen3-8B. The objective of our synthesis framework is to delegate the generation of capability-related text to GPT-OSS-120B, while assigning the generation of stylistic text to the student model Qwen3-8B.
Figure 2: Overview of the teacher–student cooperation data synthesis framework. The teacher and student models alternately generate capability and style tokens, while the boundary predictors enforce truncation to the exact capability or style spans.
In-domain Test Sets
Out-of-domain Test Sets
LCB-V5
LCB-V6
LCB-Pro
OJBench
GPQA-Diamond
AIME2024
AIME2025
OlympiadBench
Qwen3-8B
55.09
49.58
25.35
18.75
60.16
76.67
69.17
69.81
+ Reject-Sampling
54.55
48.00
26.06
18.32
60.61
75.42
66.77
69.53
+ Self-Distillation
51.50
43.43
19.26
14.22
60.61
71.98
62.29
67.80
+ Teacher-Answer
35.33
33.14
21.39
14.01
60.42
74.79
64.48
69.40
+ Teacher-Think
47.90
43.43
24.36
12.50
54.29
77.81
69.06
69.59
Table 1: Performance comparison between TESSY and SFT baselines on in-domain code generation and out-of-domain test sets, where LiveCodeBench is abbreviated as LCB for brevity.
Figure 4Figure 5
Figure 7: Comparison of TESSY and Teacher-Only across Base and Thinking versions of models.
Figure 8: Visualization of the data distribution changes caused by TESSY across three teacher models.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
LCB-V5
LCB-V6
LCB-Pro
OJBench
DeepSeek-R1-Distill-Llama-8B
34.73
33.71
17.56
5.82
+ Self-Distillation
19.76
24.00
8.07
5.82
+ Teacher-Only
38.32
36.57
24.22
11.42
+ TESSY
52.10
41.71
26.20
12.50
Appendix
Table 2: Performance comparison using the DeepSeek-R1-Distill-Llama-8B as the student model.
Figure 9Figure 10Figure 11
LCB-V5
LCB-V6
LCB-Pro
OJBench
Qwen3-8B
55.09
49.58
25.35
18.75
+ TESSY (Predictors trained on 500 samples)
59.58
54.00
35.13
23.98
+ TESSY (Predictors trained on 100K samples)
60.68
55.15
35.84
26.51
Appendix
Table 3: Comparison of training performance using TESSY synthetic data with boundary predictors trained on different dataset sizes. Different from Table 1 (80K samples), we used 40K synthesized samples here to reduce computational cost for comparison.
Figure 13
Figure 17: Comparison of the average output similarity between Qwen3-8B and three teacher models.
Figure 18: Comparison of training loss curves on Base and Reasoning models using SFT data synthesized by TESSY and the Teacher-Only.
Figure 19: Comparison of frequently used words across models. Darker colors indicate higher word frequencies in the outputs of the corresponding models.
Figure 20: Average ratio of teacher-generated and student-generated tokens under the TESSY framework.
Prompt
You are a text analysis expert. Task: Extract all spans of text that are transitional, filler, or tone-setting phrases. What to extract: • Include phrases or sentences that: – Express hesitation, tone, or attitude (e.g., “well”, “okay”, “so”, “let’s see”, “I think”) – Indicate transition or setup (e.g., “to begin with”, “in this case”, “for example”, “but if”) – Serve as narration or connection, not analysis • Do not include: – Actual reasoning, deduction, or explanation – Code or formula descriptions – Problem-solving steps Output format (STRICT JSON): • Return a JSON array of strings, e.g.: ["<span 1>", "<span 2>", ...] • Rules: 1. Each span must be copied verbatim from the original text. 2. Preserve order of appearance. 3. If there are none, return an empty list: [] 4. Output only the JSON array — no explanation or extra text. <input_text> {think_text} </input_text>
Appendix
Table 4: Prompt used to annotate the training data of boundary predictor.
Problem Description
Given a transcript of a Fizz Buzz game (where numbers divisible by a are replaced with “Fizz”, by b with “Buzz”, and by both with “FizzBuzz”), find any valid values of a and b that could generate the transcript. Input: Two integers c and d ( 1≤c≤d≤105 ) indicating the range, followed by d−c+1 tokens (integers or “Fizz”/“Buzz”/“FizzBuzz”). Output: Two positive integers a and b ( 1≤a,b≤106 ) consistent with the transcript. If multiple solutions exist, output any one.
Response of Qwen3-8B-Base
To solve this problem, we need to determine the values of a and b that could have generated the given transcript. The key insight is that the transcript consists of numbers and strings (“Fizz”, “Buzz”, “FizzBuzz”) that correspond to multiples of a and b … To find a and b , we can: … • Identify the positions of “Fizz” and “Buzz” in the transcript. • … Approach • Identify the positions of “Fizz” and “Buzz”: … Solution Code #include <iostream> ... Explanation • Input Reading: – The program reads the range [c,d] and the transcript. • Position Identification: – The program identifies the positions of “Fizz” and “Buzz” in the transcript. • Validation Loop: – The program iterates over possible values of a and b and checks if they satisfy the conditions for the transcript. • Output: – The first valid pair (a,b) is output, and the program terminates. This approach ensures that we find a valid pair (a,b) that could have generated the given transcript.
Appendix
Table 5: Example output of Qwen3-8B-Base
Problem Description
Given a transcript of a Fizz Buzz game (where numbers divisible by a are replaced with “Fizz”, by b with “Buzz”, and by both with “FizzBuzz”), find any valid values of a and b that could generate the transcript. Input: Two integers c and d ( 1≤c≤d≤105 ) indicating the range, followed by d−c+1 tokens (integers or “Fizz”/“Buzz”/“FizzBuzz”). Output: Two positive integers a and b ( 1≤a,b≤106 ) consistent with the transcript. If multiple solutions exist, output any one.
Thinking of Student Model (Qwen3-8B)
<think> Okay, let’s see. I need to solve this Fizz Buzz reverse problem … First, I need to understand what the transcript implies about a and b … Let me think about how to model this … But wait, the transcript could have numbers that are not divisible by a or b. So for each entry in the transcript, we can derive constraints on a and b. The challenge is to find a and b that satisfy all these constraints. Let me think about how to approach this … But how do I find a and b that satisfy all these constraints? … Thus, the code is implemented as described. </think>
Thinking of Teacher Model (GPT-OSS-120B)
<think> We need to solve the reverse FizzBuzz problem: given a transcript from c to d inclusive, each entry is either a number (as integer string) or one of the words “Fizz”, “Buzz”, “FizzBuzz”. The transcript is guaranteed to be consistent with some a,b (1 ≤ a,b ≤ 1e6). Need to output any such a,b. Constraints: c,d up to 1e5 length (max length of transcript is up to 1e5). a,b up to 1e6. Goal: find any pair (a,b) that matches. We need to parse transcript: for each integer i from c to d, we have output string s_i. If s_i is a number, it must be exactly the integer i (as decimal). If s_i is “Fizz”, then i is multiple of a but not b. If “Buzz”: multiple of b but not a. If “FizzBuzz”: multiple of both a and b (i.e., multiple of lcm(a,b)) … We need to find any a,b. This is like solving constraints. Observation: The numbers in transcript that are numeric give us direct constraints: … … Let’s write code. </think>
Thinking of TESSY (Teacher: GPT-OSS-120B, Student: Qwen3-8B)
Appendix
Table 6: Comparison of thinking content generated by the teacher, student, and TESSY for the same programming problem, with TESSY portions generated by the student highlighted in purple .
Problem Description
Given a transcript of a Fizz Buzz game (where numbers divisible by a are replaced with “Fizz”, by b with “Buzz”, and by both with “FizzBuzz”), find any valid values of a and b that could generate the transcript. Input: Two integers c and d ( 1≤c≤d≤105 ) indicating the range, followed by d−c+1 tokens (integers or “Fizz”/“Buzz”/“FizzBuzz”). Output: Two positive integers a and b ( 1≤a,b≤106 ) consistent with the transcript. If multiple solutions exist, output any one.
Final Answer of Student Model (Qwen3-8B)
To solve the Fizz Buzz reverse problem , we need to determine possible values of integers a and b that could generate a given transcript of the game. The transcript consists of numbers and the words “Fizz”, “Buzz”, and “FizzBuzz” corresponding to multiples of a , b , and both a and b respectively. Approach • Parse Input: Read the range of the transcript and the transcript itself. • Classify Entries: Identify which entries are “Fizz”, “Buzz”, “FizzBuzz”, or numbers. This helps in determining constraints on a and b . • Determine Constraints: For entries that are “Fizz” or “FizzBuzz”, a must divide the corresponding number… • Generate Candidates: For “Fizz” and “FizzBuzz” entries, compute the GCD of all such numbers and generate all its divisors as potential values for a … • Validate Candidates: Check all combinations of a and b to find a valid pair that satisfies all constraints from the transcript. Solution Code #include <bits/stdc++.h> using namespace std; ... Explanation • Input Parsing: The input is read and stored in a vector of strings representing the transcript. • Classifying Entries: We classify each entry to determine which numbers are divisible by a , b , or both. • Generating Candidates: Using the GCD of relevant numbers, we generate potential values for a and b . If no such entries exist, we find the smallest number that does not divide any number in the transcript. • Validation: For each candidate pair (a,b) , we check if it satisfies all constraints from the transcript … This approach ensures that we efficiently narrow down the possible values of a and b and validate them against the transcript constraints.
Final Answer of Teacher Model (GPT-OSS-120B)
Solution Explanation For every integer i , … 1. Reformulating the conditions S_Fizz = { i | type(i) = Fizz } … 2. Determining a If A is empty, a never has to divide any number, we may choose any value that … 3. Algorithm read c , d … 4. Correctness Proof • Lemma 1: If A is non - empty, any integer x that divides every element of A must divide G = gcd(A) ; conversely every divisor of G divides every element of A . Proof: … • … 5. Complexity Analysis N=d\textendashc+1(≤105) … Both limits are easily satisfied. 6. Reference Implementation #include <bits/stdc++.h> using namespace std; ... The program follows exactly the algorithm proven correct above and conforms to the GNU C++17 compiler.
Appendix
Table 7: Comparison of final answer content generated by the teacher and student models for the same programming problem
Training student models on synthetic data generated by strong teacher models is a promising way to distilling the capabilities of teachers. However, recent studies show that stronger models are not always optimal teachers, revealing a mismatch between teacher outputs and student learnability. To address this issue, we propose PerSyn (Personalized data Synthesis), a novel synthesis strategy that operates under a new Route then Generate'' paradigm to create data tailored to each student model, enabling it to learn more effectively. Specifically, PerSyn first assigns each prompt to its optimal teacher via a query-level router that jointly considers student learnability and teacher response quality. Each teacher then synthesizes data only for its assigned prompts, making the process more efficient than the conventional Generate then Select'' paradigm, where all teachers must generate parallel responses for the entire prompt set before constructing the final dataset. Extensive experiments across different model families and scales demonstrate that PerSyn consistently achieves superior or comparable performance to all baselines in instruct tuning and math reasoning settings. Further analysis verifies the effectiveness of PerSyn and offers extra insights to propel future research.
Hengyuan Zhang, Shiping Yang, Xiao Liang +8
The University of Hong Kong · Simon Fraser University · University of California, Los Angeles +3
Distilling large reasoning models is essential for making Long-CoT reasoning practical, as full-scale inference remains computationally prohibitive. Existing curation-based approaches select complete reasoning traces post-hoc, overlooking collaboration among heterogeneous teachers and lacking dynamic exploration, which leads to redundant sampling and missed complementary reasoning. We introduce CoRD, a collaborative multi-teacher decoding framework that performs step-wise reasoning synthesis guided by predictive perplexity-based scoring and beam search. This enables heterogeneous LRMs to jointly construct coherent reasoning trajectories while efficiently preserving diverse, high-potential hypotheses. Experiments show that CoRD produces higher-quality reasoning data and achieves near teacher-level student performance with fewer, structured supervision signals, without substantial efficiency overhead. CoRD further generalizes well to out-of-domain and open-ended settings. The dataset and model are available at https://github.com/DISL-Lab/CoRD.
Reinforcement Learning (RL) has been shown to significantly boost reasoning capabilities of large language models (LLMs) in math, coding, and multi-hop reasoning tasks. However, RL fine-tuning requires abundant high-quality verifiable data, often sourced from human annotations, generated from frontier LLMs, or scored by LLM-based verifiers. All three have considerable limitations: human-annotated datasets are small and expensive to curate, LLM-generated data is hallucination-prone and costly, and LLM-based verifiers are inaccurate and slow. In this work, we investigate a cheaper alternative: RL fine-tuning on rule-generated synthetic data for multi-hop reasoning tasks. We discover that LLMs fine-tuned on synthetic data perform significantly better on popular real-world question-answering benchmarks, despite the synthetic data containing only fictional knowledge. On stratifying performance by question difficulty, we find that synthetic data teaches LLMs to compose knowledge -- a fundamental and generalizable reasoning skill. Our work highlights rule-generated synthetic reasoning data as a free and scalable resource to improve LLM reasoning capabilities.
Anmol Kabra, Yilun Yin, Albert Gong +6
Cornell University · University of Cambridge · Stanford University