Beyond Mode Collapse: Generating Diverse Synthetic Expert Conversations via Generative Flow Networks

Sumit Asthana; Michael Ion; Kevyn Collins Thompson

cs.AISep 29, 2026

Beyond Mode Collapse: Generating Diverse Synthetic Expert Conversations via Generative Flow Networks

Authors: Sumit Asthana, Michael Ion, Kevyn Collins Thompson

Organizations: University of Michigan Ann Arbor, MI, USA

Abstract

High quality synthetic data is central to post training LLMs for adaptive AI applications that represent the diverse expert strategies and decisions in conversations. Prompting LLMs directly or conditioning them on end use scenarios yields low diversity data that collapses onto dominant modes. We propose a method to generate diverse high quality synthetic data using Generative Flow Networks (GFlowNets). We show that training GFlowNets to generate latent conversation structure using a Gaussian mixture density over key interaction features (e.g., confusion episode dynamics, scaffolding directive balance) enables sampling expert strategies in proportion to their prevalence in the training data. Across two structurally distinct domains, tutoring and emotional support dialogues, our GFlow based synthetic data generation approach offers a better balance of fidelity, mode coverage and authenticity than reinforcement-learning and end to end LLM baselines, without copying training data. Evaluated on three downstream outcome prediction tasks, classifiers trained on synthetic GFlowNet generated conversations provide a stronger training signal than competitive synthesis baselines.

Figures & tables

	MathMentorDB	ESConv
Domain	1:1 math tutoring	emotional-support dialogue
Roles	tutor, student	supporter, seeker
Move vocab.	24	9
Train convs.	2,500	1,040
Held-out convs.	1,000	260
Avg. turns/conv.	23.8	29.5

Table 1: Left: summary statistics for the two datasets. Right: a real excerpt from MathMentorDB’s training split showing the tutor and student exchange, and the annotated move labels.

GPT-5.6-terra realization
	Math tutoring			ESConv
Condition	$\mathrm{JS}_{\mathrm{trans}}$ $\downarrow$	pMSE $\downarrow$ (all features)	pMSE $\downarrow$ (excl. surface)	$\mathrm{JS}_{\mathrm{trans}}$ $\downarrow$	pMSE $\downarrow$ (all features)	pMSE $\downarrow$ (excl. surface)
Real conversations (null)	0.063	2.0	1.7	0.003	4.6	3.2
LLM only ( B2 , best tier)	0.367	97.3	94.3	0.067	98.4	95.6
RL + LLM (REINFORCE) ‡	0.267	95.3	94.8	0.108	94.2	80.8
GFlow + LLM (ours)	0.145	81.1	65.1	0.015	97.4	58.1

Table 2: GFlow achieved the best synthetic data fidelity across both datasets and LLM choices, as measured by state-transition fidelity (

\mathrm{JS}_{\mathrm{trans}}

) and pMSE across the three data generation conditions. A structural upper-bound is provided for reference (see § 4.4 for sample sizes, redraws, and protocol). pMSE is reported on all features of

\phi(x)

and with the surface feature group excluded, since surface features depend on the LLM realizer rather than the planned structure. Lower is better on all metrics.

Panel of Figure 1 — Figure 1: A breakdown of pMSE results by feature dimension (§ 4.2 ) shows that GFlow+LLM superior results were achieved by consistent reduction of pMSE across all non-surface feature dimensions, which capture underlying conversation strategies. Surface dimension pMSE was close across all three synthesis approaches. gemini-2.5-flash realization, $n{=}1500$ , per domain.

	pMSE $\downarrow$	$\mathrm{IP}_{\alpha}$ $\uparrow$	$\mathrm{IR}_{\beta}$ $\uparrow$	Auth. $\uparrow$	Outlier
Real held-out (reference)	9.4 $\pm$ 4.5	0.94 $\pm$ 0.03	0.96 $\pm$ 0.02	0.70	0.050
LLM only (B2), GPT-5.6-terra	95.0 $\pm$ 3.7	0.44 $\pm$ 0.04	0.09 $\pm$ 0.03	0.96	0.233
REINFORCE, GPT-5.6-terra	94.1 $\pm$ 4.1	0.81 $\pm$ 0.03	0.17 $\pm$ 0.04	0.92	0.006
GFlowNet (TB), GPT-5.6-terra	71.5 $\pm$ 3.3	0.87 $\pm$ 0.04	0.57 $\pm$ 0.07	0.85	0.117
LLM only (B2), Gemini 2.5 Flash	96.0 $\pm$ 2.6	0.21 $\pm$ 0.03	0.00 $\pm$ 0.01	0.99	0.738
REINFORCE, Gemini 2.5 Flash	93.4 $\pm$ 2.5	0.85 $\pm$ 0.02	0.16 $\pm$ 0.04	0.90	0.007

Table 3: Fidelity, diversity, and authenticity on MathMentorDB by synthesis condition. Metric definitions are in § 4.2 and the sampling protocol is in § 4.4 . The top row shows scores for held-out real conversations as if they had been generated, so it is the value a perfect generator reaches at this sample size. This provides an upper bound reference against which to compare the main results. ESConv results are in Appendix Table 7 .

Figure 2: Coverage of the real MathMentorDB distribution for the three studied synthesis approaches, showing how our GFlow + LLM approach (right) is the only one to achieve high fidelity coverage of the entire real support, while also producing novel but realistic outliers. Visualization uses t-SNE of non-surface $\phi(x)$ (based on Gemini, $n{=}1500$ per condition). A real point is covered if there is a synthetic point within the 95th percentile (§ Appendix F ). Real conversations are blue if covered by the condition and red if not; grey hexagons are synthetic outliers. Plots above the panels describe the authenticity, a local density sensitive measure of how close are synthetic samples to real ones. A synthetic point is authentic if $d(x,r)>R(r)$ (above $y=x$ line), and copy-like otherwise.

Label	Real oracle	LLM	RL	GFlow	LLM	RL	GFlow	Majority floor
		GPT-5.6-terra Realization			Gemini 2.5 Flash Realization
resolved	0.79 $\pm$ 0.03	0.50 $\pm$ 0.03	0.29 $\pm$ 0.01	0.65 $\pm$ 0.03	0.57 $\pm$ 0.03	0.29 $\pm$ 0.01	0.64 $\pm$ 0.03	0.37
has_breakthrough	0.92 $\pm$ 0.02	0.71 $\pm$ 0.03	0.41 $\pm$ 0.01	0.72 $\pm$ 0.03	0.69 $\pm$ 0.03	0.41 $\pm$ 0.01	0.81 $\pm$ 0.03	0.41
has_off_topic	0.89 $\pm$ 0.04	0.48 $\pm$ 0.00	0.48 $\pm$ 0.00	0.50 $\pm$ 0.03	0.48 $\pm$ 0.00	0.48 $\pm$ 0.00	0.48 $\pm$ 0.00	0.48

Table 4:

\mathrm{F1}_{\mathrm{macro}}

results for three conversation outcome prediction tasks, using the train-on-synthetic test-on-real (TSTR) in the tutoring domain. GFlow + LLM wins or ties every (label, realizer) pair.

\pm

is the half-width of a 95% bootstrap CI (300 draws over real test conversations).

Panel of Figure 3 — Figure 3: GMM mode distribution on the real held-out set for the conditions LLM-only ( B2 , best tier), RL+LLM, and GFlow+LLM (ours) (§ 4.2 ). Mode labels are our post-hoc interpretation of each cluster’s dominant feature profile. RL+LLM is characterized by a single dominant mode, and GFlow+LLM reproduces more modes than LLM-only.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Dimension	MathMentorDB (tutoring), 47 feat.	ESConv (emotional support), 30 feat.
Surface (6 / 6)	avg_student_len, std_student_len, avg_tutor_len, std_tutor_len, n_messages, turn_ratio	avg_seeker_len, std_seeker_len, avg_supporter_len, std_supporter_len, n_messages, turn_ratio
Content (21 / 8)	direct_rate, scaffolding_rate, correction_rate, confirm_pos_rate, confirm_neg_rate, tutor_question_rate, knowledge_check_rate, attempt_rate, knowledge_gap_rate, understanding_rate, breakthrough_rate, explain_problem_rate, explain_reasoning_rate, student_question_rate, student_confirm_rate, frustration_rate, encouragement_rate, empathy_rapport_rate, confidence_express_rate, acknowledgment_rate, knowledge_recall_rate	question_rate, restatement_rate, reflection_rate, self_disclosure_rate, affirmation_rate, suggestion_rate, information_rate, others_rate
Structure (7 / 4)	pct_tutor_academic, pct_student_academic, pct_non_academic, pct_socio_emotional, greeting_rate, platform_rate, smalltalk_rate	pct_empathic, pct_directive, pct_exploratory, pct_personal
Dynamics (10 / 9)	n_confusion_episodes, avg_confusion_duration, max_confusion_duration, instant_resolution_rate, extended_confusion_rate, scaffolding_density_first_third, scaffolding_density_last_third, scaffolding_density_shift, first_attempt_position, first_understanding_position	empathy_density_first_third, empathy_density_last_third, directive_density_first_third, directive_density_last_third, empathy_to_directive_shift, strategy_entropy, n_strategy_types, suggestion_position, first_empathy_position
Outcome (3 / 3)	resolved, has_breakthrough, has_off_topic	resolved_esc, has_suggestion, has_reflection

Appendix

Table 5: The full feature vector

\phi(x)\in\mathbb{R}^{47}

(tutoring) /

\mathbb{R}^{30}

(ESConv), by dimension (§ 3.2 ), following the taxonomy of Ion and Collins-Thompson (2026) . Per-dimension counts are (tutoring / ESConv). These are the only features used by any evaluation metric or reward in the paper (pMSE, mode coverage, DCR/copying, the GMM-density and Gaussian-

z

-score reward terms); the state trackers that condition the policy during generation (Appendix D ) are a separate, much smaller set and are not part of

\phi(x)

Tier	Combined	Surface	Content	Structure	Dynamics	Outcome
Tutoring B1 (naive)	98.7	75.0	95.9	79.0	71.0	10.9
Tutoring B2 (scenario)	98.4	72.2	90.5	61.3	47.9	5.8
Tutoring B3 (scenario+mode)	98.6	70.3	89.5	68.3	44.8	6.8
ESConv B1 (naive)	95.2	96.1	75.3	69.3	73.6	68.3
ESConv B2 (scenario) †	95.0	87.2	62.4	68.1	70.4	51.8
ESConv B3 (scenario+mode)	96.8	98.1	78.8	69.7	72.5	53.4

Appendix

Table 6: Per-tier, per-feature-group pMSE for the three LLM-baseline tiers (§ 4.3 ),

n{=}400

each, GPT-5.6-terra.

Condition	pMSE $\downarrow$	$\mathrm{IP}_{\alpha}$ $\uparrow$	$\mathrm{IR}_{\beta}$ $\uparrow$	Auth. $\uparrow$	Near-copy	Outlier
Real held-out (reference)	7.0 $\pm$ 1.6	0.96 $\pm$ 0.01	0.96 $\pm$ 0.01	0.67	0.072	0.050
LLM only (B2), GPT-5.6-terra	95.5 $\pm$ 2.3	0.43 $\pm$ 0.03	0.06 $\pm$ 0.02	0.94	0.000	0.123
REINFORCE, GPT-5.6-terra	96.0 $\pm$ 1.6	0.27 $\pm$ 0.02	0.06 $\pm$ 0.02	0.96	0.001	0.282
GFlowNet (TB), GPT-5.6-terra	93.8 $\pm$ 2.7	0.29 $\pm$ 0.02	0.22 $\pm$ 0.03	0.93	0.001	0.410
LLM only (B2), Gemini 2.5 Flash	91.6 $\pm$ 2.0	0.78 $\pm$ 0.04	0.42 $\pm$ 0.04	0.67	0.025	0.013
REINFORCE, Gemini 2.5 Flash	92.7 $\pm$ 2.0	0.52 $\pm$ 0.04	0.12 $\pm$ 0.02	0.75	0.025	0.103

Appendix

Table 7: Sample-level fidelity (

\mathrm{IP}_{\alpha}

), coverage (

\mathrm{IR}_{\beta}

), and authenticity for ESConv (§ G.1 ),

n{=}1500

per condition, both realizers. Each draw scores 200 generated conversations against 200 held-out real conversations on

\phi(x)

, whitened on the training split; pMSE is the mean

\pm

sd over 10 draws and

\mathrm{IP}_{\alpha}

\mathrm{IR}_{\beta}

, authenticity over 50 draws at

k{=}5

, with near-copy and outlier computed over all 1500. The first row scores held-out real conversations against the training split exactly as a synthetic condition is scored, giving the value a perfect generator attains at this sample size; read every other row against it rather than against

1

. pMSE here is

200

vs.

200

and so runs below the

n{=}1500

values of Table 2 , with the same ordering. The two providers used different ESConv realization prompts (§ 3.2 ), so rows compare within a provider; the GFlowNet rows are the gmm_density reward with the inference-time directive mask off, not the guard-on configuration.

Explore similar work

Adversarial Arena: Crowdsourcing Data Generation through Interactive Competition
Apr 20, 2026Prasoon Goyal, Sattvik Sahai, Michael Johnston +14Large Language Model Generation Adversarial Robustness
Amazon Nova Responsible AI
Graph2Counsel: Clinically Grounded Synthetic Counseling Dialogue Generation from Client Psychological Graphs
Apr 22, 2026Aishik Mandal, Hiba Arnaout, Clarissa W. Ong +5Counseling Synthetic Data
UKP Lab, Department of Computer Science and Hessian Center for AI (hessian.AI), Technische Universität Darmstadt · Zuse School · National Research Center for Applied Cybersecurity ATHENE +4
EmbGen: Teaching with Reassembled Corpora
May 19, 2026Arun K Lenin, Kai Rouse, Andrea Nicastro +1Synthetic Data Text Corpora
Commonwealth Bank of Australia Sydney, Australia

Apr 20, 2026, cs.AIJ/K move · Enter open · S save

Adversarial Arena: Crowdsourcing Data Generation through Interactive Competition

Prasoon Goyal, Sattvik Sahai, Michael Johnston, Hangjie Shi+13

Amazon Nova Responsible AI

Large Language Model Generation Adversarial Robustness Conversational Datasets Data Generation LLM Defense Mechanisms Competition

Open paper AskPDF arXiv

Post-training Large Language Models requires diverse, high-quality data which is rare and costly to obtain, especially in low resource domains and for multi-turn conversations. Common solutions are crowdsourcing or synthetic generation, but both often yield low-quality or low-diversity data. We introduce Adversarial Arena for building high quality conversational datasets by framing data generation as an adversarial task: attackers create prompts, and defenders generate responses. This interactive competition between multiple teams naturally produces diverse and complex data. We validated this approach by conducting a competition with 10 academic teams from top US and European universities, each building attacker or defender bots. The competition, focused on safety alignment of LLMs in cybersecurity, generated 19,683 multi-turn conversations. Fine-tuning an open-source model on this dataset produced an 18.47% improvement in secure code generation on CyberSecEval-Instruct and 29.42% improvement on CyberSecEval-MITRE.

Abstract

Explore similar work

Adversarial Arena: Crowdsourcing Data Generation through Interactive Competition

Graph2Counsel: Clinically Grounded Synthetic Counseling Dialogue Generation from Client Psychological Graphs

EmbGen: Teaching with Reassembled Corpora