Mine, Sample, Render: Test-Time Diversity Alignment Of Generative Models

Kostas Triaridis; Morris E. Sharp; Nebojsa Jojic

cs.AIOct 15, 2026

Mine, Sample, Render: Test-Time Diversity Alignment Of Generative Models

Authors: Kostas Triaridis, Morris E. Sharp, Nebojsa Jojic

Organizations: Stony Brook University · Microsoft

Abstract

Modern generative models produce high-quality individual outputs but struggle to sample diversely, even when explicitly instructed to do so. We introduce MINE-SAMPLE-RENDER (MSR), a test-time approach that externalizes the diversity into an explicit, corpus-derived sampling procedure. Our pipeline (i) mines an interpretable categorical schema from a corpus of real artifacts and supplies statistics for sampling; (ii) samples attribute configurations from the corpus-derived statistics; (iii) uses a generative model purely as a conditional renderer of a sampled configuration. Across weather report generation, user simulation, webpage design, text-to-image generation, and agent-generated 3D assets, we evaluate the diversity and output quality of MSR-guided generation and show that MSR improves alignment with corpus distributions and diversity of generated outputs.

Figures & tables

Figure 1: Different generations, familiar choices. Five baseline outputs per task . Top: GPT-5.5 webpages generated from the same simple prompt and a given website content. Middle: GPT Image 2 outputs for “A cat is curled up next to a light teddy bear.” Bottom: Independent GPT-6 Astra agent runs asked to build a functional 3D folding chair. See Figures 4 , 5 , and 6 for comparison with MSR, with protocols and results in Sections 3.3 , 3.4 , and 3.5 .

Figure 2: Overview of Mine–Sample–Render. (Top) We use LLMs to mine attributes to create a schema

\mathcal{A}(\mathcal{D})

by comparing small cohorts of artifacts. We then label our source data with the schema to create an empirical distribution of attributes. (Bottom) At inference time, we sample attribute configurations and a model uses them, together with the prompt, to generate diverse outputs.

Figure 3: (Left) LLMs generate weather reports that collapse to a specific mode and do not produce samples that follow conditions that are common but do not belong to the dominant mode , even when given the source dataset in the context (Right) . Generated-text distributional gap against the held-out real distribution of weather conditions.

User simulator	$\mathrm{KL}_{\mathrm{fwd}}\downarrow$	$\mathrm{KL}_{\mathrm{bwd}}\downarrow$	JS $\downarrow$	Coverage $\uparrow$
GPT-5-mini (prompt only)	0.5668	0.5764	0.5866	0.4646
$+$ PersonaHub (coding-filtered)	0.5971	0.6033	0.6157	0.4013
$+$ MSR (compact)	0.5236	0.5556	0.5319	0.5649
$+$ MSR (fine-grained)	0.5184	0.5339	0.5373	0.5106
GPT-5.5 (prompt only)	0.5258	0.5272	0.5167	0.6345
$+$ PersonaHub (coding-filtered)	0.5395	0.5505	0.5371	0.5850

Table 1: Distributional gap between real and simulated user behaviors in coding, following the evaluation protocol of Mehri et al. (2026) : an annotator describes each real and simulated conversation along behavioral facets. These descriptions are embedded and clustered, and the resulting real and simulated cluster distributions are compared using Forward and Backward KL and Jensen–Shannon divergence. Coverage is calculated as the percentage of real clusters with at least one simulated user.

Figure 4: Webpages generated by GPT-5.5 with a fixed prompt ( top ) and additionally MSR ( bottom ).

Webpage generator	Vendi Score		Odd-one-out
Webpage generator	text $\uparrow$	image $\uparrow$	% removed $\uparrow$	mean removal rank $\downarrow$
GPT-5.5 (prompt only)	2.158	2.446	-	-
$+$ MSR (WebSight)	3.956	4.116	95.075%	6.20 vs 19.30
$+$ MSR (Design2Code)	3.273	3.457	97.650%	5.77 vs 19.73
Kimi K2.6 (prompt only)	2.426	3.150	-	-
$+$ MSR (WebSight)	3.842	4.077	75.575%	9.16 vs 16.33

Table 2: Diversity evaluation for generated webpages.

Figure 5: Images generated by GPT Image 2 using the prompt “A cat is curled up next to a light teddy bear.” (Row 1) Original prompt only. (Row 2) Independent rewrite with GPT-5.6 Sol. (Row 3) MSR –Resolved, using GPT-5.6 Sol for semantic resolution.

Generator / conditioning	Vendi Score		Odd-one-out
Generator / conditioning	CLIP $\uparrow$	DINOv2 $\uparrow$	% removed $\uparrow$	mean removal rank $\downarrow$
GPT Image 2 (prompt only)	2.37	2.16	13.80%	37.97
$+$ independent rewrite	3.02	3.19	88.80%	25.26
$+$ coordinated rewrite	3.86	4.32	97.40%	16.45
$+$ MSR –Direct	5.14	4.97	-	-
$+$ MSR –Resolved	6.68	6.73	100.00%	6.82

Table 3: Image diversity across three generators. Vendi scores are averaged over 50 ten-image sets per arm. Odd-one-out compares the three baselines with MSR –Resolved, removing 30 of 40 images per request; survivors receive rank 40. Dashes indicate that Direct was not included in this evaluation.

Figure 6: 3D folding chairs generated with GPT-6 Astra.

Method	Mean odd-one-out removal rank $\downarrow$	Quality $\uparrow$
GPT-6 Astra independent generations	48.800	82.35
GPT-6 Astra coordinated generations	25.125	70.80
GPT-6 Astra $+$ MSR	17.100	83.55

Table 4: Evaluation for 3D folding chair generation.

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Axis	Data-generating attribute	Recovered attribute	Source support
Temperature	temp_cold	temperature_cold	64
	temp_mild	temperature_mild	97
	temp_warm	temperature_warm	99
	temp_hot	temperature_hot	40
Sky	sky_clear	sky_clear	74
	sky_partly	sky_partly_cloudy	84

Appendix

Table 5: Data-generating and recovered weather schemas. Names are the exact retained identifiers; the correspondence is a semantic alignment, not an exact string match. Source support counts the intended configurations of the 300 rendered source reports, not labels extracted from their text. A dash means no distinct corresponding attribute was recovered. The ten city indicators are supplied metadata and are not included in these 16 weather rows.

Condition	Generated	Complete labels	Excluded
Prompt only	3,000	2,838	162
Append reports	3,000	3,000	0
Mine–Sample–Render recovered schema	3,000	3,000	0
Known-schema control	3,000	3,000	0

Appendix

Table 6: Weather reports generated and retained for distributional evaluation. All conditions generate 300 reports per city. Prompt-only exclusions are incomplete four-axis annotations after retries, not ungenerated reports; its retained counts range from 244 to 300 per city.

Schema	Attribute	Count	Frequency
Compact	tone_no_greeting_direct	539	89.83%
Compact	expectation_of_code_full_snippet	415	69.17%
Compact	request_imperative_command	366	61.00%
Compact	no_or_small_code_sample	283	47.17%
Compact	context_provides_short_example_or_none	274	45.67%
Fine-grained	request_imperative_command	409	68.17%

Appendix

Table 7: The five most frequent non-absent attributes per user schema after back-applying the frozen vocabulary to all 600 training conversations. Identifiers are retained verbatim.

Schema	Attribute A	Attribute B	Joint count	Lift
Compact	expectation_of_code_full_snippet	tone_no_greeting_direct	376 (62.67%)	1.009
Compact	request_imperative_command	tone_no_greeting_direct	343 (57.17%)	1.043
Compact	expectation_of_code_full_snippet	request_imperative_command	297 (49.50%)	1.173
Fine-grained	request_imperative_command	wants_full_runnable_code_or_complete_artifact	258 (43.00%)	1.198
Fine-grained	evidence_pastes_existing_code_snippet	provides_code_or_artifact_example	182 (30.33%)	1.841
Fine-grained	provides_code_or_artifact_example	request_imperative_command	182 (30.33%)	0.927

Appendix

Table 8: The three most frequent unordered cross-axis attribute pairs per user schema. Joint frequency is the fraction of the 600 source conversations containing both attributes. Lift divides joint frequency by the product of the two marginal frequencies.

User simulator	Inference control	Effective $N$
GPT-5-mini	Low reasoning	4,960
GPT-5.5	Low reasoning	4,903
GPT-4o	No explicit reasoning control	4,946
Kimi K2.6	No explicit reasoning control	4,935
Qwen3.8-27B	Thinking enabled; private state removed	4,956

Appendix

Table 9: User simulators and final matched evaluation sizes.

Schema	Attribute	Count	Frequency
WebSight	type_grotesque_sans	8,134	81.34%
WebSight	color_light_neutral_background	6,653	66.53%
WebSight	header_present	6,192	61.92%
WebSight	content_left_aligned	5,811	58.11%
WebSight	top_h1_heading_present	4,314	43.14%
Design2Code	site_header_present	418	87.08%

Appendix

Table 10: The five most frequent attributes per schema in the empirical source-page matrices. Identifiers are retained verbatim. Counts use 10,000 WebSight pages and 480 Design2Code pages.

Schema	Attribute A	Attribute B	Joint count	Joint frequency	Lift
WebSight	color_light_neutral_background	type_grotesque_sans	5,462	54.62%	1.009
WebSight	header_present	type_grotesque_sans	5,148	51.48%	1.022
WebSight	content_left_aligned	type_grotesque_sans	4,821	48.21%	1.020
Design2Code	site_footer_present	site_header_present	372	77.50%	1.065
Design2Code	primary_navigation_present	site_header_present	337	70.21%	1.118
Design2Code	body_typography_sans_serif	site_header_present	325	67.71%	0.998

Appendix

Table 11: The three most frequent unordered co-occurring attribute pairs per schema. Joint frequency is the fraction of source pages containing both attributes. Lift divides joint frequency by the product of the two marginal frequencies; values near one indicate little departure from marginal independence.

Schema	Attribute pair	Joint count	Joint frequency	Lift
WebSight	buttons_full_width , stacked_form_inputs	104	1.04%	19.10
WebSight	header_fixed , nav_fixed_top	103	1.03%	11.63
WebSight	layout_sidebar_offset , nav_side_vertical	161	1.61%	11.41
Design2Code	article_caption_centered , article_image_captions_centered	5	1.04%	96.00
Design2Code	body_copy_monospace , body_typography_monospace	5	1.04%	96.00
Design2Code	coming_soon_content_upper_middle , launch_status_uppercase	5	1.04%	96.00

Appendix

Table 12: Highest-lift attribute pairs with joint support of at least 1%. The three rows per schema are ranked by lift, not frequency. Several Design2Code pairs tie at lift 96; alphabetical tie-breaking determines the displayed subset.

#	Category	Prompt ID	Exact prompt
1	baseball bat	baseball_bat_403675	A guy holding a baseball bat is ready to play ball
2	cow	cow_537695	A cow has a ring in its nose near two men.
3	wine glass	wine_glass_285773	A table with a wine glass and wine bottles on it
4	dining table	dining_table_106338	a big table that has some plates of food on it
5	spoon	spoon_306638	A bowl of food set on a table with a spoon.
6	scissors	scissors_223362	a close up of a person holding a pair of scissors

Appendix

Table 19

Generator	Direct	Resolved	Direct	Resolved
	Valid images (%)		Attributes realized (%)
GPT Image 2	99	97	61.6	80.2
MAI-Image-2.5	96	97	65.3	78.5
FLUX.2-flex	95	90	71.1	77.9

Appendix

Table 14: Model-based audits of 100 images per MSR variant and generator. Each attribute-realization rate uses 2,000 originally sampled attribute occurrences.

$\lambda$	CLIP Vendi	DINOv2 Vendi
0.0	4.23	4.40
0.5	5.14	4.97
1.0	5.34	5.21

Appendix

Table 21

Representation	$\lambda=.5$ Resolved	$\lambda=1$ Resolved
CLIP	6.68	6.62
DINOv2	6.73	6.42

Appendix

Table 22

Source	Retained records
Amazon Berkeley Objects	65
Wikimedia Commons	83
Art Institute of Chicago	1
Cleveland Museum of Art	1
Metropolitan Museum of Art	10

Appendix

Table 23

Property A	Property B	P(B $\mid$ A)	P(B $\mid$ not A)	Phi
Feet: metal	Frame: tubing	95.3%	6.3%	0.890
Back: sling	Back: visible textile weave	95.8%	8.5%	0.876
Back: light	Seat: light	91.3%	3.8%	0.879
Back: edge binding	Seat: edge binding	95.7%	11.8%	0.841
Back: geometric	Seat: geometric	71.4%	2.2%	0.750

Appendix

Table 24

Post-hoc topic description	Two enriched properties: topic probability / corpus prevalence
Textile camping frames	Hardware: connector hubs: 95.2% / 24.9%; Arms: matte: 98.2% / 35.8%
Timber slats and panels	Back: slats: 97.6% / 17.3%; Seat: wood: 96.8% / 18.6%
Carved and weathered timber	Arms: brown: 97.5% / 15.0%; Feet: distressed: 93.0% / 12.0%
Low striped slings	Continuous seat back: 96.6% / 34.9%; Low seat: 97.5% / 36.3%
Laced lounge seating	Head cushion: 97.7% / 13.0%; Back: edge laced: 98.0% / 13.5%
Solid-panel and molded seating	Back: solid panel: 94.7% / 21.2%; Seat: molded shell: 65.6% / 8.5%

Appendix

Table 25

Figure 7: (Top) Example of a report generated by GPT-6 Astra for a generated folding chair. (Bottom) Illustration of the folding mechanism, originally provided as a video.

Figure 8: From sampled attributes to an articulated chair: open, intermediate, and folded states of the same generated Blender model. The programmed sequence includes a lift and tilt during folding; the folded pose is held rather than claimed to stand unsupported. These renders demonstrate modeled folding motion, not physically verified performance.

Quality criterion	Weight	MSR	Coordinated
Coherent deployed chair and support	25%	4.40	3.70
Resolved, meaningful folding transformation	30%	4.40	3.85
Human use, handling, and folded-state usefulness	15%	3.90	3.35
Integrated form and detailed resolution	20%	4.20	3.35
Compelling and purposeful design idea	10%	3.90	3.75
Weighted total / 100	100%	84.70	72.55

Appendix

Table 15: Evaluation of chair quality. Each criterion is scored out of 5 and assigned a weight by an investigator agent, different from the final judge.

Figure 9: Webpages generated by GPT-5.5 using the prompt “Create a training hub page using the grounding file for the manufacturing onboarding program. Use orange and yellow as the main colors – I want it to feel energetic and welcoming. Include all content and images from the grounding doc.” (Left) Original prompt only. (Middle) Mine–Sample–Render using the Websight schema (Right) Mine–Sample–Render using the Design2Code schema.

Figure 10: Webpages generated by Kimi K2.6 using the prompt “Create a recap page for the Axium annual hackathon using the grounding doc, green color scheme.” (Left) Original prompt only. (Middle) Mine–Sample–Render using the Websight schema (Right) Mine–Sample–Render using the Design2Code schema.

Figure 11: Images generated by GPT Image 2 using the prompt “A closeup on some chocolate cake has a fork by it.” (Row 1) Original prompt only. (Row 2) Independent rewrite with GPT-5.6 Sol. (Row 3) Coordinated rewrite with GPT-5.6 Sol. (Row 4) MSR –Resolved.

Figure 12: Images generated by GPT Image 2 using the prompt “A table with a wine glass and wine bottles on it.” (Row 1) Original prompt only. (Row 2) Independent rewrite with GPT-5.6 Sol. (Row 3) Coordinated rewrite with GPT-5.6 Sol. (Row 4) MSR –Resolved.

Figure 13: Images generated by GPT Image 2 using the prompt “Apples and oranges sitting next to each other on a shelf.” (Row 1) Original prompt only. (Row 2) Coordinated rewrite with GPT-5.6 Sol. (Row 3) Independent rewrite with GPT-5.6 Sol. (Row 4) MSR –Resolved.

Figure 14: Images generated by GPT Image 2 using the prompt “A dragon swinging a bat with a dog watching.” (Row 1) Original prompt. (Row 2) Mine–Sample–Render .

Generative Modeling

Generative modeling learns data distributions to create new samples, from images and text to molecules, simulations, and structured records.

See all 487 papers

Where You Inject Diversity Matters: A Unified Framework for Diverse Generation
Jun 9, 2026Cheng Zhang, Rui Xin, Chudi ZhongOpen-Ended Generation Controllable Text Generation
UNC Chapel Hill · University of Washington
Exposing Diversity Bias in Deep Generative Models: Statistical Origins and Correction of Diversity Error
Feb 16, 2026Farzan Farnia, Mohammad Jalali, Azim OspanovGenerative Modeling Generative AI Evaluation
Department of Computer Science and Engineering, The Chinese University of Hong Kong.
A Universal Avoidance Method for Diverse Multi-branch Generation
Apr 19, 2026Kyeongman Park, Minha Jhang, Kyomin JungDiverse Text Generation
Seoul National University

Jun 9, 2026J/K move · Enter open · S save

Where You Inject Diversity Matters: A Unified Framework for Diverse Generation

Cheng Zhang, Rui Xin, Chudi Zhong

UNC Chapel Hill · University of Washington

Open-Ended Generation Controllable Text Generation Diverse Text Generation Output Diversity in Language Models

Open paper AskPDF arXiv

Open-ended generation tasks often require a set of meaningfully different outputs, yet large language models often produce similar generations. Existing test-time diversity methods operate at different stages of generation with varying effectiveness, but it remains unclear what design choices lead to meaningful diversity in the output. We introduce a framework that characterizes test-time diverse generation methods by the diversity source introduced during generation and provide a transmission score for measuring how effectively variation in the source reaches the final output. Guided by this framework, we propose fully automated specification-level generation methods that first generate diverse intermediate specifications and then condition on them to produce final responses. Across five open-ended tasks and four backbone models, specification-level injection improves output diversity over test-time baselines while maintaining comparable quality. Our analysis shows that successful diversity injection depends on both the diversity of the sources and their transmission to the output, highlighting source design and source-to-output realization as two key levers for building more diverse generation systems.

Abstract

Generative Modeling

Explore similar work

Where You Inject Diversity Matters: A Unified Framework for Diverse Generation

Exposing Diversity Bias in Deep Generative Models: Statistical Origins and Correction of Diversity Error

A Universal Avoidance Method for Diverse Multi-branch Generation