Shared Nature, Unique Nurture: PRISM for Pluralistic Reasoning via In-context Structure Modeling

Guancheng Tu; Shiyang Zhang; Tianyu Zhang; Yi Zhang; Diji Yang

cs.LGFeb 24, 2026

Shared Nature, Unique Nurture: PRISM for Pluralistic Reasoning via In-context Structure Modeling

Authors: Guancheng Tu, Shiyang Zhang, Tianyu Zhang, Yi Zhang, Diji Yang

Organizations: University of Pennsylvania · Yale University · MILA · Universit´e de Montréal ·

Abstract

Large Language Models (LLMs) are converging towards a singular Artificial Hivemind, where shared Nature (pre-training priors) result in a profound collapse of distributional diversity, limiting the distinct perspectives necessary for creative exploration and scientific discovery. To address this, we propose to equip models with inference-time Nurture (individualized epistemic trajectories) using Epistemic Evolution paradigm, progressing through explore, internalize, and express. We instantiate this via PRISM (Pluralistic Reasoning via In-context Structure Modeling), a model-agnostic system that augments LLM with dynamic On-the-fly Epistemic Graphs. On three creativity benchmarks, PRISM achieves state-of-the-art novelty and significantly expands distributional diversity. Moreover, we evaluate the real-world utility via a challenging rare-disease diagnosis benchmark. Results demonstrate that PRISM successfully uncovers correct long-tail diagnoses that standard LLM miss, confirming that its divergence stems from meaningful exploration rather than incoherent noise. Overall, this work establishes a new paradigm for Pluralistic AI, moving beyond monolithic consensus toward a diverse ecosystem of unique cognitive individuals capable of collective, multi-perspective discovery.

Figures & tables

Figure 1: Divergent cognitive responses to a common stimulus: Newton’s gravity vs. Kolb’s candy apple, both inspired by an apple. Analogous to an optical prism, our PRISM framework aims to restore this pluralism by refracting shared pre-training knowledge into distinct, individualized epistemic trajectories.

Figure 2: The PRISM Pipeline. The system first performs Cognitive Explosion via wild search to break local relevance, followed by Epistemic Structuring where Context Nodes and Spark Nodes are bridged via cognitive operators (Mapping, Blending, Inversion).

Figure 3: PCA visualization of response distributions. PRISM (in dot) produces multi-centered, elongated distributions compared to the concentrated clusters of vanilla generation.

Figure 4: Intra-model diversity heatmap on Artificial Hivemind.

Figure 5: Inter-model similarity heatmap on Artificial Hivemind.

	NoveltyBench	IdeaBench
Qwen3-4B-Instruct (Vanilla)	3.09	0.72
$\hookrightarrow$ PRISM	4.48	0.96
CrPO-sft-LLaMA-3.1	7.35	-
$\hookrightarrow$ PRISM	7.67	-
GPT-4o-mini (Vanilla)	2.65	0.45
$\hookrightarrow$ PRISM	3.41	0.65

Table 1: Result on NoveltyBench (Distinct score) and IdeaBench (Novelty score).

Strategy	Mechanism	Recall@1 ( $\uparrow$ )	Recall@10 ( $\uparrow$ )	Mean Rank ( $\downarrow$ )
Vanilla Model	Parametric Prior (Zero-shot)	16.0%	32.0%	1.50
Search-Only	Flat RAG (Syntactic Seeds)	14.0%	28.0%	1.79
PRISM (Syntactic)	Graph + Keyword Seeds	16.7%	39.2%	2.50
PRISM (Expert)	Graph + Expert Intent	22.0%	52.0%	2.92

Table 2: Diagnostic performance on RareBench (RAMEDIS). While Flat RAG degrades performance due to noise, introducing the PRISM graph structure (Syntactic) improves recall. Furthermore, equipping PRISM with Expert Intent (Semantic Seeds) strengthens the Epistemic Evolution framework, significantly extending the coverage of long-tail diagnoses.

Method	Benchmark	Novelty	Improvement
Search-Only (Flat RAG)	IdeaBench	0.49	–
PRISM (Full Graph)	IdeaBench	0.57	+16.33%
Search-Only (Flat RAG)	NoveltyBench	4.38	–
PRISM (Full Graph)	NoveltyBench	4.42	+0.91%

Table 3: Ablation Study: Contribution of the Epistemic Graph structure vs. Flat RAG.

I. Effect of Model Scale & Paradigm
Analysis Dimension	Configuration	NoveltyBench	IdeaBench
Qwen3-1.7B	Vanilla	4.82	0.39
	+ PRISM	5.17	0.65
Qwen3-4B-Instruct	Vanilla	5.05	0.63
	+ PRISM	5.48	0.93
Qwen3-4B-Thinking	Vanilla	3.14	0.70

Table 4: System Characterization and Design Implications. We analyze the synergistic impact of Model Scale , Seed Count , and Lexical Seed Source on creativity and discovery metrics.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Distinct	Utility
Baselines (Reference Data)
Claude-3.5 Haiku	1.94	2.50
Claude-3.5 Sonnet	1.76	2.36
Claude-3 Opus	2.04	2.67
gpt-4o	2.88	3.27
gemini-1.5-pro	1.85	2.73

Appendix

Table 5: Comprehensive NovelBench results. The upper section lists updated reference baselines for major model families. The lower section shows our testbed results, where

\hookrightarrow

denotes the PRISM-enhanced version. All models are evaluated on Distinct (Novelty) and Utility .

Model / Scenario	SemSim $\downarrow$	Overlap $\downarrow$	Novelty $\uparrow$
Main Results (Our Experiments)
Qwen3-4B-Instruct	0.60	6	0.72
$\hookrightarrow$ + PRISM	0.56	5	0.96
gpt-4o-mini	0.61	7	0.45
$\hookrightarrow$ + PRISM	0.59	7	0.65
Baselines (Reference Data)

Appendix

Table 6: Comprehensive results on IdeaBench. Our system (PRISM) significantly boosts the Novelty Insight Score compared to various baselines and model scenarios. All scores are rounded to two decimal places.

Figure 6: PCA visualization of response distributions for Prompt: “Name one meaning of life.” Compared with the base model, PRISM produces a broader and more structured semantic spread.

Figure 7: PCA visualization of response distributions for Prompt: “Write a metaphor about time.” PRISM mitigates mode collapse and encourages diversified expressions.

Figure 8: PCA visualization of response distributions for Prompt: “How can I live on $1,000 per month?” PRISM yields more separated and interpretable semantic clusters across models.

Category	Model Family	Specific Version / Checkpoint
Proprietary	GPT-4o	gpt-4o-mini-2024-07-18
	Claude 4.0	claude-sonnet-4-20250514
	Gemini 3.0	gemini-3-flash-preview
Open-Weight	Qwen3	Qwen3-4B-Instruct-2507
Open-Weight	Llama 3.1	CrPO-sft-LLaMA-3.1-8B-Instruct

Appendix

Table 7: Base Models evaluated in our experiments.

LLM Reasoning with Graphs

Large language model reasoning with graphs uses structured relationships to support multi-step inference, evidence retrieval, and planning.

See all 210 papers

PRISM: Promoting Sampling Diversity of Large Language Models with Implicit Stochastic Modulation
Oct 23, 2026Qixin Xu, Zihan Zhu, Xiaojiang Liu +1Output Diversity in Language Models LLM Training
Tsinghua University. · Apple.
Large Language Models Explore by Latent Distilling
Apr 27, 2026Yuanhao Zeng, Ao Lu, Lufei Li +3Language Model Decoding Test-Time Training
School of Information Science and Technology, ShanghaiTech University, Shanghai, China · State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing, China.
Training Reasoning Models to Generate Diverse Scientific Ideas
Oct 15, 2026Manya Wadhwa, Pranav Narayanan Venkit, Hiroaki Hayashi +3Diverse Text Generation Scientific Hypothesis Generation
New York University · Salesforce AI Research · Stony Brook University

Oct 23, 2026J/K move · Enter open · S save

PRISM: Promoting Sampling Diversity of Large Language Models with Implicit Stochastic Modulation

Qixin Xu, Zihan Zhu, Xiaojiang Liu, Bohan Lyu

Tsinghua University. · Apple.

Output Diversity in Language Models LLM Training Diversity-Aware Sampling Mode Collapse

Open paper AskPDF arXiv

Large language models (LLMs) exhibit mode collapse, where their output distribution concentrates on familiar semantic modes. When diversity disappears, repeated sampling becomes semantic repetition rather than discovery, making rare but stronger solutions unlikely to surface and leaving self-evolution and continual learning difficult to sustain. Most remedies intervene only at inference time, shaping sam- ples without making diversity a model capability. We propose PRISM, a training-time framework that internalizes diversity within the model. PRISM combines (i) submodular curation that pairs each prompt with diverse-yet-correct multi-teacher solutions, and (ii) stochastic embedding modulation, which turns special-token perturbations into implicit signals for sampling distinct outputs. A theoretical analysis establishes the stability of this mechanism. On WritingPrompts and BookMIA, PRISM moves the quality-diversity frontier outward, achieving the best generation quality in all model-benchmark settings while maintaining the strongest or near-strongest semantic diversity across two open-weight model families. It also improves sampling efficiency on math and code benchmarks. Beyond inference, PRISM yields a stronger foundation for reinforcement learning, sustaining diverse rollouts and mitigating the diversity collapse commonly observed during RL training.

Abstract

LLM Reasoning with Graphs

Explore similar work

PRISM: Promoting Sampling Diversity of Large Language Models with Implicit Stochastic Modulation

Large Language Models Explore by Latent Distilling

Training Reasoning Models to Generate Diverse Scientific Ideas