Many of the most consequential advances promised by AI, from curing diseases and inventing new technologies to formulating new scientific theories, are fundamentally creative acts. Yet creativity is uniquely difficult to learn because it faces a creative paradox: learning to repeat past examples of creative success makes those specific examples more likely to recur, inhibiting the very creativity desired. This work argues that creativity is better modeled as an explicit process rather than as a property of individual artifacts, and proposes to learn the underlying generative principles behind the creative process. In particular, to escape the paradox, we introduce creative priming (CP), wherein a trained priming model generates a prompt-specific primer that assists a response model in being creative, explicitly decoupling the creative process from response generation. The priming model learns through GRPO, rewarding primers that lead to responses that are novel, high-quality, and diverse, thereby reinforcing the process that yields creative responses without reinforcing any specific response.
Figures & tables
Figure 1: The creative priming training procedure. Diversity training (top right; Section 2.2.2 ) optimizes the response model π directly, sampling responses from the prompt X and rewarding each one for being simultaneously high-quality and diverse ( ri=Qi⋅Di ). Creative priming (bottom; Section 3 ) instead trains a separate priming model πCP over a frozen response model πresp : πCP produces a primer YCP that is combined with X and passed to πresp to generate a response. The reward flows back into πCP alone through two paths: response-level creativity signals ( C⋅Dintra⋅Dinter ) from the responses and primer-level regularizers ( JCP⋅DintraCP⋅DinterCP ) from the primer. The priming process decouples learning the creative process from reinforcing any particular responses, avoiding the creative paradox.
Figure 2: Conceptual comparison of diversity (Eq. 2 ) and deviation (Eq. 6 ). The reference model responses cluster tightly while those of the diversity-enhanced model spread about the same center (increasing diversity), whereas the creative priming responses also deviate from the reference, achieving diversity and deviation.
Atomic metrics
Compound metrics
Method
Response Diversity
Quality
Deviation
Vendi Score
Distinct Clusters
Filtered
AUC-dev
AUC-qual
VUS
Vendi
Distinct
Vendi
Dist.
Vendi
Dist.
Vendi
Dist.
Qwen3-8b (Reference)
12.3 ±1.6
90.5 ±1.4
-
2.12 ±0.19
18.1 ±3.4
-
-
-
-
-
-
-
-
CP (Untrained)
20.6 ±1.8
91.7 ±1.1
13.1 ±1.3
3.10 ±0.28
20.0 ±3.1
1.48 ±0.35
4.53 ±1.67
0.60
2.31
182.6
602.6
69.3
294.9
DGRPO
26.0 ±1.8
87.5 ±1.4
14.1 ±1.2
4.36 ±0.38
37.1 ±3.6
2.18 ±0.43
6.02 ±1.61
0.819
3.10
297.5
1010.7
107.5
509.5
DARLING
11.7 ±1.1
93.6 ±1.2
10.1 ±0.8
2.17 ±0.16
20.5 ±3.3
0.57 ±0.23
1.76 ±1.06
0.345
2.23
55.8
172.3
34.1
221.0
Table 1: Evaluation on NoveltyBench . Metrics are defined in Section 4 . Row blocks correspond to untrained methods (top rows), training without deviation (middle), and training with deviation (bottom). Cells include 95% confidence intervals where available, and the best value per column within each block is bolded. Deviation-derived metrics for the Qwen3-8B reference are 0 by construction and shown as “-”. This table shows that for the atomic metrics, every trained method trades one property for another (DGRPO gains diversity but loses quality; DARLING does the opposite). However, the atomic metrics do not measure creativity; the compound metrics do, by filtering for responses that are simultaneously deviating and high-quality and then measuring their spread, and CP dominates on compound measures of system creativity. Furthermore, CP without a deviation reward already matches or beats baselines trained with deviation, demonstrating the importance of the priming process itself in creativity.
Figure 3: Qualitative examples. Representative responses to an open-ended prompt (more cherry- and lemon-picked examples in Appendix G ). To get a qualitative understanding of creativity, each label reports the response’s judge quality score and its deviation from the reference model (as described in Section 4 ). In this example, the reference model paints an evocative scene in flowing prose; DGRPO answers in one terse line; DARLING returns a highly-formatted, bulleted analysis; and CP answers in the reference model’s style while inventing a novel, concrete image, illustrating that its deviation lies in what it says rather than in surface formatting. These characteristics are consistent across all prompts (DGRPO’s terseness and DARLING’s length and use of formatting), as analyzed in Appendix F .
Figure 4: Scaling group size does not escape the creative paradox. DGRPO is trained with G∈{8,16,32,64} on-policy samples per prompt and evaluated on NoveltyBench. (a) Each of the five atomic metrics is shown as a percent change from DGRPO with G=8 , with deviation highlighted in copper. (b) Four compound metrics of creativity are shown as a percentage of CP performance, where line color represents the measure of spread (Vendi score or distinct clusters) and marker shape represents the filtering and aggregation method (Filtered or VUS). These results demonstrate that additional on-policy samples consistently improve deviation, but all other atomic metrics ultimately decline at G=64 . System-level measures of creativity (compound metrics) peak at G=32 , but even then fall below the performance of CP , and decline significantly at G=64 . Full results are in Table 7 in Appendix H .
Figure 5: Creative priming transfers to API response models . Each panel reports one evaluation metric for four API models, comparing direct generation (copper) with the addition of the creative priming model (blue; trained with deviation). Error bars give 95% confidence intervals on the total. Results are reported in full in Table 4 in Appendix C . This figure shows that the same CP model trained to improve creativity on a Qwen3-8B response model leads to significantly increased diversity and distinct clusters when using an API response model, while maintaining or improving quality.
Figure 6: Model combinations for the creative priming pipeline . Each cell reports filtered Vendi score (Qwen3-8B direct has 0 deviation from itself). The farthest left column shows scores when using the model listed on the y-axis as the response model (i.e. without CP ). The remaining columns represent scores when using the model listed along the x-axis as the CP model and the model listed along the y-axis as the response model. These results show that, compared to direct response generation, the CP pipeline itself improves creativity, even when using CP models not explicitly trained for priming (i.e. DGRPO and DARLING).
Table 3: Penalty-based vs. criterion-mean creative primer judge. Ablation on NoveltyBench swapping only the judge template ( JCPpenalty , Eq. 19 , vs. the criterion-mean JCP , Eq. 11 ) while holding data, hyperparameters, and reward structure fixed. All metrics and their definitions match Table 1 (diversity and deviation scaled [0,1]→[0,100] ; 95% confidence intervals where available). The two blocks correspond to training without and with a deviation reward. The criterion-mean rows reproduce the CP rows of Table 1 . The better value of each pair is bolded within each block. Without a deviation reward the criterion-mean judge dominates every creativity metric; with it, the two rubrics are within confidence intervals.
Figure 8: Illustrative example of how the area under the filtered Vendi score deviation curve (AUC-dev Filtered Vendi Score) is calculated . For this example, the quality threshold is fixed at the mean quality of the direct-generation reference. Then, the deviation threshold is swept from 0 to 100 in increments of 1 and filtered vendi is calculated at each threshold. Then, the trapezoid rule is used to calculate the AUC-dev Filtered Vendi Score , represented by the shaded area under each curve, and reported in Table 1 .
Atomic metrics
Compound metrics
Response Model
Method
Response Diversity
Judge Score
Deviation
Vendi Score
Distinct Clusters
Filtered
AUC-dev
AUC-qual
VUS
Vendi
Distinct
Vendi
Dist.
Vendi
Dist.
Vendi
Dist.
Sonnet-4.6
Direct
5.3 ±0.9
90.2 ±1.1
-
1.40 ±0.07
2.8 ±0.7
-
-
-
-
-
-
-
-
CP (no div.)
20.7 ±1.8
93.3 ±0.9
15.1 ±1.5
3.22 ±0.27
22.7 ±2.6
2.80 ±0.38
10.1 ±2.08
1.006
4.28
280.6
1053.7
100.4
447.9
CP (+ div.)
24.6 ±1.7
94.3 ±0.8
20.0 ±1.8
4.04 ±0.32
28.5 ±3.1
3.72 ±0.39
16.8 ±2.82
1.338
6.52
375.2
1701.4
133.6
664.8
Opus-4.8
Direct
6.5 ±0.9
91.7 ±1.1
-
1.52 ±0.08
2.8 ±0.5
-
-
-
-
-
-
-
-
Appendix
Table 4: Full API-model transfer results. A single creative priming model, trained once, is paired with four frozen API response models; each is evaluated against its own direct-generation reference. All metrics match Table 1 and are defined in Section 4 : five atomic metrics and eight compound metrics. CP (no div.) and CP (+ div.) are the primer trained without and with a deviation reward. The filtered metrics keep responses passing a per-prompt quality threshold (the direct-generation reference mean) and a deviation threshold of 20 ; Deviation is reported as its per-prompt mean. Because each API model is its own reference, its Direct row has zero deviation by construction, so all deviation-derived columns show “-”.
Atomic metrics
Compound metrics
Response Model
Method
Response Diversity
Judge Score
Deviation
Vendi Score
Distinct Clusters
Filtered
AUC-dev
AUC-qual
VUS
Vendi
Distinct
Vendi
Dist.
Vendi
Dist.
Vendi
Dist.
Sonnet-4.6
Direct
5.3 ±0.9
90.2 ±1.1
-
1.40 ±0.07
2.8 ±0.7
-
-
-
-
-
-
-
-
Verbalized Sampling
31.0 ±1.7
90.6 ±1.1
26.1 ±2.0
4.66 ±0.30
23.9 ±2.7
3.05 ±0.44
10.1 ±2.24
1.119
3.86
371.0
1408.7
137.3
540.1
CP (+ div.)
24.6 ±1.7
94.3 ±0.8
20.0 ±1.8
4.04 ±0.32
28.5 ±3.1
3.72 ±0.39
16.8 ±2.82
1.338
6.52
375.2
1701.4
133.6
664.8
CP (+ div.) + VS
34.3 ±1.3
92.3 ±1.3
30.4 ±1.6
6.53 ±0.37
44.5 ±3.5
5.66 ±0.42
32.3 ±3.68
2.181
11.47
597.9
3522.9
235.3
1268.6
Appendix
Table 5: Composing creative priming with verbalized sampling. The same setup, references, and metrics as Table 4 , restricted to the deviation-trained primer and adding verbalized sampling (VS) and its composition with the primer. CP (+ div.) + VS has the primer emit 8 creative primers per prompt, each sent to the API model in a single verbalized-sampling call returning 8 responses, matching the 64 completions per prompt of every other row. Response diversity and deviation are shown ×100 ; cells include a 95% confidence interval where available. The filtered metrics keep responses passing a per-prompt quality threshold (the direct-generation reference mean) and a deviation threshold of 20 . Because each API model is its own reference, its Direct row has zero deviation by construction, so all deviation-derived columns show “-”. The best value per column within each block is bolded. The composition exceeds both of its parts on every compound metric for all four response models.
Feature
Reference
DGRPO
DARLING
CP
Length
Response length (tokens)
260.8 ±52.8
22.7 ±5.4
410.0 ±49.3
140.6 ±15.9
Sentence length (words)
12.88 ±0.99
5.37 ±0.71
11.42 ±0.56
20.87 ±1.25
Formatting
Uses header (frac.)
0.247 ±0.072
0.001 ±0.001
0.464 ±0.086
0.004 ±0.005
Uses bold (frac.)
0.813 ±0.063
0.005 ±0.004
0.882 ±0.055
0.399 ±0.056
Appendix
Table 6: Impact of training on response style, evaluated on NoveltyBench. Each cell is the feature’s per-prompt mean, averaged over prompts, with a 95% confidence interval; the Reference column is the direct-generation qwen3-8b-parallel model and the three method columns are the with-deviation variants. The function-word Jensen-Shannon divergence is measured against the reference (reference-vs-itself is 0 , shown as “-”) and therefore has no confidence interval.
Figure 9: Representative responses to a single prompt, one per method (the first sampled response of each). The four methods occupy the distinct style regimes quantified in Table 6 : the reference’s name-plus-blurb, DGRPO’s terse one-liner, DARLING’s bolded and bulleted formatting, and CP ’s long-sentence prose naming a different fighter and framing its historical significance.
Figure 10: Example outputs. Highest- and lowest-scoring responses per method for the first prompt in NoveltyBench, scored by individual response creativity: quality × deviation. Each cell shows the judge quality score and the response’s deviation from the reference set. CP cells also show the primer for that response. Responses in the “Least Creative” column are markedly more similar to the reference response, whereas the most creative responses have much more diversity among them.
Figure 11: Highest- and lowest-scoring responses per method for a single prompt, scored by individual response creativity: quality × deviation. Each cell reports the judge quality score and the response’s deviation from the reference model ( qwen3-8b-parallel ). CP cells also show the primer that steered the response. Text is truncated for space.
Figure 12: Highest- and lowest-scoring responses per method for a single prompt, scored by individual response creativity: quality × deviation. Each cell reports the judge quality score and the response’s deviation from the reference model ( qwen3-8b-parallel ). CP cells also show the primer that steered the response. Text is truncated for space.
Figure 13: Highest- and lowest-scoring responses per method for a single prompt, scored by individual response creativity: quality × deviation. Each cell reports the judge quality score and the response’s deviation from the reference model ( qwen3-8b-parallel ). CP cells also show the primer that steered the response. Text is truncated for space.
Atomic metrics
Compound metrics
Method
Response Diversity
Quality
Deviation
Vendi Score
Distinct Clusters
Filtered
AUC-dev
AUC-qual
VUS
Vendi
Distinct
Vendi
Dist.
Vendi
Dist.
Vendi
Dist.
DGRPO ( G=8 )
34.0 ±1.6
77.3 ±2.9
25.6 ±1.6
5.98 ±0.47
33.4 ±3.2
2.85 ±0.46
7.59 ±1.77
1.015
2.78
448.1
1779.4
168.8
658.0
DGRPO ( G=16 )
33.8 ±1.5
74.7 ±3.1
26.9 ±1.5
5.95 ±0.45
33.9 ±3.2
2.72 ±0.47
7.92 ±2.08
0.993
2.90
441.4
1864.9
167.2
672.8
DGRPO ( G=32 )
31.6 ±1.0
81.5 ±2.4
28.0 ±1.1
5.72 ±0.29
35.8 ±2.9
3.91 ±0.43
15.3 ±2.37
1.371
5.12
482.8
2474.7
174.2
834.5
DGRPO ( G=64 )
28.6 ±0.8
69.1 ±3.5
33.5 ±1.0
5.04 ±0.20
24.5 ±2.6
2.95 ±0.35
9.70 ±1.75
1.055
3.24
396.5
1740.1
150.2
586.9
Appendix
Table 7: Evaluation on NoveltyBench with DGRPO group-size scaling. The same setup and metrics as Table 1 , adding DGRPO trained with group sizes G=16 , 32 , and 64 to the default G=8 result. All scaling runs include the deviation term. Cells include 95% confidence intervals where available.
Atomic metrics
Compound metrics
wQ
Response Diversity
Judge Score
Deviation
Vendi Score
Distinct Clusters
Filtered
AUC-dev
AUC-qual
VUS
Vendi
Distinct
Vendi
Dist.
Vendi
Dist.
Vendi
Dist.
1.0
34.0 ±1.6
77.3 ±2.9
25.6 ±1.6
5.98 ±0.47
33.4 ±3.2
2.85 ±0.46
7.59 ±1.77
1.015
2.78
448.1
1779.4
168.8
658.0
1.2
33.1 ±1.6
76.5 ±3.0
25.0 ±1.6
5.68 ±0.47
30.9 ±3.2
2.49 ±0.46
6.65 ±1.85
0.898
2.42
421.1
1642.0
158.5
603.8
1.5
33.0 ±1.6
80.1 ±2.6
24.6 ±1.6
5.70 ±0.45
32.9 ±3.3
2.94 ±0.48
8.50 ±2.02
1.070
3.12
439.5
1782.7
164.0
659.9
2.0
33.0 ±1.7
78.2 ±3.0
24.7 ±1.6
5.66 ±0.48
30.8 ±3.4
2.57 ±0.49
6.61 ±1.77
0.954
2.50
424.2
1635.8
160.9
606.9
Appendix
Table 8: Reweighting quality in the DGRPO reward. NoveltyBench metrics (defined in Section 4 ) as the quality exponent wQ of Eq. 20 increases, holding wD=wDev=1 . Response diversity and deviation are shown ×100 ; cells include a 95% confidence interval where available. The best value per column is bolded. wQ=1.0 is the DGRPO (+Deviation) model from Table 1 .
Atomic metrics
Compound metrics
Epochs
Response Diversity
Judge Score
Deviation
Vendi Score
Distinct Clusters
Filtered
AUC-dev
AUC-qual
VUS
Vendi
Distinct
Vendi
Dist.
Vendi
Dist.
Vendi
Dist.
2
13.4 ±1.6
92.0 ±1.1
5.1 ±0.7
2.27 ±0.20
19.3 ±3.3
0.46 ±0.23
1.20 ±0.85
0.294
1.22
47.7
131.0
30.2
139.0
4
13.9 ±1.5
94.0 ±1.0
7.8 ±0.9
2.42 ±0.21
26.5 ±3.8
0.76 ±0.28
2.35 ±1.33
0.387
2.37
74.1
234.0
37.8
235.2
6
13.4 ±1.3
94.7 ±0.8
9.1 ±0.9
2.37 ±0.18
25.0 ±3.6
0.65 ±0.27
2.49 ±1.41
0.390
2.57
64.2
243.4
37.8
250.9
8
12.2 ±1.2
94.9 ±0.8
9.1 ±0.8
2.23 ±0.18
22.1 ±3.6
0.56 ±0.24
1.95 ±1.23
0.353
2.22
55.4
190.2
34.4
218.1
Appendix
Table 9: DARLING evaluated every 2 epochs on NoveltyBench , without (top) and with (bottom) the deviation reward. The 10-epoch rows match the corresponding DARLING rows in Table 1 ; the 2-epoch rows use the same data budget as DGRPO and CP . Response diversity and deviation are shown ×100 ; cells include a 95% confidence interval where available. Best value per column within each block is bolded. DARLING’s diversity peaks around epochs 4–6 and then erodes with further training, and no checkpoint — at any epoch, with or without deviation — approaches the system-creativity of CP .
Univeristy of Illinois, Urbana-Champaign · Center for Brain Science, Harvard University · CBS-NTT Program in Physics of Intelligence, Harvard University +2