Testing Associative Creativity in Humans and Multimodal LLMs with Difficult-to-Name Visual Stimuli

Manar Ali; Lisa Gottschalk; Simeon Junker; Lisa Hudalla; Alon Fishman; Joana Cholin; Stefan Kopp; Hendrik Buschmeier

cs.CLOct 20, 2026

Testing Associative Creativity in Humans and Multimodal LLMs with Difficult-to-Name Visual Stimuli

Authors: Manar Ali, Lisa Gottschalk, Simeon Junker, Lisa Hudalla, Alon Fishman, Joana Cholin, Stefan Kopp, Hendrik Buschmeier,

Organizations: CRC 1646 ‘Linguistic Creativity in Communication’ Bielefeld University, Bielefeld, Germany

Abstract

We investigate associative creativity in descriptions of abstract visual stimuli by humans and multimodal large language models (MLLMs). While conceptual association is central to human communication, it remains unclear how humans and models adapt when visual stimuli become increasingly difficult to map onto familiar semantic concepts. We evaluate several state-of-the-art MLLMs and human participants along two dimensions of creativity, originality and effectiveness, using two datasets with controlled manipulations of difficulty. SilCC varies naming difficulty through progressively more complex combinations of abstract shapes, and PixCC varies difficulty through increasing levels of random visual noise. Results show that humans produce more diverse associative descriptions than MLLMs, including greater semantic distance, higher entropy, and more novel expressions relative to reference lexica. As difficulty increases, effectiveness declines in SilCC, and humans increasingly shift from holistic associations to descriptions of low-level visual properties in PixCC.

Figures & tables

Panel of Figure 1 — Figure 1: Example items from the two datasets, SilCC (left) from the highest difficulty level and PixCC (right) in clean condition. Human descriptions reflect associative naming (e.g., “Krabbenraumschiff” (crab spaceship), “Legohaarexplosion” (lego hair explosion) and “Teufel” (devil) for SilCC and “sofa and two cushions”, “car”, “spaceship” for PixCC.

Panel of Figure 2 — Figure 2: Visual stimuli for each difficulty level from SilCC and examples for possible labels given by a source for each stimulus. Difficulty increases from A to D.

Panel of Figure 3 — Figure 3: Four types of stimuli presented in PixCC: creatures, objects, nature, and symbols. All stimuli here are unique clean items with no noise applied. Each stimulus was assigned one of three color palettes.

Figure 4: Gradual distortion of stimuli procedure presented in PixCC. The clean condition (far left) represents the original undistorted item. Noise increases progressively from left to right, with noise levels of 10%, 20%, 30%, 40%, and 70% applied to the stimulus.

Figure 5: Result plots for reflects, entropy, and novelty in SilCC by difficulty level.

Figure 6: Result plots for reflects and entropy (on the full data) and novelty in PixCC by difficulty level.

Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: PixCC example responses for an item (category symbol, item: musical note) with 20% noise added.

Coefficients	Estimate	Std. Error	z	OR (95% CI)	p
(Intercept)	-1.20656	0.14104	-8.555	0.30 [0.23, 0.39]	< .001
Human	2.29927	0.47783	4.812	9.97 [3.91, 25.43]	< .001
LLaVA 34B	2.89487	0.14858	19.484	18.08 [13.51, 24.19]	< .001
Qwen 72B	-0.80519	0.15457	-5.209	0.45 [0.33, 0.61]	< .001
Qwen 235B	-2.99458	0.24812	-12.069	0.05 [0.03, 0.08]	< .001
noise_level.L	-0.90766	0.17094	-5.310	0.40 [0.29, 0.56]	< .001

Appendix

Table 1: Logistic mixed model for associativeness in PixCC:

\textit{associative}\sim\textit{source}*\textit{noise\_level}+(0+\textit{is\_human}+\textit{human\_diff}\mid\textit{participant})+(1\mid\textit{image})

Effect	$\boldsymbol{\chi^{2}}$	df	p
source	1454.90	4	< .001
noise_level	92.06	5	< .001
source:noise_level	123.18	20	< .001

Appendix

Table 2: Type III Wald

\chi^{2}

tests for associativeness in PixCC. Logistic mixed model:

\textit{associative}\sim\textit{source}*\textit{noise\_level}+(0+\textit{is\_human}+\textit{human\_diff}\mid\textit{participant})+(1\mid\textit{image})

Noise	Contrast	Odds ratio	95% CI	p
0%	Human / LLaVA 34B	5.29	[0.508, 55.1]	.296
	Human / Qwen 72B	90.7	[8.99, 916]	< .001
	Human / Qwen 235B	524	[47.4, 5800]	< .001
	Human / InternVL 38B	310	[29.3, 3280]	< .001
	LLaVA 34B / Qwen 72B	17.1	[8.36, 35.1]	< .001
	LLaVA 34B / Qwen 235B	99.0	[37.4, 262]	< .001

Appendix

Table 3: Tukey-adjusted pairwise comparisons for associativeness by noise level in PixCC. Estimates are odds ratios of an associative description (first relative to second source;

>1

: first source more associative). CIs are Tukey-adjusted and back-transformed from the log-odds scale. Odds ratios are conditional on a typical participant.

Figure 8: Associativity on all Data (PixCC)

Coefficients	Estimate	Std. Error	z	OR (95% CI)	p
InternVL 38B	1.37756	0.06672	20.646	3.97 [3.48, 4.52]	< .001
LLaVA 34B	-1.04433	0.03506	-29.783	0.35 [0.33, 0.38]	< .001
Qwen 235B	-1.50122	0.03887	-38.619	0.22 [0.21, 0.24]	< .001
Qwen 72B	-0.26512	0.03640	-7.283	0.77 [0.71, 0.82]	< .001
difficulty_level.L	-3.99126	0.39184	-10.186	0.02 [0.01, 0.04]	< .001
difficulty_level.Q	1.20349	0.38539	3.123	3.33 [1.57, 7.09]	.002

Appendix

Table 4: Cumulative Link Mixed Model for effectiveness in SilCC

\textit{reflects}\sim\textit{source}*\textit{difficulty\_level}+(0+\textit{is\_human}+\textit{human\_diff}\mid\textit{source\_id})+(1\mid\textit{shapeID})

Effect	$\boldsymbol{\chi^{2}}$	df	p
source	2149.812	4	< .001
difficulty_level	111.099	3	< .001
source:difficulty_level	844.932	12	< .001

Appendix

Table 5: Type III Wald

\chi^{2}

tests for effectiveness in SilCC. Cumulative link mixed model on the graded reflects label (no

<

okay

<

yes):

\textit{reflects}\sim\textit{source}*\textit{difficulty\_level}+(0+\textit{is\_human}+\textit{human\_diff}\mid\textit{source\_id})+(1\mid\textit{shapeID})

Difficulty	Contrast	Estimate	95% CI	p
A	Human / InternVL 38B	3.410	[ 2.777, 4.043]	< .001
	Human / LLaVA 34B	3.943	[ 3.290, 4.596]	< .001
	Human / Qwen 72B	2.858	[ 2.218, 3.498]	< .001
	Human / Qwen 235B	-0.262	[-1.085, 0.561]	.909
	InternVL 38B / LLaVA 34B	0.533	[ 0.178, 0.888]	< .001
	InternVL 38B / Qwen 72B	-0.552	[-0.907, -0.197]	< .001

Appendix

Table 6: Tukey-adjusted pairwise comparisons for effectiveness by difficulty level in SilCC. Estimates are differences between the first and second source; positive values indicate a higher estimated effectiveness for the first source.

Coefficients	Estimate	Std. Error	z	OR (95% CI)	p
Human	1.76944	0.15175	11.660	5.87 [4.36, 7.90]	< .001
LLaVA 34B	-0.19099	0.07635	-2.502	0.83 [0.71, 0.96]	.012
Qwen 72B	-0.39592	0.07658	-5.170	0.67 [0.58, 0.78]	< .001
Qwen 235B	1.96963	0.10986	17.928	7.17 [5.78, 8.89]	< .001
noise_level.L	-0.28540	0.18299	-1.560	0.75 [0.53, 1.08]	.119
noise_level.Q	-0.13177	0.14217	-0.927	0.88 [0.66, 1.16]	.354

Appendix

Table 7: Cumulative link mixed model for effectiveness in PixCC:

\textit{reflects}\sim\textit{source}*\textit{noise\_level}+(0+\textit{is\_human}\mid\textit{participant})+(1\mid\textit{image})

Effect	$\boldsymbol{\chi^{2}}$	df	p
source	708.91	4	< .001
noise_level	12.21	5	.032
source:noise_level	482.04	20	< .001

Appendix

Table 8: Type III Wald

\chi^{2}

tests for effectiveness in PixCC. Cumulative link mixed model:

\textit{reflects}\sim\textit{source}*\textit{noise\_level}+(0+\textit{is\_human}\mid\textit{participant})+(1\mid\textit{image})

Noise	Contrast	Estimate	95% CI	p
0%	Human – LLaVA 34B	4.418	[3.045, 5.792]	< .001
	Human – Qwen 72B	4.653	[3.276, 6.030]	< .001
	Human – Qwen 235B	2.240	[0.797, 3.684]	< .001
	Human – InternVL 38B	5.980	[4.574, 7.387]	< .001
	LLaVA 34B – Qwen 72B	0.235	[-0.288, 0.757]	.736
	LLaVA 34B – Qwen 235B	-2.178	[-2.860, -1.496]	< .001

Appendix

Table 9: Tukey-adjusted pairwise comparisons for effectiveness by noise level in PixCC. Estimates are differences in log-odds of a higher reflects rating (first minus second source); CIs are Tukey-adjusted.

Coefficients	Estimate	Std. Error	t	95% CI	p
(Intercept)	0.064439	0.002639	24.415	[ 0.06, 0.07]	< .001
InternVL 38B	0.094523	0.004528	20.874	[ 0.09, 0.10]	< .001
LLaVA 34B	-0.050661	0.004528	-11.188	[-0.06, -0.04]	< .001
Qwen 235B	0.026590	0.004544	5.852	[ 0.02, 0.04]	< .001
Qwen 72B	-0.041895	0.004528	-9.252	[-0.05, -0.03]	< .001
difficulty_level.L	0.082332	0.005276	15.606	[ 0.07, 0.09]	< .001

Appendix

Table 10: Linear mixed model for novelty in SilCC.

\textit{novelty}\sim\textit{source}*\textit{difficulty\_level}+(1\mid\textit{shapeID})

Effect	$\boldsymbol{\chi^{2}}$	df	p
source	576.61	4	< .001
difficulty_level	254.04	3	< .001
source:difficulty_level	255.37	12	< .001

Appendix

Table 11: Type III Wald

\chi^{2}

tests for novelty in SilCC. Linear mixed model for novelty

\textit{novelty}\sim\textit{source}*\textit{difficulty\_level}+(1\mid\textit{shapeID})

Difficulty	Contrast	Estimate	95% CI	p
A	Human / InternVL 38B	0.019	[-0.024, 0.062]	.757
	Human / LLaVA 34B	0.002	[-0.041, 0.045]	1.000
	Human / Qwen 72B	0.025	[-0.018, 0.068]	.502
	Human / Qwen 235B	0.017	[-0.026, 0.060]	.809
	InternVL 38B / LLaVA 34B	-0.016	[-0.059, 0.027]	.837
	InternVL 38B / Qwen 72B	0.006	[-0.037, 0.049]	.994

Appendix

Table 12: Tukey-adjusted pairwise comparisons for novelty by difficulty level in SilCC. Estimates are differences between the first and second source; positive values indicate a higher estimated mean for the first source.

Coefficients	Estimate	Std. Error	t	95% CI	p
(Intercept)	0.36700	0.01355	27.075	[0.34, 0.39]	< .001
Human	0.08400	0.01283	6.546	[0.06, 0.11]	< .001
noise_level.L	0.09687	0.02869	3.376	[0.04, 0.15]	< .001
noise_level.Q	-0.04675	0.02869	-1.629	[-0.10, 0.01]	.105
noise_level.C	0.01187	0.02869	0.414	[-0.04, 0.07]	.680
noise_level^4	0.04946	0.02869	1.724	[-0.01, 0.11]	.087

Appendix

Table 13: Linear mixed model for weighted novelty in PixCC (humans vs. LLaVA 34B, noise levels 10–70), fitted on one value per image

\times

source cell:

\textit{novelty}\sim\textit{source}*\textit{noise\_level}+(1\mid\textit{image})

Effect	$\boldsymbol{\chi^{2}}$	df	p
source	42.86	1	< .001
noise_level	17.20	4	.002
source:noise_level	18.70	4	< .001

Appendix

Table 14: Type III Wald

\chi^{2}

tests for weighted novelty in PixCC (humans and LLaVA 34B, noise levels 10–70%). Linear mixed model on one value per image and source:

\textit{novelty}\sim\textit{source}*\textit{noise\_level}+(1\mid\textit{image})

Noise	Contrast	Estimate	95% CI	p
10%	Human – LLaVA 34B	0.015	[-0.098, 0.128]	.794
20%	Human – LLaVA 34B	0.055	[-0.058, 0.168]	.339
30%	Human – LLaVA 34B	0.225	[0.112, 0.338]	< .001
40%	Human – LLaVA 34B	0.285	[0.172, 0.398]	< .001
70%	Human – LLaVA 34B	0.260	[0.147, 0.373]	< .001

Appendix

Table 15: Pairwise comparisons for weighted novelty by noise level in PixCC (Human minus LLaVA 34B;

SE=0.057

df=171

Figure 9: Semantic similarity for SilCC illustrating the average human and model responses across difficulty levels.

Figure 10: Semantic similarity for PixCC illustrating the average human and model responses across difficulty levels (on the full descriptions).

Coefficients	Estimate	Std. Error	t	95% CI	p
(Intercept)	2.19404	0.08352	26.268	[ 2.03, 2.36]	< .001
InternVL 38B	1.65111	0.09747	16.940	[ 1.46, 1.84]	< .001
LLaVA 34B	-0.61332	0.09747	-6.292	[-0.80, -0.42]	< .001
Qwen 235B	0.38589	0.09784	3.944	[ 0.19, 0.58]	< .001
Qwen 72B	-0.70489	0.09747	-7.232	[-0.90, -0.51]	< .001
difficulty_level.L	2.07201	0.16700	12.407	[ 1.74, 2.40]	< .001

Appendix

Table 16: Linear mixed model for entropy in SilCC.

\textit{entropy}\sim\textit{source}*\textit{difficulty\_level}+(1\mid\textit{shapeID})

Effect	$\boldsymbol{\chi^{2}}$	df	p
source	359.175	4	< .001
difficulty_level	155.143	3	< .001
source:difficulty_level	51.474	12	< .001

Appendix

Table 17: Type III Wald

\chi^{2}

tests for entropy in SilCC. Linear mixed model for entropy

\textit{entropy}\sim\textit{source}*\textit{difficulty\_level}+(1\mid\textit{shapeID})

Difficulty	Contrast	Estimate	95% CI	p
A	Human / InternVL 38B	0.755	[-0.172, 1.681]	.169
	Human / LLaVA 34B	0.140	[-0.786, 1.066]	.994
	Human / Qwen 72B	1.235	[ 0.308, 2.161]	.003
	Human / Qwen 235B	1.162	[ 0.236, 2.088]	.006
	InternVL 38B / LLaVA 34B	-0.615	[-1.541, 0.312]	.363
	InternVL 38B / Qwen 72B	0.480	[-0.447, 1.406]	.615

Appendix

Table 18: Tukey-adjusted pairwise comparisons for entropy by difficulty level in SilCC. Estimates are differences between the first and second source; positive values indicate a higher estimated mean for the first source.

Coefficients	Estimate	Std. Error	t	95% CI	p
(Intercept)	0.93905	0.02010	46.728	[0.90, 0.98]	< .001
Human	1.58553	0.03202	49.523	[1.52, 1.65]	< .001
LLaVA 34B	1.15853	0.03202	36.186	[1.10, 1.22]	< .001
Qwen 72B	-0.91573	0.03202	-28.602	[-0.98, -0.85]	< .001
Qwen 235B	-0.92355	0.03202	-28.846	[-0.99, -0.86]	< .001
noise_level.L	0.03986	0.03921	1.016	[-0.04, 0.12]	.310

Appendix

Table 19: Linear mixed model for entropy in PixCC, fitted on one value per image

\times

source cell:

\textit{entropy}\sim\textit{source}*\textit{noise\_level}+(1\mid\textit{image})

Effect	$\boldsymbol{\chi^{2}}$	df	p
source	4968.61	4	< .001
noise_level	2.66	5	.753
source:noise_level	42.08	20	.003

Appendix

Table 20: Type III Wald

\chi^{2}

tests for semantic head entropy in PixCC. Linear mixed model on one value per image and source:

\textit{entropy}\sim\textit{source}*\textit{noise\_level}+(1\mid\textit{image})

. Robustness check with humans and LLaVA 34B only: source

\chi^{2}(1)=31.21

p<.001

; noise level

\chi^{2}(5)=9.53

p=.090

; interaction

\chi^{2}(5)=7.43

p=.191

Noise	Contrast	Estimate	95% CI	p
0%	Human – LLaVA 34B	0.170	[-0.169, 0.509]	.647
	Human – Qwen 72B	2.093	[1.754, 2.433]	< .001
	Human – Qwen 235B	2.070	[1.730, 2.409]	< .001
	Human – InternVL 38B	1.957	[1.618, 2.296]	< .001
	LLaVA 34B – Qwen 72B	1.923	[1.584, 2.263]	< .001
	LLaVA 34B – Qwen 235B	1.900	[1.560, 2.239]	< .001

Appendix

Table 21: Tukey-adjusted pairwise comparisons for semantic head entropy by noise level in PixCC. Estimates are differences in entropy (bits; first minus second source;

SE=0.124

df=551

); CIs are Tukey-adjusted.

Figure 11: Effectiveness and novelty distributions across sources in SilCC

Figure 12: Effectiveness and entropy distributions across sources in SilCC

Figure 13: Effectiveness and associativness distributions across sources in PixCC

Figure 14: Effectiveness and novelty distributions across sources in PixCC

Figure 15: Effectiveness and entropy distributions across sources in PixCC

Creativity Assessment

Creativity assessment measures originality, usefulness, diversity, and other creative qualities in people and AI-generated work.

See all 44 papers

Measuring Combinatorial Creativity in Large Language Models and Humans
Sep 25, 2025Samuel Schapiro, Akshay K. Jagadish, Alexi Gladstone +11Computational Creativity Creativity Assessment
Knowledge Lab, University of Chicago · AI Innovation Institute, Stony Brook University · Princeton AI Lab +6
How LLMs See Creativity: Zero-Shot Scoring of Visual Creativity with Interpretable Reasoning
Jun 29, 2026William Orwig, Roger E. BeatyCreativity Assessment LLM-as-a-Judge
Harvard University, Department of Psychology, Cambridge, MA, USA · Pennsylvania State University, Department of Psychology, University Park, PA, USA
CREATE: Testing LLMs for Associative Creativity
Mar 10, 2026Manya Wadhwa, Tiasa Singha Roy, Harvey Lederman +2Creativity Assessment Computational Creativity
New York University · The University of Texas at Austin

Sep 25, 2025J/K move · Enter open · S save

Measuring Combinatorial Creativity in Large Language Models and Humans

Samuel Schapiro, Akshay K. Jagadish, Alexi Gladstone, Qiawen Ella Liu+10

Knowledge Lab, University of Chicago · AI Innovation Institute, Stony Brook University · Princeton AI Lab+6

Computational Creativity Creativity Assessment LLM Evaluation

Open paper AskPDF arXiv

Combinatorial creativity, the process of combining familiar ideas in unfamiliar ways, is one of the foremost engines of scientific, technological, and artistic progress. This work introduces Kombine, an LLM benchmark for combinatorial creativity tasking models to discover surprising, original, and useful associations, analogies, and blends between unrelated entities, and to use those analogies and blends to invent novel concepts by re-combining the parts of existing ones. In a large-scale study across 35 LLMs, we find that: (1) LLMs are 10-16 percentage points better at discovering associations and analogies than blends, and (2) exhibit an abstraction bottleneck on blending, in which even frontier LLMs struggle to find a shared abstraction between inputs and instantiate a coherent blended space. (3) LLMs also exhibit inventive homogeneity, in which one in five concepts invented for a given item reuses properties invented by another model. Moreover, (4) models are

14\times

more likely to reuse other invented properties on blends than analogies, and (5) models from the same provider are

2.3\times

more likely to reuse properties than those from different providers. Next, in a comparative study involving 75 humans on the same tasks, we find that (6) humans score below nearly every LLM on each task and (7) both humans and LLMs find blending most difficult, but association is easier than analogy for LLMs, while analogy is easier than association for humans. Lastly, (8) humans and LLMs access distinct regions of semantic space on each task, suggesting that combinatorial creativity could benefit from systematic human-AI collaboration to enhance conceptual diversity. In summary, we provide a framework to assess progress towards LLMs, AI systems, and human-AI collaborations more broadly, that can elicit combinatorial creativity in a setting relevant to scientific and technological innovation.

Abstract

Creativity Assessment

Explore similar work

Measuring Combinatorial Creativity in Large Language Models and Humans

How LLMs See Creativity: Zero-Shot Scoring of Visual Creativity with Interpretable Reasoning

CREATE: Testing LLMs for Associative Creativity