Creativity Assessment

Follow this topic Ask about this topic

Creativity assessment studies how to define and measure creative capacity across tasks such as divergent thinking, association, writing, design, and scientific problem-solving. Methods include human ratings, psychometric tests, task-specific benchmarks, semantic novelty and diversity metrics, and language-model judges. Good assessments distinguish novelty from usefulness, constraint satisfaction, and coherence, and may examine the creative process as well as the final result. A central challenge is that measures often fail to generalize across domains, while automated evaluations can disagree with human judgments or favor particular styles and sources.

Latest papers

44 papers5 papers in the last four weeks, up 67% on the four weeks before. 0.0% of all new papers.

Testing Associative Creativity in Humans and Multimodal LLMs with Difficult-to-Name Visual Stimuli
Oct 20, 2026Manar Ali, Lisa Gottschalk, Simeon Junker +6Creativity Assessment Multimodal Large Language Models
CRC 1646 ‘Linguistic Creativity in Communication’ Bielefeld University, Bielefeld, Germany
Evaluating LLM Creativity through Long-Tail Performance
Oct 9, 2026Yichen Wang, Tenghao Huang, Vishakh Padmakumar +1Creativity Assessment LLM Evaluation
University of Chicago · Microsoft · Stanford University
Towards Defining, Measuring, and Leveraging Creativity in AI Applications
Oct 9, 2026Jingyi Yang, Alexander TuzhilinCreativity Assessment Computational Creativity
New York University, Stern School of Business
Design Creativity Bench: Measuring creativity in LLM-Generated UI
Oct 8, 2026Aman Rusia, Abhijit Bhole, Prashank Gupta +1Creativity Assessment Language Model Generation Evaluation
Kombai Inc.
PainterBench: A Figural Divergent-Thinking Benchmark for Tool-Using Language Models
Sep 28, 2026Shane K. A. Dalumura Hettige, Jonas OppenlaenderCreativity Assessment Agentic Image Generation
Computer Science and Engineering University of Oulu Oulu, Finland · Centre for Applied Computing University of Oulu Oulu, Finland
Measuring the Creativity of Frontier LLMs in Automated Research
Sep 12, 2026Yiheng Zhao, Mengzhuo Chen, Chengming Hu +3Creativity Assessment LLM Evaluation
Concordia University, Montreal, Canada · Independent Researcher · Mcgill University, Montreal, Canada +2
Decoupled Analysis-Judging: An Automated Creativity Evaluator Using LLMs in Complex Multi-step Creativity Tasks
Sep 3, 2026Xiangyu Wang, Jin Wu, Xiaoyu Li +2Creativity Assessment LLM-as-a-Judge
Department of Educational Psychology, East China Normal University · 2Shanghai Institute of Artificial Intelligence for Education, East China Normal University · School of Computer Science and Technology, East China Normal University +2
CLIN: an Objective Framework for Evaluating Creativity in Short Persian Literary Text
Aug 31, 2026Mohammad Reza Modarres, Armin Tourajmehr, Yadollah Yaghoobzadeh +1Creativity Assessment LLM Evaluation
School of Electrical and Computer Engineering, College of Engineering, University of Tehran, Tehran, Iran · Tehran Institute for Advanced Studies, Khatam University, Iran · Cardiff University, United Kingdom
Can LLM Agents Discover? Evaluating Creativity on ML Engineering Tasks
Aug 30, 2026Shitanshu Bhushan, Yunxiang Zhang, Lu WangCreativity Assessment LLM Agent Evaluation
Computer Science and Engineering University of Michigan Ann Arbor, MI, USA
The Limits of Automatic Evaluation of Creativity in Large Language Models
Aug 24, 2026Alessandro Tutone, Giorgio Franceschelli, Mirco MusolesiCreativity Assessment Language Model Generation Evaluation
Department of Computer Science and Engineering, University of Bologna, Bologna, Italy. · Centre for Artificial Intelligence, Department of Computer Science, University College London, London, United Kingdom.
Are LLMs becoming similarly creative? Evidence from three years of models
Aug 19, 2026Nirav Patel, Josiah Crossman, Eva Aggarwal +1Homogenization in Language Models Computational Creativity
Department of Computer Science, Duke University · Department of Electrical & Computer Engineering, Duke University
Assessing LLMs' mathematical abilities requires understanding the various mechanisms of mathematical creativity
Aug 17, 2026Silvère GangloffLLM Mathematical Reasoning Creativity Assessment
University of Ostrava, Ostrava, Czech Republic.
Recipes for Creativity: Iterative Generation and Evaluation in Large Language Models
Aug 7, 2026Rens Anderson, Tessa Verhoef, Amirhossein ZohrehvandCreativity Assessment LLM Evaluation
LIACS Leiden University Einsteinweg 55 2333 CC Leiden The Netherlands
Can Language Models Imagine Without Seeing? Ekphrasis: Measuring Visual Creative Ideation in Text-Only LLMs
Aug 7, 2026Hongyu Luo, He Wang, Huihao Jing +6Creativity Assessment LLM Evaluation
The Hong Kong University of Science and Technology, Hong Kong SAR, China
Can MLLMs Decode the Creative Leap? Introducing C4 for Cross-Concept Understanding
Aug 6, 2026Ming Wang, Yuqing Zhang, Tingna Xie +5Creativity Assessment Multimodal Large Language Models
School of Computer Science and Engineering, Northeastern University, Liaoning, China · School of Computing and Information Systems, Singapore Management University, Singapore · School of Computer Science and Engineering, Shandong University of Science and Technology, Shandong, China
Why Large Language Models and Humans Converge and Diverge in Evaluating Creativity
Jul 24, 2026Pengzhao Lyu, Yeun Joon Kim, Hanlin Xiao +1Creativity Assessment LLM Evaluation
Judge Business School, University of Cambridge, Cambridge, CB2 1AG, United Kingdom · Institute of Metabolic Science, School of Clinical Medicine, University of Cambridge, CB2 0QQ, United Kingdom · Department of Computer Science, University of Manchester, M13 9PL, United Kingdom +2
AGC-Bench: Measuring Artificial General Creativity
Jul 1, 2026Roger Beaty, Vijeta Deshpande, Clin K. Y. Lai +9Creativity Assessment LLM Evaluation
1Pennsylvania State University · University of Massachusetts Lowell · University of Amsterdam +2
The Human Creativity Benchmark
Jun 29, 2026Aspen Hopkins, Allison Nulty, Alexandria Minetti +2Human Preference Evaluation Creativity Assessment
Massachusetts Institute of Technology Cambridge, MA, USA · Contra New York, New York, USA
How LLMs See Creativity: Zero-Shot Scoring of Visual Creativity with Interpretable Reasoning
Jun 29, 2026William Orwig, Roger E. BeatyCreativity Assessment LLM-as-a-Judge
Harvard University, Department of Psychology, Cambridge, MA, USA · Pennsylvania State University, Department of Psychology, University Park, PA, USA
Creative Integration: A Decidable Criterion of Creativity
Jun 11, 2026Yoshinori NomuraCreativity Assessment Computational Creativity
Mirage Mountain Technologies
IntElicit: Eliciting and Assessing Contextualized Creativity via Dialogue Policy Optimization
Jun 10, 2026Mingjia Li, Jin Wu, Hong Qian +7Creativity Assessment Policy Optimization
East China Normal University · Shanghai Innovation Institute
Automated Creativity Evaluation of Language Models Across Open-Ended Tasks
Jun 10, 2026Min Sen Tan, Zachary Kit Chun Choy, Syed Ali Redha Alsagoff +4Creativity Assessment LLM Evaluation
1Raffles Institution · College of Computing and Data Science, Nanyang Technological University · 3Lee Kong Chian School of Medicine, Nanyang Technological University +1
Introducing multiplex semantic networks as multifaceted representations of creative associative knowledge across multilingual samples
Jun 8, 2026Edith Haim, Kurt Haim, Roger E. Beaty +2Creativity Assessment Cognitive Modeling
CogNosco Lab, Department of Psychology and Cognitive Science, University of Trento, Rovereto, Trento · Department of Science Education, University of Education Upper Austria, Linz, Austria · Department of Psychology, The Pennsylvania State University, Pennsylvania, USA +1
DEFINED: A Data-Efficient Computational Framework for Fine-Grained Creativity Assessment in Debate Scenarios
Jun 5, 2026Tongzhou Yu, Mingjia Li, Hong Qian +6Creativity Assessment Computational Creativity
Nanjing University Nanjing, China · Shanghai Innovation Institute Shanghai, China · East China Normal University Shanghai, China
Before and After Temperature: A Distributional View of Creative LLM Generation
May 31, 2026V. S. Raghu Parupudi, Harsha Ponnada, Aditi Kaushal +3Open-Ended Generation Creativity Assessment
University of California, San Diego · Indian Institute of Technology, Kharagpur · Delhi Technological University +3
IDEAFix: Evaluation Framework for Creative Defixation Prompting in LLMs
May 30, 2026F. Carichon, S. Sharma, M. Girard +2Creativity Assessment LLM Evaluation
McGill University, Montreal, Canada · Mila, Montreal, Canada · Concordia University, Montreal, Canada
QUIET: A Multi-Blank Cascaded Story Cloze Benchmark for LLM Creative Generation Capability
May 25, 2026Bo Zou, Chao XuOpen-Ended Generation Creativity Assessment
When Reasoning Supervision Hurts: TTCW-Based Long-Form Literary Review Generation
May 19, 2026Jinlong Liu, Mohammed Bahja, Mark LeeSupervised Fine-Tuning Creativity Assessment
School of Computer Science, University of Birmingham, United Kingdom
Creativity Bias: How Machine Evaluation Struggles with Creativity in Literary Translations
May 13, 2026Kyo Gerrits, Rik van Noord, Ana Guerberof ArenasCreativity Assessment LLM-as-a-Judge
Centre for Language and Cognition, University of Groningen
Assessing the Creativity of Large Language Models: Testing, Limits, and New Frontiers
May 13, 2026Samuel Schapiro, Alexi Gladstone, Jonah Black +1Creativity Assessment LLM Evaluation
University of Illinois Urbana-Champaign

Oct 20, 2026J/K move · Enter open · S save

Testing Associative Creativity in Humans and Multimodal LLMs with Difficult-to-Name Visual Stimuli

Manar Ali, Lisa Gottschalk, Simeon Junker, Lisa Hudalla+5

CRC 1646 ‘Linguistic Creativity in Communication’ Bielefeld University, Bielefeld, Germany

Creativity Assessment Multimodal Large Language Models Computational Creativity Multimodal Model Evaluation

Open paper AskPDF arXiv

We investigate associative creativity in descriptions of abstract visual stimuli by humans and multimodal large language models (MLLMs). While conceptual association is central to human communication, it remains unclear how humans and models adapt when visual stimuli become increasingly difficult to map onto familiar semantic concepts. We evaluate several state-of-the-art MLLMs and human participants along two dimensions of creativity, originality and effectiveness, using two datasets with controlled manipulations of difficulty. SilCC varies naming difficulty through progressively more complex combinations of abstract shapes, and PixCC varies difficulty through increasing levels of random visual noise. Results show that humans produce more diverse associative descriptions than MLLMs, including greater semantic distance, higher entropy, and more novel expressions relative to reference lexica. As difficulty increases, effectiveness declines in SilCC, and humans increasingly shift from holistic associations to descriptions of low-level visual properties in PixCC.