Lines Are Ideas: Visual Creativity Assessment in Hyperbolic Space
Authors: Baoliang Chen, Zihao Lin, Hao Liu, Xiangjie Sui
Organizations: City University of Hong Kong, Kowloon, Hong Kong SAR, China · South China Normal University, Guangzhou, Guangdong, China · Mango TV (MGTV), Changsha, Hunan, China · City University of Macau, Taipa, Macao SAR, China
Assessing human creativity from drawings provides a scalable way to evaluate human creative ability. However, existing approaches either rely on costly expert annotation or treat creativity as a single regression target, overlooking its underlying structure. Psychological theories have long emphasized that creativity emerges from balancing novelty and appropriateness. Motivated by this principle, we recast creativity as structured semantic deviation between paired drawings, where novelty measures the extent of departure from the original base sketch, and appropriateness determines whether such departures remain meaningful. This formulation naturally leads to a hyperbolic representation, in which novelty is quantified by geodesic distance and appropriateness is evaluated conditionally via a learned decision boundary, further regulated by an abstraction-conditioned adaptive constraint. Experiments on AuDrA-Drawings demonstrate state-of-the-art alignment with expert judgments and strong robustness under rater and task shifts. Beyond performance gains, our framework reveals an interpretable radial organization of creative variation, offering a principled geometric foundation for automatic creativity assessment.
Figures & tables
Figure 1: Motivations . (a) Euclidean geometry is struggles to model the non-uniform, hierarchical concept spaces; (b) hyperbolic geometry provides structure-aware representations via radius- and geodesic-based deviation.
Figure 2: Hyperbolic-space creativity assessment framework .The pipeline consists of four stages: (1) multimodal feature extraction using CLIP encoders to obtain visual features (Fv,Fv) and semantic anchors Ft ; (2) semantic category alignment via cosine similarity to produce content logits zc and abstraction level alignment via an MLP head to obtain zs ; (3) constrained hyperbolic representation by exponential mapping into a Poincaré ball with an abstraction-conditioned adaptive constraint; and (4) creativity scoring via novelty Dnov=dH(h,h) and appropriateness computed from (Dnov,Hsem) , yielding the final creativity score.
Subset
Type
#Samples
Raters
ICC
PD
Abstract
11,075
50
>0.89
RG1
Abstract
670
3 new raters
0.73
RG2
Abstract
722
6 new raters
0.90
FG
Objects
679
3 new raters
0.63
Table 1: Summary of the AuDrA dataset [ 19 ] .
Group
Model
Year
SRCC
PLCC
VGG
VGG16
2014
0.52
0.04
ResNet
ResNet34
2016
0.79
0.81
ResNet50
2016
0.78
0.79
CLIPIQA+
CLIP (ViT-B/32)
2023
0.63
0.65
CLIP (ViT-B/16)
2023
0.67
0.68
CLIP (ViT-L/14)
2023
0.67
0.69
Table 2: Performance comparison on the primary test set.
Group
Model
Year
RG1
RG2
FG
SRCC
PLCC
SRCC
PLCC
SRCC
PLCC
VGG
VGG16
2014
0.58
0.56
0.39
0.32
0.21
0.12
ResNet
ResNet34
2016
0.78
0.75
0.68
0.66
0.45
0.47
ResNet50
2016
0.80
0.77
0.67
0.65
0.46
0.47
CLIPIQA+
CLIP (ViT-B/32)
2023
0.69
0.65
0.41
0.43
0.43
0.44
CLIP (ViT-B/16)
2023
0.73
0.70
0.46
0.47
0.37
0.39
Table 3: Generalization performance on the three unseen subsets in terms of SRCC and PLCC.
Nov.
App.
Adaptive Radius
Hyperbolic
PD
RG1
FG
SRCC / PLCC
SRCC / PLCC
SRCC / PLCC
✓
✓
✓
✓
0.87/0.88
0.79/0.77
0.55/0.56
✗
✓
✓
✓
−0.01/−0.01
−0.06/−0.06
0.05/0.05
✓
✗
✓
✓
0.86/0.85
0.78/0.76
−0.25/−0.20
✓
✓
✗
✓
0.84/0.85
0.78/0.76
0.49/0.51
✓
✓
✗
✗
0.86/0.86
0.78/0.76
0.50/0.50
Table 4: Ablation study results on PD, RG1, and FG. “Nov.” and “App.” denote novelty and appropriateness, respectively.
Figure 3: Correlation between creativity, novelty, and conditional appropriateness. (a) and (b): Correlations of novelty and appropriateness. (b) and (d): Example drawings illustrating how creativity and appropriateness tend to increase with novelty. Under similar novelty levels, higher conditional appropriateness leads to substantially higher creativity scores. “App.” denotes the appropriateness.
Figure 4: Representation magnitude versus abstraction level. (a) Mean norm of CLIP visual embeddings for drawings grouped by abstraction level. (b) Example drawings corresponding to each abstraction category.
Figure 5: Feature geometry in Euclidean vs. hyperbolic space. (a) Euclidean features (b) corresponding hyperbolic embeddings.
Figural divergent thinking is the ability to develop a given shape fragment into an original drawing. In humans, this ability is assessed with incomplete-drawing tasks. We introduce PainterBench, a benchmark that ports the incomplete-drawing task to the agentic setting. The agent draws on a canvas through tool calls and observes the result after every turn. The canvas includes a starting shape which cannot be erased, and the agent's goal is to incorporate this shape into the most original drawing it can produce. The task is open-ended, and the agent itself decides when the drawing is finished. The benchmark tests incremental visual planning over a short horizon and the transfer of creative ability from pretraining to multi-turn tool use. We evaluate 14 multimodal language models from small to frontier scale. Across the primary study and six sensitivity analyses, we collect 2,700 drawings and crowdsource creativity and recognizability ratings for every drawing and for 300 human reference drawings. We also present ViDrA-adapted, an automated scorer that predicts human creativity ratings of agent drawings (r = 0.85 on random held-out test split). Figural divergent thinking varies widely across the 14 models, and GPT-6 Astra produces the most creative drawings. Relative to the human drawings, the agent drawings score higher in creativity but lower in recognizability. We release the final drawings, per-round canvas snapshots, tool call traces, stimulus bank, benchmark harness, crowdsourced ratings (N = 72,000), and ViDrA checkpoint.
Shane K. A. Dalumura Hettige, Jonas Oppenlaender
Computer Science and Engineering University of Oulu Oulu, Finland · Centre for Applied Computing University of Oulu Oulu, Finland
What makes an image creative? Although this may seem like a highly subjective decision, research in psychology suggests that creativity may be based on a set of fundamental creativity dimensions such as novelty, surprise, and arousal. A deeper understanding of image creativity can benefit both research and practical applications. Exploring how human viewers use creativity dimensions to evaluate image creativity may allow us to predict a target image's human-rated creativity automatically. We present a novel framework to estimate the perceived creativity of an image by: (1) predicting the strengths of an image's novelty, surprise, and arousal creativity dimensions, and (2) combining the creativity dimension strengths to estimate an image creativity score. Controlled human-subject experiments were conducted on Amazon Mechanical Turk to collect data to train supervised machine learning models to estimate an image's strengths on the three creativity dimensions. A final machine learning model takes the predicted dimension strengths as input to produce an estimated image creativity score. Results showed that our estimated creativity scores are statistically equivalent to human ratings of creativity for the images we tested. The novel contributions of our research are: (1) identification and validation of three psychological creativity dimensions used to estimate image creativity; (2) supervised machine learning models to automatically estimate novelty, surprise, and arousal strengths for an input image; (3) a supervised machine learning model to predict a creativity score from novelty, surprise, and arousal strengths for an input image, and (4) experimental methods to confirm the accuracy, advantages, and limitations of our approach.
Nisan Chhetri, Christopher G. Healey, Sina Baghbanijam
Evaluating the originality of visual images poses enduring challenges for creativity assessment. Automated scoring using AI models has proven effective in the verbal domain, yet key questions remain about evaluating visual creativity and understanding how models arrive at their ratings. The present research asks whether multimodal large language models (LLMs) can serve as judges of visual creativity zero-shot (without any fine-tuning or examples of human ratings) and whether their "reasoning" output offers an interpretable window into their evaluation process. We tested six multimodal LLMs (Gemini 3 Flash, Gemma 4 31B IT, GPT-5.4 Mini, GLM-5v Turbo, Kimi K2.5, and Qwen 3.6 Plus) on 992 AI-generated images (based on human-written prompts) and 1,500 hand-drawn sketches scored for creativity by human raters. In Study 1, all models showed substantial alignment with human creativity ratings on both datasets (r = .57-.68 on AI-generated images; r = .29-68 on sketches). In Study 2, we analyzed the step-by-step reasoning processes of three LLMs evaluating the same images and drawings. Although reasoning made model evaluations interpretable -- showing what they attend to, how they balance originality vs. quality, and how they justify their ratings -- reasoning did not improve alignment with human ratings. In sum, our findings indicate that multimodal LLMs can match human judgments of visual creativity without any additional training, and that their reasoning reveals how AI models evaluate creativity. An open scoring app implementing this pipeline is available at https://review-visual-eval-scoring.hf.space.
William Orwig, Roger E. Beaty
Harvard University, Department of Psychology, Cambridge, MA, USA · Pennsylvania State University, Department of Psychology, University Park, PA, USA