Abstract4D: A Large-Scale Dataset and Framework for Understanding the Visual Language of Abstract Art

Haowei Zhang; Yuanpei Zhao; Ji-Zhe Zhou; Mao Li

cs.CVAug 28, 2026

Abstract4D: A Large-Scale Dataset and Framework for Understanding the Visual Language of Abstract Art

Authors: Haowei Zhang, Yuanpei Zhao, Ji-Zhe Zhou, Mao Li

Organizations: College of Computer Science, Sichuan University, Chengdu, China

Abstract

Artificial intelligence can classify artistic styles and synthesize images, but it still lacks a model of the visual language that gives art meaning. Abstract painting minimizes object semantics and foregrounds structural cues, making it an ideal testbed for computational perception. We introduce Abstract4D, the largest dataset of abstract paintings to date: more than 120,000 images paired with rich metadata and multi-dimensional prompts that capture each work's perceptual attributes---form, color, texture, and composition. Annotations are produced by a hybrid human--VLM pipeline for quality and consistency. Using Abstract4D, we (i) analyze the semantic structure of abstract art through large-scale embedding visualization, uncovering how perceptual relationships organize artistic meaning, and (ii) establish benchmark tasks for classification, cross-modal retrieval, and text-to-image generation to evaluate how AI models perceive and reproduce abstract visual language. Together, these analyses demonstrate how Abstract4D enables both exploration and quantitative assessment of AI's ability to represent and interpret abstract art.

Figures & tables

Figure 1: Overview of the Abstract4D dataset. Each abstract painting is paired with normalized metadata and a multi-dimensional perceptual prompt describing form, color, texture, and composition. Color-coded tokens indicate different perceptual dimensions.

Figure 2: Overview of the Abstract4D data processing and annotation pipeline. Abstract artworks are collected from open sources, cleaned, and filtered through visual and semantic validation. Each valid painting is paired with metadata and a four-dimensional textual description generated via visual language models (VLMs) under both template-guided and free-form prompting. Human evaluators refine the outputs to form the final 4D Prompt , integrating perceptual dimensions of form, color, texture, and composition.

Dimension	Description	Representative Keywords
Form	Structural and geometric organization of visual elements.	geometric, organic, angular, curved, fragmented, fractal
Color	Chromatic relationships of tone, contrast, and temperature.	warm, cool, monochrome, contrasting, gradient
Texture	Perceived surface quality and material expression.	smooth, rough, layered, reflective
Composition	Spatial arrangement and visual balance.	centered, diagonal, balanced, hierarchical, radial, dispersed

Table 1: Four-dimensional perceptual framework of abstract art. Each dimension captures a core aspect of abstract visual perception.

Figure 3: Overview of the Abstract4D dataset statistics. (a) Top-30 artists ranked by the number of artworks. (b) Distribution of artworks across decades, showing a marked increase after 2000. (c) Proportion of artworks by artist nationality. Together, these visualizations illustrate the temporal growth and geographical diversity of the Abstract4D dataset, highlighting its balance across artistic contributors, historical periods, and cultural origins.

Dataset	#Images	Abstract Focus	Perceptual / Semantic Annotations
WikiArt wik (2025)	196K	$\sim$ 9% (18K) abstract works	Style / genre / artist labels
BAM! Wilber et al. (2017)	2.5M artworks	Low (few-abstract)	Content, media, emotion tags
OmniArt Strezoski and Worring (2018)	2M	Low (few abstract)	Multi-label: style, material, technique
iMet Collection Zhang et al. (2019)	130K	Minimal	Fine-grained expert attributes
ArtEmis Achlioptas et al. (2021)	80K	Low (mostly figurative)	Emotion + explanation texts
Painter by Numbers small yellow duck and Kan (2016)	100K	Negligible	Author labels (pairs)

Table 2: Comparison of Abstract4D with major existing art datasets. While prior datasets such as WikiArt and OmniArt provide large-scale style or metadata labels, they contain limited abstract works and lack structured perceptual annotations. Abstract4D uniquely focuses on purely abstract paintings , offering multi-dimensional perceptual descriptions aligned with artistic language.

Figure 4: UMAP Visualization of Abstract4D Semantic Space. (a): Raw distribution of 120K prompts showing a continuous manifold. (b): KMeans clustering ( $K{=}30$ ) revealing smooth transitions in perceptual themes. (c)-(f): Cross-semantic visualization color-coded by perceptual dimensions ( Form , Color , Texture , Composition ), highlighting the fluidity of abstract art semantics.

Task	Precision	Recall	F1-score	Top1	Top5	Macro F1
Artist	0.680	0.496	0.487	0.7382	0.8328	0.7320
Nationality	0.272	0.196	0.164	0.1960	0.4383	0.1635

Table 3: Classification performance of the CLIP base linear probe on the artist-level and nationality-level subsets of the Abstract4D dataset. All metrics are macro-averaged across classes.

Figure 5: Heatmap showing the mean $z$ -normalized scores of paintings across four semantic axes (Form, Color, Texture, Composition) grouped by nationality.

Figure 6: Confusion matrices for CLIP linear probe classification. (a) Artist classification shows clear diagonal dominance, indicating distinct individual styles. (b) Nationality classification exhibits dispersed patterns, reflecting weak cross-national regularities.

Figure 7: Qualitative comparison of Flux generative results. The main grid (upper area) shows three representative prompts (left) with their reference paintings and generations from Flux (no finetune) , Flux + WD-LoRA , and Flux + Abstract4D-LoRA . The Abstract4D-LoRA model produces outputs better meet the four-dimensional description requirements and closely align with the reference images, demonstrating improved compositional balance, coherent chromatic relationships, and painterly texture realism. The blue panel at the bottom-left ( Generations ) displays additional artworks freely generated by the Abstract4D-LoRA model, illustrating its creative diversity and fidelity to the visual language of abstract art.

Model	Dir.	R@1	MedR	nDCG@10
Zero-shot CLIP	T $\to$ I	0.7035	5.0	0.7363
	I $\to$ T	0.7308	4.0	0.7590
Fine-tuned CLIP	T $\to$ I	0.8934	1.0	0.9226
	I $\to$ T	0.8956	1.0	0.9240

Table 4: Text–image retrieval results on the Abstract4D validation and test sets. Fine-tuning substantially improves cross-modal alignment over the zero-shot CLIP baseline.

T2I Generation Evaluation

Text-to-image generation evaluation measures how accurately, safely, and consistently image models turn prompts into images.

See all 163 papers

MMArt: A Multi-Perspective Multimodal Dataset for Visual Art Understanding
Aug 11, 2026Shuai Wang, Wangyuan Ding, Yixian Shen +5Image Captioning Vision-Language Models
University of Amsterdam, Amsterdam, The Netherlands · Amazon AGI, United States of America · College of Business and Economics, University of Johannesburg, South Africa
SynGallery: A Synthetic Gallery of Real Paintings for Instance-Level Artwork Recognition
Jul 21, 2026Patryk Bartkowiak, Jakub Markil, Bartosz Kotrys +3Synthetic-to-Real Domain Adaptation Synthetic Data Augmentation
Adam Mickiewicz University · ArtiCollect · KAUST +1
ArtSeek: Deep artwork understanding via multimodal in-context reasoning and late interaction retrieval
Jul 29, 2025Nicola Fanelli, Gennaro Vessio, Giovanna CastellanoMultimodal RAG Multimodal IR
Department of Computer Science, University of Bari Aldo Moro, Bari, Italy

Aug 11, 2026J/K move · Enter open · S save

MMArt: A Multi-Perspective Multimodal Dataset for Visual Art Understanding

Shuai Wang, Wangyuan Ding, Yixian Shen, Jia-Hong Huang+4

University of Amsterdam, Amsterdam, The Netherlands · Amazon AGI, United States of America · College of Business and Economics, University of Johannesburg, South Africa

Image Captioning Vision-Language Models Multimodal Dataset

Open paper AskPDF arXiv

Recent vision-language models demonstrate impressive general visual understanding, yet their art interpretation remains shallow: they describe surface content but struggle with formal analysis, grounded historical interpretation, or affective characterization. We argue this is not only a model but also a dataset limitation. Existing art datasets are single perspective resources, where no dataset provides narrative, formal, emotional, and historical perspectives simultaneously for the same artworks. We introduce MMArt, a large-scale dataset of 74,234 WikiArt paintings, each annotated with four independently annotated perspectives plus a harmonized unified caption, produced by specialized vision-language models or human annotation and validated through complementary quality evaluations. Two complementarity analyses establish that perspectives encode genuinely distinct information. A generative analysis shows that formal analysis descriptions best preserve compositional style, and historical descriptions carry strong affective signal in reconstructed images. A discriminative retrieval analysis reveals task-asymmetry: narrative descriptions drive retrieval (R@1 = 44.0%), while formal descriptions, strongest for reconstruction, are nearly nondiscriminative at retrieval scale (R@1 = 7.8%). Leave-one-out analysis further confirms that historical descriptions are the least replaceable perspective across both tasks. Together, the two analyses establish that no single perspective suffices for all tasks, directly motivating MMArt multi-perspective design. The dataset, code, and additional information are available at https://shuaiwang97.github.io/MMArt/.

Abstract

T2I Generation Evaluation

Explore similar work

MMArt: A Multi-Perspective Multimodal Dataset for Visual Art Understanding

SynGallery: A Synthetic Gallery of Real Paintings for Instance-Level Artwork Recognition

ArtSeek: Deep artwork understanding via multimodal in-context reasoning and late interaction retrieval