CiQi-Agent: Aligning Vision, Tools and Aesthetics in Multimodal Agent for Cultural Reasoning on Chinese Porcelains

Wenhan Wang; Zhixiang Zhou; Zhongtian Ma; Yanzhu Chen; Ziyu Lin; Hao Sheng; Pengfei Liu; Honglin Ma

cs.CVMar 30, 2026

CiQi-Agent: Aligning Vision, Tools and Aesthetics in Multimodal Agent for Cultural Reasoning on Chinese Porcelains

Authors: Wenhan Wang, Zhixiang Zhou, Zhongtian Ma, Yanzhu Chen, Ziyu Lin, Hao Sheng, Pengfei Liu, Honglin Ma,

Organizations: Shanghai Innovation Institute · Shanghai AI Laboratory · Shaanxi Academy of Cultural Relics Conservation

Abstract

The connoisseurship of antique Chinese porcelain demands extensive historical expertise, material understanding, and aesthetic sensitivity, making it difficult for non-specialists to engage. To democratize cultural-heritage understanding and assist expert connoisseurship, we introduce CiQi-Agent -- a domain-specific Porcelain Connoisseurship Agent for intelligent analysis of antique Chinese porcelain. CiQi-Agent supports multi-image porcelain inputs and enables vision tool invocation and multimodal retrieval-augmented generation, performing fine-grained connoisseurship analysis across six attributes: dynasty, reign period, kiln site, glaze color, decorative motif, and vessel shape. Beyond attribute classification, it captures subtle visual details, retrieves relevant domain knowledge, and integrates visual and textual evidence to produce coherent, explainable connoisseurship descriptions. To achieve this capability, we construct a large-scale, expert-annotated dataset CiQi-VQA, comprising 29,596 porcelain specimens, 51,553 images, and 557,940 visual question--answering pairs, and further establish a comprehensive benchmark CiQi-Bench aligned with the previously mentioned six attributes. CiQi-Agent is trained through supervised fine-tuning, reinforcement learning, and a tool-augmented reasoning framework that integrates two categories of tools: a vision tool and multimodal retrieval tools. Experimental results show that CiQi-Agent (7B) outperforms all competitive open- and closed-source models across all six attributes on CiQi-Bench, achieving on average 12.2% higher accuracy than GPT-5. The model and dataset have been released and are publicly available at https://huggingface.co/datasets/SII-Monument-Valley/CiQi-VQA.

Figures & tables

Figure 1: Comparison between (a) General-purpose MLLM and (b) CiQi-Agent. (a) Conventional MLLMs rely on single-pass answering , directly outputting a label leading to superficial or inaccurate identifications. (b) The proposed CiQi-Agent introduces Tool-Augmented Reasoning , enabling multi-step porcelain analysis through image zoom-in, image/text retrieval. This iterative process yields more reliable final answers aligned with expert connoisseurship reasoning.

Figure 2: Visualization of the four key attributes in porcelain connoisseurship. Shown are the distributions of dynasty, glaze color, vessel shape, and decorative motif in the raw porcelain dataset. For each attribute, the top 10 most frequent categories are presented.

Figure 3: Examples from CiQi-VQA dataset. The figure illustrates annotations across four key attributes: dynasty, glaze color, decorative motif and vessel shape.

	Porcelains	Images	Questions		Attributes
	Porcelains	Images	VQA	Multiple-choice	Attributes
SFT	28,821	50,675	557,165	—
RL ∗	10,275	10,275	10,275	—	dynasty, reign, kiln, color, motif, shape
Evaluation	775	878	775	5,425
Total	29,596	51,553	557,940	5,425	dynasty, reign, kiln, color, motif, shape

Table 1: Overall statistics of CiQi-VQA dataset and CiQi-Bench.

Figure 4: Framework of the CiQi-Agent. The agent integrates visual zoom-in, image/text retrieval tools within a two-phase training pipeline. Supervised fine-tuning establishes tool-calling skills and porcelain connoisseurship knowledge, while reinforcement learning with an LLM-as-a-Judge refines accuracy and strategic tool-calling.

	Dynasty	Reign period	Kiln	Glaze color	Motif	Shape
Pearson $r$	0.995	1.000	0.979	0.958	0.938	0.859
MAE	0.013	0.000	0.036	0.028	0.065	0.077

Table 2: Pearson Correlation and MAE between human expert scores and LLM-as-a-Judge scores on CiQi-Bench. All values are reported to three decimal places.

Model	Multiple-choice Accuracy (%)								Free-form Accuracy (%)
Model	Dynasty	Reign	Kiln	Color	Motif	Shape	Naming	Average	Dynasty	Reign	Kiln	Color	Motif	Shape	Average
GPT-5 [ 30 ]	65.7	61.4	79.6	86.5	69.3	83.8	84.3	75.8	39.4	32.8	42.6	74.4	35.3	63.9	48.0
GPT-4.1 [ 31 ]	59.3	68.3	71.1	85.0	62.2	81.8	77.9	72.2	36.7	27.2	29.0	67.5	27.6	60.1	41.3
GPT-4o [ 29 ]	59.1	60.4	68.6	89.2	70.1	84.2	82.1	73.4	26.9	13.4	15.1	53.9	21.1	47.6	29.7
o3 [ 32 ]	57.6	57.4	72.2	82.6	62.4	76.8	76.6	69.4	42.7	36.6	44.4	74.2	33.1	62.1	48.8
Gemini 2.5 Pro [ 10 ]	54.4	57.4	68.0	65.2	58.5	64.2	82.5	64.3	48.1	34.9	48.4	50.2	16.1	39.0	39.5

Table 3: Performance on CiQi-Bench (Multiple-choice & Free-form). Bold indicates the best performance, and underline indicates the second-best performance.

Model

Multiple-choice Accuracy (%)

Free-form Accuracy (%)

Dynasty

Reign

Kiln

Color

Motif

Shape

Naming

Average

Dynasty

Reign

Kiln

Color

Motif

Shape

Average

Qwen2.5-VL-7B-Instruct [ 35 ]

54.0

60.4

55.1

83.6

69.8

79.6

69.0

67.4

20.3

22.2

15.5

70.6

28.4

61.4

36.4

+ SFT

65.5 ↑

53.5 ↓

77.1 ↑

92.6 ↑

72.8 ↑

88.2 ↑

81.9 ↑

75.9 ↑

64.6 ↑

45.2 ↑

56.6 ↑

81.6 ↑

35.8 ↑

70.9 ↑

59.1 ↑

+ SFT + RL

77.6 ↑

70.3 ↑

81.8 ↑

91.4 ↓

75.7 ↑

88.1 ↓

85.2 ↑

81.5 ↑

71.3 ↑

49.1 ↑

69.8 ↑

85.4 ↑

49.7 ↑

75.0 ↑

66.7 ↑

Table 4: Ablation study on Qwen2.5-VL-7B-Instruct variants. Arrows indicate improvement (↑) or decline (↓) relative to the previous variant, and bold indicates the best performance.

Model	Multiple-choice Accuracy (%)
Model	Dynasty	Reign	Kiln	Color	Motif	Shape	Naming	Average
GPT-5 [ 30 ]	65.7	61.4	79.6	86.5	69.3	83.8	84.3	75.8
+ vision tool	62.4	68.3	80.4	86.9	67.7	83.2	85.0	76.3
+ retrieval tools	54.9	58.4	69.4	85.5	65.6	81.4	80.2	70.8
+ all tools	55.7	58.4	68.3	85.8	66.7	82.3	78.6	70.8
Qwen2.5-VL-7B-Instruct [ 35 ]	54.0	60.4	55.1	83.6	69.8	79.6	69.0	67.4

Table 5: Ablation study on GPT-5, Qwen2.5-VL-7B-Instruct and CiQi-Agent with different tool configurations. Bold indicates the best performance.


Datasets	Porcelain Specimens	Dynasties	Reigns	Kilns	Colors	Motifs	Shapes
Ling et al. [ 19 ]	5,993	2	—	10	8	—	12
Ma et al. [ 23 ]	5,624	—	—	—	—	—	7
Sun et al. [ 38 ]	232	3	—	—	—	—	1
Yang et al. [ 48 ]	2,750	—	—	—	7	—	—
Hu et al. [ 12 ]	8,213	6	—	—	20	6	7

Table 6: Dataset Comparison.


Model	Glaze/Kiln	Shape	Weighted Avg. †
GPT-5 [ 30 ]	86.9	91.4	88.1
GPT-4.1 [ 31 ]	86.4	88.6	87.0
GPT-4o [ 29 ]	86.4	91.4	87.7
o3 [ 32 ]	86.9	91.4	88.1
Claude Opus 4 [ 1 ]	84.4	90.0	85.9

Table 7: Comparative experiments on the Hu et al. [ 12 ] dataset.

Cultural Heritage

Cultural heritage uses AI to document, analyze, preserve, and interpret artifacts, sites, archives, and living traditions.

See all 57 papers

Jun 8, 2026

ChinaHeritaQA: A Culturally-Grounded Visual Question Answering Dataset for World Heritage Sites in China

We introduce ChinaHeritaQA, a multimodal benchmark dataset for evaluating the cultural reasoning abilities of vision-language models (VLMs) on UNESCO World Heritage sites in China. The dataset comprises 2,279 in-the-wild images paired with 14,133 bilingual (Chinese/English) multiple-choice QA pairs spanning seven cognitive dimensions, from basic identity recognition to historical periodization and architectural analysis. Guided by a UNESCO-aligned heritage ontology and verified through rigorous human annotation, the dataset ensures linguistic quality and factual consistency. Evaluations of state-of-the-art VLMs reveal that while top models exceed human performance on average, substantial task-level variation emerges: models excel at visual recognition but struggle with culturally grounded reasoning. Performance also varies by dynasty and region. ChinaHeritaQA reveals that strong visual retrieval does not extend to cultural and historical understanding. We release the dataset to support future research on culturally aware multimodal learning.

Yi Zhang, Bolei Ma, Yong Cao +3

1LMU Munich · 2FAU Erlangen-Nuremberg · 3Munich Center for Machine Learning +4

VLM Evaluation Visual Question Answering Multimodal QA VLM Reasoning Multilingual QA Cultural Knowledge in Language Models+1

Ask PDF

Apr 13, 2026

CArtBench: Evaluating Vision-Language Models on Chinese Art Understanding, Interpretation, and Authenticity

We introduce CARTBENCH, a museum-grounded benchmark for evaluating vision-language models (VLMs) on Chinese artworks beyond short-form recognition and QA. CARTBENCH comprises four subtasks: CURATORQA for evidence-grounded recognition and reasoning, CATALOGCAPTION for structured four-section expert-style appreciation, REINTERPRET for defensible reinterpretation with expert ratings, and CONNOISSEURPAIRS for diagnostic authenticity discrimination under visually similar confounds. CARTBENCH is built by aligning image-bearing Palace Museum objects from Wikidata with authoritative catalog pages, spanning five art categories across multiple dynasties. Across nine representative VLMs, we find that high overall CURATORQA accuracy can mask sharp drops on hard evidence linking and style-to-period inference; long-form appreciation remains far from expert references; and authenticity-oriented diagnostic discrimination stays near chance, underscoring the difficulty of connoisseur-level reasoning for current models.

Xuefeng Wei, Zhixuan Wang, Xuan Zhou +5

Nara Institute of Science and Technology · Liaoning Normal University

VLM Evaluation VLM Reasoning Cultural Heritage

Ask PDF

Jun 19, 2026

CulMind: Benchmarking Multimodal Understanding and Reasoning in Chinese Cultural Heritage

Evaluating Multimodal Large Language Models (MLLMs) in Chinese Cultural Heritage (CCH) requires fine-grained reasoning over visual, textual, stylistic, and historical clues. However, existing CCH benchmarks mainly emphasize final-answer accuracy, while the accuracy and completeness of reasoning processes remain underexplored. To address this gap, we introduce CulMind and CulMind-R: a high-quality benchmark for multimodal CCH covering 50 tasks from collections of more than 100 museums, and a 24-task reasoning subset that adaptively defines task-specific dimensions for reasoning process evaluation. To evaluate reasoning quality, we propose ReaScore, a task-adaptive metric that evaluates reasoning by automatically weighting task-relevant dimensions. Experiments on 14 leading MLLMs reveal a substantial gap between answers and reasoning, especially on challenging tasks. Further analysis shows that task-adaptive dimension selection and weighting better align evaluation results with expert judgments. Overall, our benchmark and metric support a more expert-aligned assessment of CCH understanding and offer a transferable reference for broader evaluations of cultural heritage. We publicly release the data, code, and evaluation scripts at https://github.com/ZevTsao/CulMind to facilitate reproducible research.

Zhangwei Cao, Shuhan Fan, Yuting Wei +5

University of International Relations · 2Kedge Business School · 3Peking University +3

Benchmark Construction Reasoning Evaluation Multimodal Reasoning Cultural Knowledge in Language Models Cultural Heritage Multimodal Model Evaluation+1

Ask PDF

Abstract

Cultural Heritage

Explore similar work

ChinaHeritaQA: A Culturally-Grounded Visual Question Answering Dataset for World Heritage Sites in China

CArtBench: Evaluating Vision-Language Models on Chinese Art Understanding, Interpretation, and Authenticity

CulMind: Benchmarking Multimodal Understanding and Reasoning in Chinese Cultural Heritage