Vision-language models are increasingly used to identify and describe places, yet their recognition and knowledge of specific cultures remain poorly understood, particularly in underrepresented regions such as the Philippines. This study introduces the Tanawpamana Dataset, a repository of Philippine heritage site images with metadata based on the Talapamana registry of the National Commission for Culture and the Arts of the Philippines. We also introduce the Tanawpamana Benchmark, comprising 4,046 tasks designed to evaluate models' recognition and knowledge of heritage sites. Evaluating seven vision-language models, we find that they struggle more with fine-grained visual discrimination among similar heritage sites than with coarse recognition, and that their geographic knowledge is weaker when inferred from image inputs than from text.
Figures & tables
Figure 1: Overview of the Tanawpamana Dataset and Tanawpamana Benchmark. Top: sample dataset entry showing a heritage image and metadata fields. Bottom: Recognition and Knowledge task suites with simplified example questions.
Figure 2: Heritage categories used in Tanawpamana, each shown with a sample photo.
Category
Metro Manila
Luzon
Visayas
Mindanao
Religious Building
21
42
23
11
Civic Building
16
15
16
7
Institutional Building
16
16
15
4
Historic Building
15
16
17
2
Old House
17
15
15
3
Monument
15
15
16
7
Table 1: Number of sites per category and geographic area.
Figure 3: Distribution of heritage sites by geographic area and designation status.
Suite
Task Count
Recognition Suite
Image-to-Name (Easy)
499
Image-to-Name (Hard)
481
Name-to-Image (Easy)
499
Name-to-Image (Hard)
481
Recognition Total
1,960
Table 2: Distribution of tasks in the Tanawpamana Benchmark.
Figure 4: Model accuracy across seven task clusters spanning the Recognition (Easy/Hard) and five Knowledge tasks.
Vision-language model evaluation measures multimodal systems’ perception, reasoning, grounding, robustness, and reliability across controlled and real-world tasks.
University of Manchester Manchester, UK · School of Artificial Intelligence, Wuhan University Wuhan, Hubei, China · Getty Conservation Institute Los Angeles, CA, USA