Vision-Language Based Expert Reporting for Painting Authentication and Defect Detection
Authors: Eman Ouda, Mohammed Salah, Arsenii O. Chulkov, Gianfranco Gargiulo, Gian Luca Tartaglia, Stefano Sfarra, Yusra Abdulrahman
Organizations: Khalifa University of Science and Technology, Department of Aerospace Engineering, Abu Dhabi, United Arab Emirates · National Research Tomsk Polytechnic University, Russia · Academy of Fine Arts of Naples, Naples, Italy · Academy of Fine Arts of Sassari, Sassari, Italy · Department of Industrial and Information Engineering and Economics (DIIIE), University of L’Aquila, L’Aquila I-67100, Italy · Advanced Research and Innovation Center (ARIC), Khalifa University of Science & Technology, Abu Dhabi, UAE
Authenticity and condition assessment are central to conservation decision-making, yet interpretation and reporting of thermographic output remain largely bespoke and expert-dependent, complicating comparison across collections and limiting systematic integration into conservation documentation. Pulsed Active Infrared Thermography (AIRT) is sensitive to subsurface features such as material heterogeneity, voids, and past interventions; however, its broader adoption is constrained by artifact misinterpretation, inter-laboratory variability, and the absence of standardized, explainable reporting frameworks. Although multi-modal thermographic processing techniques are established, their integration with structured natural-language interpretation has not been explored in cultural heritage. A fully automated thermography-vision-language model (VLM) framework is presented. It combines multi-modal AIRT analysis with modality-aware textual reporting, without human intervention during inference. Thermal sequences are processed using Principal Component Thermography (PCT), Thermographic Signal Reconstruction (TSR), and Pulsed Phase Thermography (PPT), and the resulting anomaly masks are fused into a consensus segmentation that emphasizes regions supported by multiple thermal indicators while mitigating boundary artifacts. The fused evidence is provided to a VLM, which generates structured reports describing the location of the anomaly, thermal behavior, and plausible physical interpretations while explicitly acknowledging the uncertainty and diagnostic limitations. Evaluation on two marquetries demonstrates consistent anomaly detection and stable structured interpretations, indicating reproducibility and generalizability across samples.
Figures & tables
Figure 1: Optical images of the Italian marquetry samples analyzed in this study: (a) the “Boy” and (b) the “Girl” panel.
Figure 2: Active pulsed infrared thermography setup used for the acquisition of thermal sequences under conservation-safe conditions.
Figure 3: Thermography–VLM framework. Multi-modal thermographic maps are fused into a consensus anomaly map and provided, with the optical image, to a VLM that generates structured defect reports and quantitative summaries.
Table 1: Representative thermographic outputs for the Boy and Girl marquetries, including PCT (representative principal component), TSR slope maps, and PPT phase maps, and the corresponding consensus anomaly maps. All images are derived from the selected ROI and contrast-normalized for visualization.
Mask / Map
Boy area (%)
Girl area (%)
PCT
1.00
1.00
TSR slope
5.68
3.80
PPT amplitude
1.00
1.00
PPT phase-edge
5.00
5.00
Consensus anomaly mask
8.84 (8 regions)
7.02 (9 regions)
Table 2: Area of anomaly masks and consensus regions for the Boy and Girl samples.