cs.AINov 7, 2026

Evaluating classification and explainability in multimodal petrographic image analysis

Authors: Thomas James Wood, Oktay Karakuş, Can Eyüpoğlu

Organizations: Department of Earth and Environmental Sciences, Cardiff University, Cardiff, United Kingdom. · Department of Computer Science and Informatics, Cardiff University, Cardiff, United Kingdom. · Department of Computer Engineering, Turkish Air Force Academy, National Defence University, Istanbul, Türkiye.

Abstract

Automated thin-section classifiers increasingly combine plane-polarised and cross-polarised light (PPL and XPL), two complementary views central to petrographic identification. Classification accuracy alone does not reveal whether model errors reflect mineralogical relationships, how predictions depend on each optical view, or whether attribution maps identify regions that influence model decisions. Here, we examine these complementary aspects of the released LITHOS dual-stream vision transformer using outputs from an earlier evaluation of 105,802 expert-annotated grains across 25 categories. Classification errors frequently followed the label taxonomy: 72.6% of misclassified plagioclase, microcline and sanidine grains were assigned the generic Feldspar label, while 87.1% of quartz-family grains received a label within that family, compared with 66.6% assigned their exact label. Replacing the XPL input with its per-channel spatial means produced a greater reduction in true-class probability than equivalent PPL replacement in 20 of 25 categories, but not in the three quartz categories. Five explainable artificial intelligence (XAI) attribution procedures generated visually distinct maps for identical inputs. In a superpixel-deletion analysis involving 30 grains from 29 thin sections, method-minus-random contrasts ranged from −0.043-0.043 to 0.0570.057, with all 41 simultaneous 95% confidence intervals including zero. Together, these findings show that error structure, modality sensitivity and attribution behaviour capture distinct aspects of model performance that cannot be established from classification accuracy or visual inspection of attribution maps alone.

Figures & tables

Explore similar work

Oct 17, 2026

Cross-polarization pairing for label-free few-shot thin-section classification

Few-shot classifiers of rock thin sections are meta-trained on many rock types that a petrographer has identified slide by slide, and this identification is the costly step that new collections lack. We study few-shot recognition of unseen rock types when no training slide is identified, and the only information available is what the microscope records anyway: which slide an image comes from, and whether it was taken in plane-polarized (PPL) or cross-polarized (XPL) light. We first show why generic unsupervised pseudo-tasks fit this setting poorly: the positives they find without labels are either wrong or near-duplicates of the anchor, so they teach the model little. Polarized-light microscopy provides positives that avoid both problems. The PPL and XPL views of one field show the same grains, so they are correct by construction, yet they look very different. We propose cross-polarization pairing (XPP), which treats each slide as a pseudo-class and pairs its PPL and XPL views of the same field location, so that matching must rely on grain shape and texture rather than on colour. On the Nanjing University collection of 108 rock types, with support and query images taken from different slides, XPP is the best label-free method on average over three random splits, and matches a supervised model trained on 78 identified rock types. Controls show that colour augmentation and other acquisition positives do not reproduce the gain. We also show that the image-level protocol of earlier work is saturated, and propose a cross-slide protocol in its place.
Apr 16, 2026

From Boundaries to Semantics: Prompt-Guided Multi-Task Learning for Petrographic Thin-section Segmentation

Grain-edge segmentation (GES) and lithology semantic segmentation (LSS) are two pivotal tasks for quantifying rock fabric and composition. However, these two tasks are often treated separately, and the segmentation quality is implausible albeit expensive, time-consuming, and expert-annotated datasets have been used. Recently, foundation models, especially the Segment Anything Model (SAM), have demonstrated impressive robustness for boundary alignment. However, directly adapting SAM to joint GES and LSS is nontrivial due to 1) severe domain gap induced by extinction-dependent color variations and ultra-fine grain boundaries, and 2) lacking novel modules for joint learning on multi-angle petrographic image stacks. In this paper, we propose Petro-SAM, a novel two-stage, multi-task framework that can achieve high-quality joint GES and LSS on petrographic images. Specifically, based on SAM, we introduce a Merge Block to integrate seven polarized views, effectively solving the extinction issue. Moreover, we introduce multi-scale feature fusion and color-entropy priors to refine the detection.
Sep 19, 2026

Reconstructed holograms and explanation-aware evaluation for low-cost computational pollen analysis in veterinary cytology

Automated pollen analysis supports veterinary cytology, but brightfield microscopy is costlier and more complex than lens-less digital in-line holographic microscopy. We evaluate whether reconstructed holograms can narrow this gap and whether model explanations remain reliable under modality change. Six pollen species were imaged by brightfield and holographic microscopy. Raw, single back-propagation and iterative phase retrieval holograms were evaluated with YOLOv26s detection and MobileNetV4 classification after anchor-based annotation transfer. Six attribution methods were assessed for spatial grounding and faithfulness with the Attribution Health Inspection and Repair (AHIR) protocol, which tests model brittleness under weak noise and corrects attribution-map granularity when needed. Brightfield achieved 0.6890 mAP50-95 (0.8865 mAP50) for detection and 0.9687 macro-F1 (0.9705 accuracy) for classification. Reconstructed holograms narrowed the gap with a task-dependent split: p-type was strongest for detection at 0.5324 mAP50-95 (0.8229 mAP50), while r-type was strongest for classification at 0.7695 macro-F1 (0.7866 accuracy), both far above raw-hologram baselines. Activation-based explanations localized strongly on grains, and region-based methods retained ~60 to ~80% of faithfulness under holography. The holographic detector was highly brittle to weak perturbations, saturating deletion-based evaluation while insertion remained informative. Pixel-level gradient explanations approached random floor, yet spatial smoothing restored p-type gradient faithfulness from 0.05 to 0.51. For holographic classification, perturbation-based explanations remained faithful while gradient-based methods fell below random floor. Reconstruction improves low-cost holographic pollen analysis, while AHIR distinguishes genuine attribution failure from artifacts caused by model brittleness and map granularity.