Artistic style classification is usually studied on complete artworks, where models can exploit global composition, spatial organisation, and iconographic structure. In archaeological settings, however, artworks often survive only as fragmented remains, forcing recognition from incomplete, irregular, and context-limited visual evidence. We study fresco-fragment style classification using a progressive transformer-based framework. Starting from a ViT-B/16 baseline, we introduce foreground-guided masking to suppress background-only tokens, inpainting-based geometric regularisation to align irregular fragment supports with the ViT patch grid, and a supervised contrastive objective that operates on predictive distributions through a Kullback-Leibler similarity and consistently improves every branch. We combine the branches with a deliberately simple learnable logit ensemble. Experiments on CLEOPATRA and POMPAAF show that fragment-aware modelling improves over the standard ViT baseline, with the ensemble increasing accuracy from 0.604 to 0.656 and macro-F1 from 0.596 to 0.648 on CLEOPATRA, and outperforming the best single branch in four of six fragmentation settings on POMPAAF. We additionally evaluate a more complex graph-fusion variant and find that it matches the simple ensemble on POMPAAF while offering only a small, dataset-specific gain on CLEOPATRA, which does not justify its added complexity. Beyond these empirical gains, our contribution is twofold: a distribution-level contrastive objective that consistently sharpens single-branch recognition, and an interpretability analysis that verifies the models exploit genuine painted evidence, while quantifying that the inpainting-based branch draws part of its attribution from the synthesised surround.
Figures & tables
Figure 1 : ViT-B/16 tokenization. A 224×224 image is split into a 14×14 grid of 16×16 patches, followed by a learnable [CLS] token.
Figure 2 : Foreground-guided masking. Background-only tokens are suppressed.
Figure 3 : Inpainting-based geometric regularisation. Each irregular fragment is extrapolated to a full square so that its boundary matches the ViT patch grid; the synthesised pixels are used only for regularisation, not as authentic evidence. For CLEOPATRA, the removed region can be recovered exactly, providing a ground-truth reference for the extrapolation; for POMPAAF it cannot.
Figure 4 : Secondary graph-fusion variant merging patch and global representations from the three ViT branches for final prediction.
Table 4 : Region-level SHAP-BPT attribution on CLEOPATRA, reported as positive-attribution fractions (mean ± std, 100 test fragments).
Figure 5 : Interpretability analysis on a CLEOPATRA fragment. (a) Controlled ablation transforms. (b)–(d) SHAP-BPT attribution maps for the baseline, masked, and inpainted branches across style classes. The baseline and masked branches concentrate attribution on the painted fragment, whereas the inpainted branch, although still interior-dominant, assigns a larger share to the synthesised surround (quantified in Table 4 ).
Attributing an artwork to an artist has traditionally relied on detailed visual observations and descriptions, known as stylistic analysis in art history. By contrast, current artificial intelligence (AI) models used in the field offer only unexplained probabilistic classifications. To bridge this methodological gap, we present an AI framework that automates stylistic analysis of paintings, providing a foundation for enhancing evidence collection, discovery, and verification. By training a vision transformer (ViT) on a large corpus of paintings with metadata, our system encodes this art history-specific data as embeddings. These representations are factorized via sparse dictionary learning into a shared set of features that recur across the training set. A large language model (LLM) then interprets each feature by retrieving associated artworks and their accompanying curator-written texts, and synthesizes them into descriptions that reflect their stylistic attributes. Finally, an autonomous coordinator LLM applies a reasoning-and-action (ReAct) framework to weight, test, and refine these features into cohesive descriptions of an artwork, or comparisons of artworks. This approach converts detailed visual features into descriptive terms, addressing a key challenge in art history. It thus connects the use of images as data with the semantic concerns of humanists, establishing vision-based computational art history as an area for future growth.
Marc S. Walton, Astrid Harth
Museum Studies Programme, The University of Hong Kong, Hong Kong SAR · Department of Chinese and History, City University Hong Kong, Hong Kong SAR
Text-to-image diffusion transformers learn about objects and scenes by learning to generate them, making them strong candidates for training-free zero-shot open-vocabulary semantic segmentation. State-of-the-art attribution methods score each pixel independently, comparing its features against a fixed text-derived class representation, whether as an output-space similarity or as a cross-attention weight. This discards structured signals the model itself exposes: the temporal structure of the generative trajectory, the visual appearance statistics of each concept, and the image's own pairwise feature geometry. We present MAVISEG, a training-free refinement layer that recovers these signals. Because its operators consume only a pixel-by-concept score field and a pixel feature space, MAVISEG is capture-agnostic rather than tied to one attribution method. Across six benchmarks it achieves the strongest overall results among training-free methods, including the best mIoU on every benchmark. Interestingly, gains are largest where the initial capture is weakest, and individual operators contribute depending on the noise in the field they refine. Our results indicate that diffusion transformers carry more concept-level information than current attribution methods recover, and that much of it is lost on the way to the mask rather than absent from the model.
Rajatsubhra Chakraborty, Xujun Che, Ritabrata Chakraborty +2
University of North Carolina at Charlotte, Charlotte, NC, USA · Manipal University Jaipur, Jaipur, Rajasthan, India
We present Subtoken Vision Transformer (SubViT), a selective image tokenization method for fine-grained visual recognition. Standard Vision Transformers compress each fixed-size patch into a single token, although fine-grained distinctions often depend on localized variations within only a few patches. SubViT addresses this mismatch by representing discriminative patches with multiple subtokens while retaining the original token sequence for global context, thereby allocating additional capacity where it is most needed. Since attention heads encode complementary semantics and extracting attention maps at inference requires an extra backbone forward, we adopt a two-stage training strategy. Stage 1 fine-tunes the ViT using subdivision regions sampled from random attention heads, exposing the model to diverse subdivision patterns. Stage 2 identifies informative attention maps through feature-degradation distances and distills them into a lightweight single-map router, which directly predicts deterministic token-importance scores without a separate attention forward. We evaluate SubViT on Generalized Category Discovery (GCD), a challenging task requiring both fine-grained discrimination and generalization to unlabeled novel categories. Across CUB, FGVC-Aircraft, and Stanford-Cars, SubViT improves the average novel-category accuracy of DINOv2 from 81.3% to 84.7%, with only 0.50 ms additional latency and 3.4% more FLOPs, while reducing latency by 73.8% relative to Retina Patch. Code: SubViT.
Jie Zhu, Ivy Zhang, Minchul Kim +1
Michigan State University · Cranbrook Kingswood School · University of North Carolina at Chapel Hill