Semantically Stable Image Composition Analysis via Saliency and Gradient Vector Flow Fusion

Armin Dadras; Robert Sablatnig; Franziska Proksa; Markus Seidl

cs.CVApr 14, 2026

Semantically Stable Image Composition Analysis via Saliency and Gradient Vector Flow Fusion

Authors: Armin Dadras, Robert Sablatnig, Franziska Proksa, Markus Seidl

Organizations: Computer Vision Lab, TU Wien Vienna, Austria · UAS St. Pölten St. Pölten, Austria

Abstract

The reliable computational assessment of photographic composition requires features that are discriminative of spatial layout yet robust to semantic content. This paper proposes a low-level representation grounded in the assumption that composition can be understood as the flow of visual attention across geometric structure. We introduce VFCNet, which fuses saliency and edge information into a gradient vector flow (GVF) field. The model computes dual-stream GVF representations, integrates them via attention, and extracts multi-scale flow features with a DINOv3 backbone. VFCNet achieves state-of-the-art performance on the PICD benchmark (CDA-1: 0.683, CDA-2: 0.629), improving by 33.1% and 36.1% over the previous best method. We also show that a simple classifier on self-supervised DINOv3 features substantially outperforms more sophisticated, composition-specialized models. Code is available at https://github.com/ADadras/VFCNet

Figures & tables

Figure 1: Overview of VFCNet Architecture: saliency and gradient based GVF are fused in the multi-head module and serve as input to the DINOv3 feature extractor.

Panel of Figure 2 — Figure 2: VFCNet input representation: saliency captures attention distribution; GVF propagates edge structure throughout the image. The prominent feature of cactus draws attention as visible in (b) and modifies the flow as visible in (d).

Panel of Figure 3 — Figure 3: Multi-scale GVF features: divergence captures convergence/divergence of flow, curl captures rotational patterns, magnitude captures flow strength.

Model	Layer	Dim
DINOv3 ViT-B/16	CLS token (block 11)	768
VFCNet	After dual-stream fusion	512
DINOv3+C / RGB baselines	After MLP head	512

Table 1: Embedding specification for CDA evaluation.

Model	CDA-1	CDA-2	DBI $\downarrow$	Sil. $\uparrow$	CDA-1 (P)	CDA-2 (P)
DINOv3 (base)	$0.489{\pm}.002$	$0.392{\pm}.008$	12.34	0.004	–	–
MUSIQ	$0.434{\pm}.002$	$0.369{\pm}.007$	12.09	$-$ 0.079	0.488	0.367
GANC	$0.456{\pm}.005$	$0.423{\pm}.015$	9.64	$-$ 0.154	0.478	0.445
CACNet	$0.513{\pm}.005$	$0.462{\pm}.006$	7.36	$-$ 0.156	0.419	0.405
DINOv3+C	$0.621{\pm}.003$	$0.505{\pm}.003$	5.10	$-$ 0.152	–	–
VFCNet (Ours)	$\mathbf{0.683{\pm}.002}$	$\mathbf{0.629{\pm}.004}$	4.01	$-$ 0.664	–	–

Table 2: SOTA comparison. CDA-2 is primary metric (semantic robustness). VFCNet achieves best performance on both metrics. Results are mean

\pm

std over 5 seeds. (P) refers to the value in the original evaluation.

Configuration	CDA-1	CDA-2
Saliency-only ( $S$ )	0.527	0.493
GVF-only ( $u,v$ )	0.659	0.593
Full input ( $S,u,v$ )	0.621	0.566
RGB-only †	0.621	0.506
RGB + Saliency †	0.673	0.585
RGB + GVF †	0.666	0.583

Table 3: Input modality ablation. All use unfrozen DINOv3 backbone.

\dagger

: RGB input.

Configuration	$\mu$	Iter	CDA-2
VFCNet (Best)	0.15	10	0.629
Higher $\mu$	0.25	30	0.601
Lower $\mu$	0.05	30	0.617
More iterations	0.15	30	0.603
Even more iterations	0.15	50	0.582
Edge Source (10 iter, $\mu$ =0.15)

Table 4: GVF parameters and edge source ablation.

Configuration	CDA-2	$\Delta$
VFCNet (10 iter)	0.629	—
w/o divergence	0.575	$-$ 8.6%
w/o curl	0.603	$-$ 4.1%
w/o magnitude	0.583	$-$ 7.3%

Table 5: GVF feature ablation (10 iterations,

\mu

=0.15).

Saliency Source
Configuration	CDA-2	$\Delta$
DeepGaze (learned)	0.629	—
Edge-based	0.592	$-$ 5.9%
Center-bias	0.588	$-$ 6.5%
Uniform	0.568	$-$ 9.7%
Backbone Fine-tuning

Table 6: Saliency source and backbone ablation (10 iterations).

Figure 4: Per-class CDA-2 comparison: VFCNet vs DINOv3+C.

Flow Map Learning

Flow map learning trains neural models to predict finite-time state or distribution transitions directly, enabling fast simulation, control, and generation.

See all 48 papers

Learning visual representations for compositional analysis of artworks and photographs
Aug 6, 2026Fatemeh Behrad, Tinne Tuytelaars, Johan WagemansVisual Representation Learning Compositional Representation Learning
KU Leuven University Belgium
FlowComposer: Composable Flows for Compositional Zero-Shot Learning
Mar 17, 2026Zhenqi He, Lin Li, Long ChenZero-Shot Learning Compositional Representation Learning
The Hong Kong University of Science and Technology
FlowCIR: Semantic Transport via Flow Matching for Zero-Shot Composed Image Retrieval
Jul 2, 2026Zhenqi He, Ziqi Jiang, Yuanpei Liu +3Vision-Language Models Composed Image Retrieval
The Hong Kong University of Science and Technology, Hong Kong SAR · The University of Hong Kong, Hong Kong SAR

Aug 6, 2026J/K move · Enter open · S save

Learning visual representations for compositional analysis of artworks and photographs

Fatemeh Behrad, Tinne Tuytelaars, Johan Wagemans

KU Leuven University Belgium

Visual Representation Learning Compositional Representation Learning Self-Supervised Visual Representation Learning Composed Image Retrieval

Open paper AskPDF arXiv

Composition, the deliberate arrangement of visual elements, is central to how meaning, emotion, and aesthetic quality are conveyed in artwork, yet it remains among the least formalized dimensions of visual understanding. Prior work highlights a persistent gap in learning meaningful compositional representations, attributing it to semantic bias and suggesting that human-inspired approaches may be key. We compare two parallel paradigms for composition analysis: a human-inspired method grounded in perceptual grouping, and fine-tuned foundation models enabled by recent large-scale compositional datasets. The human-inspired approach uses object-centric models for region-level decomposition and a graph attention network to capture spatial relationships between elements. Both paradigms are evaluated on composition score/category prediction, compositional image retrieval, and visual saliency detection. With frozen encoders, the human-inspired method achieves competitive performance while remaining interpretable. When sufficient data enables fine-tuning, large self-supervised models outperform significantly, but at the cost of interpretability and cross-domain generalization.

Abstract

Flow Map Learning

Explore similar work

Learning visual representations for compositional analysis of artworks and photographs

FlowComposer: Composable Flows for Compositional Zero-Shot Learning

FlowCIR: Semantic Transport via Flow Matching for Zero-Shot Composed Image Retrieval