PhotoGuider: Composition-Aware Cropping and 360$^{\circ}$ View Framing with Explainable Photographic Assessments

Yuan-Kang Lee; Kuan-Lin Chen; Chia-Che Chang; Yu-Lun Liu

cs.CVNov 2, 2026

PhotoGuider: Composition-Aware Cropping and 360 $^{\circ}$ View Framing with Explainable Photographic Assessments

Authors: Yuan-Kang Lee, Kuan-Lin Chen, Chia-Che Chang, Yu-Lun Liu

Organizations: MediaTek Inc. · Department of Computer Science, National Yang Ming Chiao Tung University · National Taiwan University

Abstract

Traditional image cropping assumes the photograph already contains the composition worth keeping, and asks only where to crop. A full 360 $^{\circ}$ scene offers no such guarantee: the viewing direction in a scene has to be discovered before anything can be framed. We present PhotoGuider, a measure-then-frame method that answers both questions with compact learned scorers over one shared set of composition-aware features. The designed features are extracted using foundation and task-specific vision models and structured into four photographic tokens: Subject, Composition, Aesthetics, and Integrity. Our MLLM explainer is fine-tuned to generate composition-related explanations conditioned on images and measured token assessments. To our knowledge, PhotoGuider is the first method to combine explicit composition-based direction selection in 360 $^{\circ}$ panoramas with refined framing, camera-parameter output for native re-rendering, and measurement-conditioned explanations. On two standard 2D cropping benchmarks, PhotoGuider achieves the highest average mIoU performance against ground-truth crops. In a blind ranking study, 120 participants ranked three cropping methods over 100 panoramic scenes from the public Poly Haven HDRI library, placing our framings first in 53.2% of cases, against 27.3% and 19.5% for two strong alternatives. Project page: https://ntuneillee.github.io/research/photoguider/

Figures & tables

Figure 1: PhotoGuider turns a whole scene into a framed photograph and provides a reasoned explanation for its choice of framing. The four stages follow what a photographer actually does upon arriving at a scene: look around, settle on a direction, adjust the frame, and press the shutter. (1) Given a 360° panorama, the system scans look directions and (2) scores each using the proposed four photographic tokens: Subject, Composition, Integrity, and Aesthetics. The maximum fixes the camera direction. (3) The frame is then refined within that view, and (4) the final photograph is rendered natively from the panorama. The same four tokens also ground the explanation: our MLLM explainer is fine-tuned to cite these tokens and conditioned on their measured assessments. Below, two results show the selected crop and the produced explanation; phrases grounded in the four assessments are highlighted.

Figure 2: One set of photographic tokens, three uses. Each token yields one assessment score for a given frame, allowing the decision layer’s output to be separated into four interpretable metrics. These drive our entire pipeline by (1) scoring all view directions in a 360° scene, (2) ranking shortlisted framings and gating each refinement step, and (3) anchoring the fine-tuned MLLM’s explanations. An optional generalist MLLM acts only as a consultant during refinement.

Figure 3: The three learned components of PhotoGuider. We use single ground-truth crops from FCDB and top-rated crops from GAICD, with labels defined by the overlap (IoU) between candidate framings and reference crops. (1) Ranker. Candidates are scored using four LightGBM models (two regression and two ranking) operating on two feature tiers: a 17D core set (saliency, geometry) and a 60D full set (adding SAM object semantics, aesthetic scores, and CLIP prompts). Blending these model outputs yields three quality scores ( $q_{A},q_{B},q_{C}$ ) to determine the optimal crop. (2) Refinement Gates. Six LightGBM gates adjust the selected crop by predicting IoU gains: one gate recenters the frame (via rule-of-thirds and CLIP-attention), four evaluate expansion (based on subject coverage, composition, scale-aesthetic variance, and golden ratio alignment), and a final gate shrinks the frame if no expansion triggers. (3) Explainer MLLM. It generates explanations conditioned on the crop, measured geometry, and ranker attributions. Training data drafts are filtered via four strict quality checks (unmasked geometry, numerical consistency, feature citations, and vocabulary validity).

Figure 4: Feature Partitioning and Token Definitions. The four tokens partition the 60 extracted features into disjoint subsets. Each reported score represents one such component, expressed as a percentile against the same quantity derived from training crops that match the human annotations most closely. The assessed frame is highlighted by the yellow box.

Method	Publication	FCDB		FLMS		Average
Method	Publication	BDE $\downarrow$	IoU $\uparrow$	BDE $\downarrow$	IoU $\uparrow$	BDE $\downarrow$	IoU $\uparrow$
DIC Wang & Shen (2017)	ICCV 2017	0.090	0.630	0.057	0.810	0.074	0.720
DIC ∗ ( Wang et al., 2019 )	TPAMI 2019	0.080	0.650	0.052	0.830	0.066	0.740
A2RL ( Li et al., 2018 )	CVPR 2018	0.089	0.664	0.045	0.821	0.067	0.743
A3RL ( Li et al., 2019 )	TIP 2019	0.077	0.696	–	0.839	–	0.768
VPN ( Wei et al., 2018 )	CVPR 2018	0.073	0.711	0.044	0.836	0.059	0.774

Table 1: Comparison with state-of-the-art methods on FCDB and FLMS datasets.

	Drop-one				Keep-only
	FCDB		FLMS		FCDB		FLMS
Token	BDE $\downarrow$	IoU $\uparrow$	BDE $\downarrow$	IoU $\uparrow$	BDE $\downarrow$	IoU $\uparrow$	BDE $\downarrow$	IoU $\uparrow$
Subject (S)	0.063	0.740	0.046	0.821	0.064	0.735	0.042	0.834
Composition (C)	0.064	0.737	0.042	0.834	0.064	0.735	0.046	0.820
Integrity (I)	0.060	0.754	0.042	0.834	0.068	0.718	0.044	0.826
Aesthetics (A)	0.060	0.753	0.042	0.834	0.070	0.714	0.052	0.792

Table 2: Photographic Token ablation of PhotoGuider in fast mode.

Method	Rank-1	Rank-3	Mean	Scenes
			rank	won
Ours	53.2%	15.2%	1.62	59
Venus	27.3%	37.4%	2.10	26
CACNet	19.5%	47.4%	2.28	15

Table 3: User study. Rank-1/3 report the first- and third-place shares, and Scenes Won counts scenes receiving the most first-place votes. PhotoGuider gets 53.2% first-place and 15.2% third-place rankings, wins 59 of 100 scenes, and is preferred head-to-head over Venus in 65.2% and CACNet in 72.8% of comparisons.

Figure 5: Examples from PhotoGuider on FCDB ( Chen et al., 2017a ) and FLMS ( Fang et al., 2014 ) testing images. Green boxes are ground-truth crops from both datasets, and yellow boxes show crops from PhotoGuider. Bottom-right: IoU between the predicted and ground-truth crops.

Figure 6: Visual comparison of 360° view framing from the Poly Haven HDRI library. Score heatmaps are produced by each method across 120 look directions (24 pan × 5 tilt). Scores are displayed as percentile ranks, with red denoting highly favored directions and a star marking the peak score, alongside the corresponding rendered photograph. Each panel labels the chosen camera parameters (pan / tilt / field of view) and shows the percentage of participants who rated it first. Across 100 scenes, the first-place preference shares are 19.5%, 27.3%, and 53.2%, respectively.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Dataset Bias from FCDB and FLMS datasets. Each point is one crop selected by PhotoGuider, positioned by the contributions of two of the four photographic tokens. Normalized by the pooled standard deviation, the median-to-median displacement is 0.34 SD in the S-A plane and 0.47 SD in the C-I plane.

Source	Anchor	Construction
Base grid	image geometry	center-shift grid, small-area grid and seeded random boxes, each re-aspected to the target ratio
Subject anchors	InSPyReNet subject mask	mask centroid, blobs and bounding box $\times$ seven medium areas
Saliency peaks	DINOv3 saliency map	top-five peaks $\times$ seven medium areas
Multi-scale	the three sources above	top-20 by saliency coverage $\times$ scale factors $\{0.7,0.85,1.15,1.3\}$
Diagonal	subject mask and saliency	frames placing the subject on an image diagonal
Negative space	subject mask and saliency	frames leaving open space opposite the subject

Appendix

Table 4: The eight candidate sources of the aspect ratio-aware pool. Every source is deduplicated by non-maximum suppression at IoU 0.95.

Image	PhotoGuider	Venus	ArtiMuse
fcdb_16002047337	20	16	14
fcdb_9349007347	20	15	15
flms_15659590	20	19	11
flms_799562644	21	15	14
flms_1588688990	19	17	14
All (250)	100 (40.0%)	82 (32.8%)	68 (27.2%)

Appendix

Table 6: Choices per system in the blind three-way study.

Figure 8: An example from the human preference study on composition explanations. PhotoGuider, Venus, and ArtiMuse explain the same crop. The counts above each explanation indicate how many of the 50 participants selected it as best explaining the crop’s compositional aesthetics. Method names are shown for reference and were concealed during the study.

Image Editing

Image editing changes selected visual content while preserving the parts of an image that should remain unchanged.

See all 321 papers

COMEX: A Composition-Grounded Benchmark and Learning Framework for Explainable Aesthetic Image Cropping
Aug 4, 2026Rui Yang, Wei Zhou, Dingyong Gou +5VLM Evaluation Image Editing
State Key Laboratory of Mobile Network and Mobile Multimedia Technology, ZTE Corporation · School of Data Science and Institute of Artificial Intelligence, Chang’an University
ShutterMuse: Capture-Time Photography Guidance with MLLMs
Jun 24, 2026Jiayu Li, Yixiao Fang, Tianyu Hu +5VLM Evaluation Multimodal Large Language Models
1Fudan University · 2StepFun
ShotCrop $^3$ : Cropping Human-Centric Images into Cinematic Triple-Shot Compositions
Jun 4, 2026Dehong Kong, Lina Lei, Lingtao Zheng +10Compositional Image Generation Image Editing
1Huawei Noah’s Ark Lab · 2Sun Yat-sen University

Aug 4, 2026J/K move · Enter open · S save

COMEX: A Composition-Grounded Benchmark and Learning Framework for Explainable Aesthetic Image Cropping

Rui Yang, Wei Zhou, Dingyong Gou, Xiaohui Cui+4

State Key Laboratory of Mobile Network and Mobile Multimedia Technology, ZTE Corporation · School of Data Science and Institute of Artificial Intelligence, Chang’an University

VLM Evaluation Image Editing

Open paper AskPDF arXiv

Explainable aesthetic image cropping requires not only localizing a visually pleasing crop but also explaining why it is preferred. Existing crop-and-explain methods largely treat explanation as post-hoc text generation and overlook composition, a key aesthetic factor that links crop decisions with interpretable reasoning. In this paper, we reformulate explainable aesthetic image cropping as a structured crop-composition-explanation problem. To support this setting, we introduce COMEX, a new benchmark built through image expansion and an IO-reversal pipeline. COMEX contains 33,161 quadruples, each consisting of an expanded image, a crop box, a composition category, and a composition-grounded explanation, enabling joint learning of crop localization, composition understanding, and explanation generation. We further propose a two-stage SFT+GRPO framework, where supervised fine-tuning establishes the structured output protocol and basic cropping ability, and GRPO further improves crop quality, composition prediction, and explanation faithfulness. We benchmark 15 large vision-language models and existing cropping methods on COMEX, establishing a comprehensive testbed for composition-grounded explainable aesthetic cropping. Experiments on both COMEX and prior benchmarks demonstrate the effectiveness and transferability of our framework, with strong performance across evaluation metrics.

Abstract

Image Editing

Explore similar work

COMEX: A Composition-Grounded Benchmark and Learning Framework for Explainable Aesthetic Image Cropping

ShutterMuse: Capture-Time Photography Guidance with MLLMs

ShotCrop3^33: Cropping Human-Centric Images into Cinematic Triple-Shot Compositions

ShotCrop $^3$ : Cropping Human-Centric Images into Cinematic Triple-Shot Compositions