cs.CVNov 6, 2026

BayesCrop: Real-Time Probabilistic Multi-Crop Video Reframing

Authors: Cheng-Han Lee, Rajesh Sureddi, Neil Birkbeck, Yilin Wang, Balu Adsumilli, Alan Bovik

Organizations: The University of Texas at Austin · YouTube, Google · University of Colorado Boulder

Abstract

Automatic video reframing is increasingly important for adapting landscape content to heterogeneous display formats. Existing methods typically predict a single deterministic crop, despite the subjective nature of reframing and the presence of multiple plausible regions of interest. We present BayesCrop, a real-time probabilistic framework for multi-crop video reframing. BayesCrop represents crop preferences using a two-component spatially isotropic Gaussian mixture over crop position and scale. Each component defines a representative reframing mode together with its local uncertainty, enabling both dual-view prediction and sampling of additional plausible crop candidates. For efficient video processing, we introduce an anchor-and-propagate strategy that performs neural prediction only on sparse anchor frames and propagates crop trajectories using motion vectors available from the compressed video bitstream. The resulting trajectories can be combined into a single output using a lightweight periodic view-switching strategy. Experiments on LIVE-YT-MC, LIVE-YT-VC++, and a multi-target long-video benchmark demonstrate competitive single-output localization, high multi-crop reference coverage, diverse yet coherent crop hypotheses, and stable perceptual quality. BayesCrop achieves real-time throughput of 112 FPS at the default operating point, while sampling from the learned distributions further improves reference coverage beyond the two representative crop means.

Figures & tables

Explore similar work

CardsList
  1. Show Me When and Where: Towards Referring Video Object Segmentation in the Wild

    Mar 15, 2026Mingqi Gao, Jinyu Yang, Jingnan Luo +4Multimodal TransformerReferring Video Object Segmentation

  2. RefCaptioner: Multi-Reference Image-Grounded Video Captioning

    Jul 30, 2026Tengfei Liu, Yang Shi, Yuran Wang +16Video Caption EvaluationVideo Captioning