A Dual-Transformer for Multi-Camera View Recommendation

Josep Cabacas-Maso; Carles Ventura; Ismael Benito-Altamirano

cs.CVAug 26, 2026

A Dual-Transformer for Multi-Camera View Recommendation

Authors: Josep Cabacas-Maso, Carles Ventura, Ismael Benito-Altamirano

Organizations: eHealth Center, Faculty of Computer Science, Multimedia and Telecommunicactions, Universitat Oberta de Catalunya, 08018 Barcelona, Spain · Department of Electronic and Biomedical Engineeering, Universitat de Barcelona, 08028 Barcelona, Spain

Abstract

Multi-camera systems are foundational to modern media production, and multi-camera editing is a critical task. This involves the proper selection of the appropriate camera view at each moment. In this paper, we propose a novel Dual-Transformer architecture with Cross-Attention that heavily outperformed the current SOTA models over the TVMCE dataset (TV Shows Multicamera Editing dataset). Our model decouples these tasks: (1) a dedicated temporal encoder first processes the sequence of past frames to build a rich memory of the recent history, and (2) the candidate camera views then act as queries to this memory via a cross-attention module, allowing each candidate to independently interrogate the historical context and find the most relevant information for its own evaluation. Our approach achieved 56.60% [email protected], representing a substantial improvement over the prior best result of 37.16%. We further conducted an ablation study exploring the use of lightweight backbone architectures, where the SwinV2 backbone yielded the best performance, achieving 69.65% [email protected]. Using this best-performing configuration, we then investigated the feasibility of adapting the model to replicate the editing style of a specific human editor. To this end, we fine-tuned the model using varying proportions of the initial segment of a target video. Our results demonstrate that even with only 20% of the video used for fine-tuning, the model exhibited measurable improvements in [email protected], indicating strong potential for data-efficient personalization of editing style adapted to each individual TV show or producer.

Figures & tables

Figure 1: Overview of the multi-camera editing process using an example from the TVMCE dataset [ 13 ] . A professional editor (or an automated system) selects the most appropriate viewpoint from multiple concurrent camera tracks (shown here as Tracks 1, 2, …, 6) to construct a final edited sequence that maintains narrative and visual continuity.

Figure 2: Overview of the proposed Dual-Transformer architecture. The model decouples temporal context encoding from candidate view evaluation. Past frames are processed through a shared backbone (e.g. SwinV2 ) and integrated with temporal and camera embeddings via a Temporal Transformer Encoder . A Cross-Attention mechanism then aligns these historical memory features with candidate frame features, followed by a Candidate Self-Attention layer and a Multi-Layer Perceptron (MLP) Head to produce final recommendation scores.

Category	Model	[email protected] (%)
Baselines	Random Chance	16.67
	Gonzálbez-Biosca et al. [ 6 ]	17.65
	TC-Transformer [ 13 ]	22.50
	Lee et al. [ 9 ] (Previous SOTA)	37.16
Re-impl. SOTA †	Single Transformer [ 9 ] (BCE + SwinV1)	47.97
	Single Transformer [ 9 ] (BCE + SwinV2)	54.06

Table 1: Main results on the TVMCE benchmark. [email protected] (%) is used as the primary metric. Our Dual-Transformer architecture with SwinV2 backbone establishes a significant new performance upper bound. To isolate the architectural contribution from the backbone, we additionally re-implement the prior SOTA [ 9 ] under both backbones.

Backbone	[email protected] (%)	Backbone	[email protected] (%)
SwinV2-Tiny [ 11 ]	69.65	ConvNeXt-Tiny	40.82
MaxViT-Tiny	58.63	EfficientNet-B0	40.66
SwinV1-Tiny [ 12 ]	56.60	CoAtNet-0	34.42
ConvNeXtV2-Tiny	56.23	EVA02-Tiny	33.95
ResNet-50	46.74	ViT-Base [ 18 ]	25.85

Table 2: Backbone sensitivity analysis. All models utilize our Dual-Transformer with Focal Loss. Results highlight the superiority of hierarchical window-based attention for high-resolution editing cues.

Figure 3: Sample predictions from the best-performing model (Dual-Transformer with SwinV2-Tiny backbone and Focal Loss). For each row, the first element is the last frame from the past frames and the remaining elements are the ranking of the C candidate views given by the model with the associated confidence score. The candidate framed in green is the ground truth. Top: Best candidate matches with ground truth (P@0.5=1). Middle: Ground truth frame is placed at 3rd position (P@0.5=0). Bottom: Ground truth fame is placed at 5th position (P@0.5=0). — Figure 3: Sample predictions from the best-performing model (Dual-Transformer with SwinV2-Tiny backbone and Focal Loss). For each row, the first element is the last frame from the past frames and the remaining elements are the ranking of the C candidate views given by the model with the associated confidence score. The candidate framed in green is the ground truth. Top: Best candidate matches with ground truth ([email protected]=1). Middle: Ground truth frame is placed at 3rd position ([email protected]=0). Bottom: Ground truth fame is placed at 5th position ([email protected]=0).

Threshold ( $\tau$ )	Precision (%)	Recall (%)	$F_{1}$ -Score (%)
Validation Set Sweep
0.1	67.81	95.21	75.74
0.2	76.87	90.59	81.21
0.3	79.41	85.48	81.41
0.4	78.51	80.28	79.10
0.5	74.46	74.67	74.53

Table 3: Threshold selection study on the validation set vs. final performance on the test set (Macro-Averaged). The optimal threshold

\tau=0.3

(identified via validation

F_{1}

-score) is used for the final test evaluation.

Threshold ( $\tau$ )	Precision (%)	Recall (%)	$F_{1}$ -Score (%)
Validation Set Sweep
0.1	54.90	95.21	69.65
0.2	68.33	90.59	77.90
0.3	76.01	85.48	80.47
0.4	81.02	80.28	80.65
0.5	85.62	74.67	79.77

Table 4: Threshold selection study using micro-averaged (global) Precision, Recall, and

F_{1}

-score. The optimal threshold

\tau=0.4

(identified via validation

F_{1}

-score) is used for the final test evaluation.

Source Video	Data %	Target Video	[email protected] (%)
General TVMCE	0%	video_0000	74.44
General TVMCE	0%	video_0001	81.76
video_0000	10%	video_0000	78.76
video_0000	10%	video_0001	78.82
video_0000	20%	video_0000	83.65
video_0000	20%	video_0001	76.47

Table 5: Cross-video fine-tuning results. We report [email protected] (%) across different training data percentages.

Video Editing

Video editing uses AI to alter, compose, relight, or reorganize moving footage while controlling how changes persist across frames.

See all 180 papers

CameraEditor: Camera-Controlled Image Editing via Video-Prior Sequential Modeling
Sep 1, 2026Xin Shen, Chengyou Jia, Keshuo Xing +6Diffusion-Based Image Editing Image Editing
Xi’an Jiaotong University, China · Agency for Science, Technology and Research (A*STAR), Singapore
MSEditor: Toward Consistent Multi-Shot Video Editing
Aug 18, 2026Kunyu Feng, Yue Ma, Bingyuan Wang +6Video Editing Video Diffusion Models
HKUST (Guangzhou) · HKUST · Baidu Inc. +2
OmniXRef: A Unified Framework for Multimodal Multi-Reference Video Editing
Oct 15, 2026Xiaoyan Cong, Tianyu Wang, Hyeonho Jeong +7Video Editing Video Diffusion Models
Brown University · Adobe Research

Sep 1, 2026J/K move · Enter open · S save

CameraEditor: Camera-Controlled Image Editing via Video-Prior Sequential Modeling

Xin Shen, Chengyou Jia, Keshuo Xing, Zifeng Zhu+5

Xi’an Jiaotong University, China · Agency for Science, Technology and Research (A*STAR), Singapore

Diffusion-Based Image Editing Image Editing

Open paper AskPDF arXiv

Beyond semantic content, camera parameters play a pivotal role in dictating the geometric perspective and appearance of any given image. While recent image editing models excel at semantic and stylistic manipulation, they struggle with explicit camera parameter control. When handling large perspective shifts, instruction-driven models face a dilemma: they either suffer from structural tearing or generate conservative outputs that ignore geometric instructions. To address this, we introduce CameraEditor, a framework that reformulates camera-controlled editing from a spatial problem into a temporal sequence prediction task. By leveraging the temporal coherence of video diffusion models, our approach integrates an explicit geometric perception module with a dynamic reference routing mechanism. This allows us to construct geometrically rigorous visual reference pairs via dynamic panorama cropping, overcoming the ambiguity of text-based instructions. Furthermore, CameraEditor strategically inserts intermediate transition frames to decompose large perspective shifts, providing a robust temporal buffer that preserves content identity and spatial coherence. We construct a training dataset of 5,760 instances. As an independent contribution, we introduce CamEditor-Bench, a model-agnostic evaluation suite of 462 test cases. Extensive experiments demonstrate that CameraEditor achieves state-of-the-art camera control precision and source identity preservation, outperforming existing methods.

Abstract

Video Editing

Explore similar work

CameraEditor: Camera-Controlled Image Editing via Video-Prior Sequential Modeling

MSEditor: Toward Consistent Multi-Shot Video Editing

OmniXRef: A Unified Framework for Multimodal Multi-Reference Video Editing