cs.CVOct 7, 2025

SD-MVSum: Script-Driven Multimodal Video Summarization Method and Datasets

Authors: Manolis Mylonas, Charalampia Zerva, Evlampios Apostolidis, Vasileios Mezaris

Organizations: CERTH-ITI Thermi, Greece

Abstract

In this work, we present a method and two large-scale datasets for Script-Driven Multimodal Video Summarization. The proposed method, SD-MVSum, builds on our earlier SD-VSum method for script-driven video summarization. SD-VSum considered the relevance of a provided script, which outlines the desired content of the summary, only with the visual content of the video. SD-MVSum takes into account, in addition to the visual modality, the relevance of the script with the spoken content (i.e., the audio transcript) of the video. The dependence between each considered pair of data modalities, i.e., script-video and script-transcript, is modeled using a new weighted cross-modal attention mechanism. This mechanism explicitly exploits the semantic similarity between the paired modalities in order to promote the parts of the full-length video with the highest relevance to the script. Furthermore, we extend two large-scale datasets for script-driven (S-VideoXum) and generic (MrHiSum) video summarization, to make them suitable for training and evaluation of script-driven multimodal video summarization methods. Experimental comparisons document the competitiveness of the proposed SD-MVSum method against other SotA approaches for script-driven and generic video summarization. Our new method and extended datasets are available at: https://github.com/IDT-ITI/SD-MVSum.

Figures & tables

Explore similar work

CardsList
  1. TripleSumm: Adaptive Triple-Modality Fusion for Video Summarization

    Mar 1, 2026Sumin Kim, Hyemin Jeong, Mingu Kang +3Video SummarizationMultimodal Fusion

  2. Cut to the Chase: Training-free Multimodal Summarization via Chain-of-Events

    Mar 6, 2026Xiaoxing You, Qiang Huang, Lingyu Li +2Video SummarizationMultimodal Generation