Short-video platforms now host vast multimodal ads whose deceptive visuals, speech and subtitles demand finer-grained, policy-driven moderation than community safety filters. We present BLM-Guard, a content-audit framework for commercial ads that fuses Chain-of-Thought reasoning with rule-based policy principles and a critic-guided reward. A rule-driven ICoT data-synthesis pipeline jump-starts training by generating structured scene descriptions, reasoning chains and labels, cutting annotation costs. Reinforcement learning then refines the model using a composite reward balancing causal coherence with policy adherence. A multitask architecture models intra-modal manipulations (e.g., exaggerated imagery) and cross-modal mismatches (e.g., subtitle-speech drift), boosting robustness. Experiments on real short-video ads show BLM-Guard surpasses strong baselines in accuracy, consistency and generalization.
Figures & tables
Figure 1: BLM-Guard Benchmark Taxonomy. Our benchmark organizes commercial short-video ads into a hierarchical risk taxonomy with seven core violation scenarios and fine-grained subtypes. Each node reflects a policy-sensitive violation type (e.g., income exaggeration, privacy leak, feudal superstition), and is further associated with a severity level (high, medium, low). This structure supports interpretable supervision and enables fine-grained evaluation of model performance on diverse and nuanced moderation cases. A dedicated “No Risk” category is also included to balance risk distribution.
Figure 2: Our method adopts a progressive two-stage pipeline for policy-controllable content moderation. In Stage 1 (Rule-driven SFT Cold Start), we synthesize structured visual-language Chain-of-Thought (ICoT) data via keyframe selection and multi-step prompting using InternVL. This enables supervised fine-tuning with rule-anchored causal supervision. In Stage 2 (Self-adaptive GRPO Reinforcement Learning), we apply a safety-aware data curation strategy and propose a Self-Adaptive Critique Reward (SACR) to dynamically evaluate reasoning outputs. The model is optimized using a modified Group-wise Relative Policy Optimization (GRPO) algorithm with token-level normalization and dynamic sampling. This multi-stage process enables the model to first learn fine-grained compliance reasoning and then refine its moderation behavior through adaptive, reward-driven training.
Algorithm 1 BLM-Guard: Self-Adaptive GRPO Training
Figure 3: Comprehensive comparison of model strategies and components. Left : Radar plots highlight improvements in accuracy, precision, and consistency across key metrics. Middle : Heatmaps visualize performance across fine-grained risk scenarios and external benchmarks. Right : Histograms show the distributional gains in consistency induced by Rule-SFT and SCA-R reward learning.
Video content moderation detects and explains harmful, misleading, or policy-violating material in videos, including its visual, audio, and temporal context.
Rigorous content moderation is crucial for online advertising but leads to millions of daily rejections. This scale renders manual rectification infeasible, particularly for video advertisements. However, existing safety-driven methods often suffer from aggressive over-editing, which compromises the advertiser's original semantic intent merely to satisfy compliance. In this work, we target the rectification of textual violations in video ads, covering both speech transcripts and on-screen text. We propose R^3, a novel framework designed to harmonize compliance with original semantic intent preservation. Our approach integrates three key innovations: (1) an experience-driven data synthesis framework that bootstraps high-quality supervision via a group-Relative compliance experience extractor; (2) a curriculum Reinforcement learning strategy with hierarchical rewards designed to enforce compliance while maximizing semantic consistency; and (3) a comprehensive video Rectification framework seamlessly integrating text recognition, rewriting, and re-rendering for industrial deployment. Extensive experiments on industrial datasets and online A/B testing demonstrate that R^3 significantly outperforms state-of-the-art baselines, achieving an optimal trade-off between violation rectification and intent preservation.
Global-scale video moderation faces a dual challenge: the need for fine-grained multi-modal reasoning and the demand for interpretable outputs to support downstream enforcement. Traditional moderation systems often rely on fragmented black-box classifiers that are difficult to maintain and lack transparency. In this paper, we present UNIVID, a UNIfied VIsion-language model for video moDeration. Unlike standard classification models, UNIVID generates policy-aware captions that serve as an interpretable intermediate representation, enabling human-verifiable decisions and multi-task reusability. While existing open-source and commercial VLMs often suffer from safety-guardrail refusals and lack fine-grained policy alignment, we develop a specialized training data recipe that combines expert human-refined labels with synthetic data to align the model with our safety guidelines. By integrating UNIVID as the core captioner, we design a novel end-to-end video moderation system that reduces violation leakage by 42.7% and overkill rate by 37.0% relatively. Meanwhile, by replacing over 1,000 policy-specific models with a single UNIVID backbone, we recycled extensive computation resources while reducing engineering maintenance overhead. To our knowledge, this is one of the first reports of a high-efficiency captioning VLM successfully supporting industrial-scale moderation and cross-functional business.
The growing complexity of content moderation policies presents a critical challenge for their consistent operationalization. While foundation models possess the basic capabilities needed to confront this challenge, whether they can reliably moderate online content remains an unanswered question. In this paper, we systematically compare two competing paradigms for Vision-Language Model (VLM) guidance: an instruction-driven approach where models reason from policy precepts, and an example-driven approach where they generalize from prior precedents. We ground this investigation in ModerationBench, a new benchmark of 4,000 manually annotated, in-the-wild posts from the Bluesky platform. Our experiments reveal that foundation models can substantially outperform Bluesky's deployed moderation system, nearly tripling its F1 score (0.60 vs. 0.22) on Random Posts in the benchmark, with both instruction- and example-driven paradigms achieving comparable peak effectiveness. Our findings thus chart a path toward reliable and adaptable policy operationalization at scale.
Ayan Majumdar, Shounak Paul, Pushpdeep Singh +6
MPI-SWS, Germany · Saarland University, Germany · INRIA, France