Complementing Text Based Moderation with Multimodal Grounding: Detecting Fraudulent Inducement Behaviors on Short Video Platforms

Xiaoyong Yu

cs.CVOct 28, 2026

Complementing Text Based Moderation with Multimodal Grounding: Detecting Fraudulent Inducement Behaviors on Short Video Platforms

Authors: Xiaoyong Yu

Organizations: Huazhong University of Science and Technology, Wuhan, China

Abstract

Short video platforms have become a primary channel for black and gray industry operations in internet finance, where inducements such as prompts to like, follow, join private chat groups, or click external links frequently steer users toward fraud. Because such videos are short, dynamic, and often convey intent through fleeting visual cues rather than stable text, a moderation system must guarantee detection recall and localize the offending segment so reviewers inspect only the relevant clip. Existing OCR+LLM pipelines transcribe on-screen text frame by frame and classify the transcript, but exploit only textual information. Many inducements are conveyed through visual cues OCR cannot observe, and keyframe-based localization cannot recover precise temporal boundaries. We explore OCR+MLLM, which augments the text pipeline with a Multimodal Large Language Model, and compare three routes: OCR+LLM, OCR+MLLM, and a large zero-shot MLLM, along detection, latency, token consumption, and deployment complexity. OCR preserves the text signal that guarantees recall, while the MLLM jointly performs OCR-grounded discrimination and temporal grounding, localizing both textual and non-textual inducements. We construct a dataset of 2,419 internet finance short videos (66.49 hours) with 3,194 annotated events across four categories, and study reward design for RL-based grounding under class imbalance. We report two findings: auxiliary reward terms are exploited through reward hacking, and stronger localization rewards degrade class-balanced detection. We therefore design a recall-weighted F2 localization reward with class-balanced resampling. Experiments show OCR+MLLM localizes better than the keyframe OCR+LLM baseline (mIoU 26.52 vs. 14.49) and detects non-textual inducements that text-only pipelines miss, while maintaining 67.0 percent macro recall.

Figures & tables

Figure 1. Comparison of the OCR+LLM moderation pipeline and the OCR+MLLM moderation framework.

ID	Class	Description
0	Simple interaction	Likes, favorites, shares, follows, @ mentions, screenshot saving
1	Complex interaction	Comments, DMs, live stream interaction, one on one consultation
2	Private conversion	WeChat/QQ group ID or QR code, fan group joins, memberships
3	Page transition	Clicking bottom links, installing apps, searching software

Table 1. Fine grained inducement behavior taxonomy.

Figure 2. Overview of the RL post training pipeline for the grounding model.

Class	Train #Events	Train #Videos	Test #Events	Test #Videos
0 Simple interaction	1,258	1,136	613	527
1 Complex interaction	435	414	223	208
2 Private conversion	135	61	35	31
3 Page transition	382	266	113	84

Table 2. Class distribution of the training and test sets (event level and video level).

Stat.	Train Ev.	Test Ev.	Test Dur. (s)
Min	1	1	11.1
Max	14	6	532.0
Mean	1.30	1.36	106.3

Table 3. Event density and duration statistics.

Method	Macro Recall	Macro F1	mIoU	[email protected]	Non text. Rec.
OCR+LLM (keyframe)	69.2%	54.9%	14.49	4.41	$\approx$ 0
OCR+MLLM (RL only)	55.0%	46.5%	26.52	15.86	41.7%
OCR+MLLM (SFT+RL)	67.0%	45.6%	14.97	8.68	60.1%
Large MLLM (32B, zero shot)	36.7%	34.0%	4.72	1.02	–

Table 4. Comparison of the three moderation routes on the rebalanced test set.

Localization reward	mIoU	[email protected]	$R_{1}$	$R_{2}$	$R_{3}$
$\text{IoU}\times$ class scaling	24.84	15.23	0.53	0.07	0.51
soft IoU	14.57	1.22	0.29	0.08	0.53
Per event soft F1 $+$ balance	29.82	18.54	0.50	1.00	0.64
Per event soft F 2 $+$ balance (recall first)	26.52	15.86	0.72	0.46	0.53

Table 5. Reward design ablation.

Aspect	Large MLLM	OCR+LLM
Visual tokens per video	${\sim}14$ K	${\sim}450$ K
Forward passes	1	${\sim}7$ (OCR)
Time per video	${\sim}1.9$ min	${\sim}1.7$ – $2.6$ min
Models / calls / pipeline	1 / 1 / single	2 / 2 / two stage

Table 6. Efficiency and deployment comparison (single H100).

Video Content Moderation

Video content moderation detects and explains harmful, misleading, or policy-violating material in videos, including its visual, audio, and temporal context.

See all 24 papers

Feb 20, 2026

BLM-Guard: Explainable Multimodal Ad Moderation with Chain-of-Thought and Policy-Aligned Rewards

Short-video platforms now host vast multimodal ads whose deceptive visuals, speech and subtitles demand finer-grained, policy-driven moderation than community safety filters. We present BLM-Guard, a content-audit framework for commercial ads that fuses Chain-of-Thought reasoning with rule-based policy principles and a critic-guided reward. A rule-driven ICoT data-synthesis pipeline jump-starts training by generating structured scene descriptions, reasoning chains and labels, cutting annotation costs. Reinforcement learning then refines the model using a composite reward balancing causal coherence with policy adherence. A multitask architecture models intra-modal manipulations (e.g., exaggerated imagery) and cross-modal mismatches (e.g., subtitle-speech drift), boosting robustness. Experiments on real short-video ads show BLM-Guard surpasses strong baselines in accuracy, consistency and generalization.

Yiran Yang, Zhaowei Liu, Yuan Yuan +10

1Kuaishou Technology · 2Beijing University of Posts and Telecommunications · 3Shandong University

Video Content Moderation Content Moderation Multimodal Content Moderation

Ask PDF

Jun 4, 2026

UNIVID: Unified Vision-Language Model for Video Moderation

Global-scale video moderation faces a dual challenge: the need for fine-grained multi-modal reasoning and the demand for interpretable outputs to support downstream enforcement. Traditional moderation systems often rely on fragmented black-box classifiers that are difficult to maintain and lack transparency. In this paper, we present UNIVID, a UNIfied VIsion-language model for video moDeration. Unlike standard classification models, UNIVID generates policy-aware captions that serve as an interpretable intermediate representation, enabling human-verifiable decisions and multi-task reusability. While existing open-source and commercial VLMs often suffer from safety-guardrail refusals and lack fine-grained policy alignment, we develop a specialized training data recipe that combines expert human-refined labels with synthetic data to align the model with our safety guidelines. By integrating UNIVID as the core captioner, we design a novel end-to-end video moderation system that reduces violation leakage by 42.7% and overkill rate by 37.0% relatively. Meanwhile, by replacing over 1,000 policy-specific models with a single UNIVID backbone, we recycled extensive computation resources while reducing engineering maintenance overhead. To our knowledge, this is one of the first reports of a high-efficiency captioning VLM successfully supporting industrial-scale moderation and cross-functional business.

Kejuan Yang, Yizhuo Zhang, Mingyuan Du +6

Bytedance

Unified Multimodal Models VLM Adaptation Video Content Moderation Video Captioning Content Moderation Multimodal Content Moderation

Ask PDF

Mar 16, 2026

Exposing Cross-Modal Consistency for Fake News Detection in Short-Form Videos

Short-form video platforms are major channels for news but also fertile ground for multimodal misinformation where each modality appears plausible alone yet cross-modal relationships are subtly inconsistent, like mismatched visuals and captions. On two benchmark datasets, FakeSV (Chinese) and FakeTT (English), we observe a clear asymmetry: real videos exhibit high text-visual but moderate text-audio consistency, while fake videos show the opposite pattern. Moreover, a single global consistency score forms an interpretable axis along which fake probability and prediction errors vary smoothly. Motivated by these observations, we present MAGIC3 (Modal-Adversarial Gated Interaction and Consistency-Centric Classifier), a detector that explicitly models and exposes cross-tri-modal consistency signals at multiple granularities. MAGIC3 combines explicit pairwise and global consistency modeling with token- and frame-level consistency signals derived from cross-modal attention, incorporates multi-style LLM rewrites to obtain style-robust text representations, and employs an uncertainty-aware classifier for selective VLM routing. Using pre-extracted features, MAGIC3 consistently outperforms the strongest non-VLM baselines on FakeSV and FakeTT. While matching VLM-level accuracy, the two-stage system achieves 18-27x higher throughput and 93% VRAM savings, offering a strong cost-performance tradeoff.

Chong Tian, Yu Wang, Chenxu Yang +5

Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · Institute of Information Engineering, Chinese Academy of Sciences (CAS)

Fake News Detection Multimodal Fusion Cross-Modal Learning

Ask PDF

Abstract

Video Content Moderation

Explore similar work

BLM-Guard: Explainable Multimodal Ad Moderation with Chain-of-Thought and Policy-Aligned Rewards

UNIVID: Unified Vision-Language Model for Video Moderation

Exposing Cross-Modal Consistency for Fake News Detection in Short-Form Videos