KidsNanny: A Two-Stage Multimodal Content Moderation Pipeline Integrating Visual Classification, Object Detection, OCR, and Contextual Reasoning for Child Safety

Viraj Panchal; Tanmay Talsaniya; Parag Patel; Meet Patel

cs.CVMar 17, 2026

KidsNanny: A Two-Stage Multimodal Content Moderation Pipeline Integrating Visual Classification, Object Detection, OCR, and Contextual Reasoning for Child Safety

Authors: Viraj Panchal, Tanmay Talsaniya, Parag Patel, Meet Patel

Organizations: KidsNanny Research Team, Vartit Technology Inc.

Abstract

We present KidsNanny, a two-stage multimodal content moderation architecture for child safety. Stage 1 combines a vision transformer (ViT) with an object detector for visual screening (11.7 ms); outputs are routed as text not raw pixels to Stage 2, which applies OCR and a text based 7B language model for contextual reasoning (120 ms total pipeline). We evaluate on the UnsafeBench Sexual category (1,054 images) under two regimes: vision-only, isolating Stage 1, and multimodal, evaluating the full Stage 1+2 pipeline. Stage 1 achieves 80.27% accuracy and 85.39% F1 at 11.7 ms; vision-only baselines range from 59.01% to 77.04% accuracy. The full pipeline achieves 81.40% accuracy and 86.16% F1 at 120 ms, compared to ShieldGemma-2 (64.80% accuracy, 1,136 ms) and LlavaGuard (80.36% accuracy, 4,138 ms). To evaluate text-awareness, we filter two subsets: a text+visual subset (257 images) and a text-only subset (44 images where safety depends primarily on embedded text). On text-only images, KidsNanny achieves 100% recall (25/25 positives; small sample) and 75.76% precision; ShieldGemma-2 achieves 84% recall and 60% precision at 1,136 ms. Results suggest that dedicated OCR-based reasoning may offer recall-precision advantages on text-embedded threats at lower latency, though the small text-only subset limits generalizability. By documenting this architecture and evaluation methodology, we aim to contribute to the broader research effort on efficient multimodal content moderation for child safety.

Figures & tables

Figure 1: KidsNanny two-stage architecture. Stage 1 performs visual screening using a ViT classifier and object detector (11.7 ms). Stage 2 extracts text via OCR and passes extracted text + object labels (not raw pixels) to a 7B text-based language model for contextual reasoning (120 ms total pipeline).

Component	Architecture	Input	Output
Image Classifier	ViT-based backbone	RGB image ( $448\times 448$ )	Safety probability $\in[0,1]$
Object Detector	CNN-based detector	RGB image ( $512\times 512$ )	Bounding boxes + class labels + confidence

Table 1: Stage 1 Components — Visual Screening (11.7 ms per image).

Component	Architecture	Input	Output
OCR Engine	Dedicated OCR module	RGB image (original resolution)	Extracted text with spatial coordinates
Contextual Reasoning LLM	7B LLM	Text: object labels + extracted text	Verdict (Safe/Unsafe) + explanation

Table 2: Stage 2 Components — Multimodal Analysis (120 ms total pipeline).

Capability	NudeNet	FalconsAI	Adam-ViT	Freepik	ShieldGemma-2	LlavaGuard	KidsNanny
Image classification	$\bullet$	$\bullet$	$\bullet$	$\bullet$	$\bullet$	$\bullet$	$\bullet$
NSFW object detection	$\bullet$ *	$\times$	$\times$	$\times$	$\times$	$\times$	$\bullet$
Dedicated OCR pipeline	$\times$	$\times$	$\times$	$\times$	Partial †	Partial †	$\bullet$
Text safety analysis	$\times$	$\times$	$\times$	$\times$	Partial †	Partial †	$\bullet$
Explicit reasoning output	$\times$	$\times$	$\times$	$\times$	$\times$	$\bullet$	$\bullet$
Inference time (ms)	35	7.3	7.2	15.5	1,136	4,138	11.7 / 120 ‡

Table 3: Functional capability comparison across evaluated models.

\bullet

= evaluated in this paper; Partial = incidental capability, not a dedicated pipeline;

\times

= not supported.

Model / Configuration	Accuracy (%)	F1 (%)	Precision (%)	Recall (%)	Inference (ms)
Regime 1 — Vision-Only (Stage 1 active; OCR and LLM disabled)
FalconsAI	59.01	56.28	91.15	40.70	7.3
NudeNet	68.03	76.01	73.96	78.18	35
Adam-ViT	68.98	73.90	81.23	67.79	7.2
Freepik	77.04	81.12	86.81	76.13	15.5
KidsNanny Stage 1	80.27	85.39	82.05	89.02	11.7

Table 4: Comprehensive evaluation results on UnsafeBench Sexual category (1,054 images). Bold = best in group.

\Delta

row shows Stage 2 net contribution over Stage 1.

Model	Accuracy (%)	F1 (%)	Precision (%)	Recall (%)	Inference (ms)
Text+Visual subset (257 images — visual content with embedded text)
ShieldGemma-2	54.86	51.67	72.94	40.00	1136
LlavaGuard	76.45	79.18	84.06	74.84	4138
KidsNanny (Stage 1+2)	81.08	85.11	80.46	90.32	115
Text-Only subset (44 images — safety depends primarily on embedded text)
ShieldGemma-2	59.09	70.00	60.00	84.00	1136

Table 5: Text-subset evaluation (Regime 2 models only). Top: Text+Visual (257 images, 155 unsafe, 102 safe). Bottom: Text-Only (44 images, 25 unsafe, 19 safe).

Panel of Figure 2 — Figure 2: Model performance visualizations. Diamond markers = multimodal models; circles = vision-only models. (a) KidsNanny (Stage 1+2) leads all models in accuracy (81.40%) and F1 (86.16%). (b) KidsNanny occupies the balanced deployment-viable zone; ShieldGemma-2 achieves the highest recall but at low precision (64.96%). (c) KidsNanny Stage 1 (11.7 ms) lies on the Pareto frontier; Stage 1+2 (120 ms) is the overall accuracy leader at 9–34 $\times$ lower latency than competing VLMs.

Dimension	KidsNanny	ShieldGemma-2	LlavaGuard
Primary focus	Balanced (FN+FP)	Recall maximisation	Balanced accuracy
Recall (full)	89.31%	93.56% (highest)	83.02%
Recall (text-only)	100.00%	84.00%	56.00%
Precision	83.22%	64.96%	86.17%
Inference speed	120 ms	1,136 ms	4,138 ms
Context input	ViT + Object + OCR	Vision encoder ( incidental text)	Vision encoder

Table 6: Safety philosophy and trade-off comparison (multimodal regime).

Multimodal Content Moderation

See all 21 papers

Nemotron 3.5 Content Safety Moderator: A Compact Multimodal, Multilingual, and Reasoning Enabled Content Safety Moderator
Aug 27, 2026Varun Singh, Anuj Doshi, Makesh Narsimhan Sreedhar +2Multimodal Content Moderation LLM Guardrails
NVIDIA Santa Clara, CA
Old Tricks, New Models: How Simple Image Transformations Break Modern AI-based Content Moderation
Jul 30, 2026Marco Alecci, Francesco Marchiori, Iyiola Emmanuel Olatunji +2Multimodal Robustness Adversarial Attacks
Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Luxembourg · Department of Mathematics, University of Padua, Padua, Italy
Shieldstral
Jul 28, 2026Antonia Calvi, Avinash Sooriyarachchi, Giada Pistilli +273Language Model Safety Evaluation Content Moderation

Aug 27, 2026J/K move · Enter open · S save

Nemotron 3.5 Content Safety Moderator: A Compact Multimodal, Multilingual, and Reasoning Enabled Content Safety Moderator

Varun Singh, Anuj Doshi, Makesh Narsimhan Sreedhar, Shaona Ghosh+1

NVIDIA Santa Clara, CA

Multimodal Content Moderation LLM Guardrails MLLM Safety LLM Safety Monitoring LLM Safety Language Model Safety Evaluation+1

Open paper AskPDF arXiv

Safety moderation for deployed AI applications is moving beyond text-only prompts: systems increasingly need to judge images, documents, screenshots, and generated responses under policies that vary across domains. Existing guardrails usually cover only part of this setting, making it difficult to combine broad coverage, custom policy control, and low compute cost. We present Nemotron 3.5 Content Safety Moderator, also referred to as Nemotron 3.5 CS in this paper for brevity, a compact 4B vision-language safety moderator that jointly classifies user prompts, images, and assistant responses across 12 languages. Nemotron 3.5 CS returns safety labels for latency-sensitive moderation and can additionally produce concise reasoning traces that apply supplied custom policies and identify violated categories when reasoning is requested. We also release a multimodal and multilingual safety dataset for guard training, spanning human-labeled real-image moderation, benign vision-language and document tasks, synthetic rare-risk and jailbreak cases, and custom-policy examples. Across evaluations spanning multimodal safety, text moderation, multilingual robustness, custom-policy following, benign false positives, and latency, Nemotron 3.5 CS demonstrates a practical coverage tradeoff: it adds image-conditioned and policy-conditioned moderation while remaining broadly competitive with specialized guard models. These results suggest that compact vision-language moderators can serve as deployable front-line safety components, with reasoning used selectively for audit and policy review.

Abstract

Multimodal Content Moderation

Explore similar work

Nemotron 3.5 Content Safety Moderator: A Compact Multimodal, Multilingual, and Reasoning Enabled Content Safety Moderator

Old Tricks, New Models: How Simple Image Transformations Break Modern AI-based Content Moderation

Shieldstral