SafeScreen: A Safety-First Screening Framework for Personalized Video Retrieval for Vulnerable Users
Organizations: Worcester Polytechnic Institute, Worcester, MA, USA
Abstract
Open-domain video platforms offer rich, personalized content that could support health, caregiving, and educational applications, but their engagement-optimized recommendation algorithms can expose vulnerable users to inappropriate or harmful material. These risks are especially acute in child-directed and care settings (e.g., dementia care), where content must satisfy individualized safety constraints before being shown. We introduce SafeScreen, a safety-first video screening framework that retrieves and presents personalized video while enforcing individualized safety constraints. Rather than ranking videos by relevance or popularity, SafeScreen treats safety as a prerequisite and performs sequential approval or rejection of candidate videos through an automated pipeline. SafeScreen integrates three key components: (i) profile-driven extraction of individualized safety criteria, (ii) evidence-grounded assessments via adaptive question generation and multimodal VideoRAG analysis, and (iii) LLM-based decision-making that verifies safety, appropriateness, and relevance before content exposure. This design enables explainable, real-time screening of uncurated video repositories without relying on precomputed safety labels. We evaluate SafeScreen in a dementia-care reminiscence case study using 30 synthetic patient profiles and 90 test queries. Results demonstrate that SafeScreen prioritizes safety over engagement, diverging from YouTube's engagement-optimized rankings in 80-93% of cases, while maintaining high levels of safety coverage, sensibleness, and groundedness, as validated by both LLM-based evaluation and domain experts.
Figures & tables
| Profile Component | Data Elements |
|---|---|
| Demographics | Age, diagnosis, cognitive stage (mild/moderate/severe) |
| Personal History | Past occupation, era preferences, cultural background |
| Interests | Topics of engagement (e.g., music, nature, sports) |
| Sensitivities | Auditory triggers (sirens, loud noises), visual triggers (medical imagery, flashing lights), content triggers (loss themes, emergency situations) |
| Cognitive Characteristics | Attention span, complexity tolerance, communication needs, preferred pacing |
| Engagement History | Previously successful content, previously distressing content |
| Component | Input | Output |
|---|---|---|
| Risk Detection | User query, population type, cognitive level | Risk level (LOW/MEDIUM/HIGH), permission required (yes/no), reasoning |
| Profile Extraction | User profile, query, context | Safety constraints (triggers to avoid), engagement factors (interests, era), appropriateness requirements (complexity, pacing), relevance criteria |
| Candidate Retrieval | Refined query | 3-5 candidate videos with metadata (ID, title, URL, duration, channel) |
| Component | Input | Output |
|---|---|---|
| Question Generation | Extracted criteria, user query | 5-6 adaptive safety questions with purpose labels (safety check, relevance, appropriateness) |
| VideoRAG Analysis | Generated questions, video segment (first 5 min) | Answers with detailed observations, evidence timestamps (MM:SS-MM:SS), confidence levels (high/medium/low) |
| Component | Input | Output |
|---|---|---|
| Safety Evaluation | VideoRAG Q/A pairs, patient profile | Decision (APPROVE/REJECT), confidence level, safety verification results per category (auditory, visual, cognitive, relevance), therapeutic value assessment |
| Sequential Screening | Evaluation results | If APPROVE: selected video with evidence summary, personalization notes, caregiver/guardian guidance. If REJECT: proceed to next candidate |
| Scenario | Level 1 | Level 2 | Level 3 |
|---|---|---|---|
| Single-video cases (first video passed) | |||
| Match with YouTube top result | 23.1% | 25.0% | 11.1% |
| Overall | 20.6% | ||
| Multi-video cases (2-3 videos screened) | |||
| Match with YouTube top result | 0% | 0% | 14.3% |
| Overall | 7.1% | ||
| Metric | Score (1-5 scale) |
|---|---|
| Safety Coverage | 4.26 1.09 |
| Sensibleness | 4.42 0.95 |
| Groundedness | 4.26 1.16 |
| Metric | Expert Mean | Weighed Cohen’s |
|---|---|---|
| Safety Coverage | 4.5 | 0.88 |
| Sensibleness | 3.5 | 0.92 |
| Groundedness | 4 | 0.94 |
| Overall Agreement | Substantial |