ReDence: A Hybrid Agentic and Decision-Model Framework for Rapid, Evidence-Grounded Detection of Modern Social Engineering Scams
Organizations: Louisiana State University
Abstract
Detecting online social engineering (SE) scams often requires examining website workflows, verifying claims against external sources, and connecting evidence across multiple pages. Existing detectors provide limited support for these investigations, leaving analysts to manually gather and validate evidence, which inevitably delays detection and subsequent takedown of these attacks. To address this gap, we present ReDence, an open-source hybrid framework for adaptive, evidence-grounded detection across diverse SE scam categories. ReDence combines two locally hosted models with complementary responsibilities: a fine-tuned multimodal agentic Investigator (Qwen3.5-4B) that performs investigative planning, evidence interpretation, and reporting, and a specialized Decider (Tev1) that handles binary and multiple-choice tasks, including browser-action selection, evidence assessment, and decisions about whether to continue an investigation. By delegating discrete decisions to a specialized model, ReDence reduces reliance on lengthy language-model responses, while preserving the reasoning capabilities required for complex investigations. Our evaluation demonstrates that ReDence achieves 96% accuracy in detecting scams across twelve SE scam categories, substantially outperforming existing detectors while being at least four times faster than comparable agentic models without compromising investigative depth. ReDence also produces structured, evidence-grounded reports that document investigative actions, supporting and contradictory evidence, and collected artifacts, which allow analysts to independently audit detection decisions and utilize the collected evidence to support subsequent mitigation and takedown efforts.
Figures & tables
| SE Scam Category | Ground Truth | Test Set |
|---|---|---|
| Ground-truth Categories | ||
| Online Purchase | 587 | 637 |
| Counterfeit Products | 109 | 203 |
| Advance-Fee Loans | 30 | 51 |
| Investment | 26 | 59 |
| Debt Collection | 37 | 53 |
| Category | Components |
|---|---|
| Navigation | Header, footer, menus, links, and redirects |
| Identity | Branding, logos, and claimed organization |
| Content | Main text, offers, representations, products, and services |
| User interaction | Forms, requested information, buttons, registration, login, and account access |
| Organization details | Contact information, policies, legal pages, and company information |
| Transactions | Payment methods and payment-related claims |
| Task | Investigator Qwen3.5-4B | Decider Tev1-4B |
|---|---|---|
| Open-ended reasoning and generation | Binary and multiple-choice decisions | |
| Planning | Identify the website’s purpose, define objectives, and formulate an investigative plan. | Choose among candidate actions addressing the current objective. |
| External verification | Identify claims requiring verification and generate search queries. | Select search results or external sources for retrieval. |
| Browser interaction | Define the interaction’s objective and supply required input values. | Select links, buttons, form options, or other available browser actions. |
| Evidence assessment | Organize observations, interpret relationships, and formulate evidence questions. | Resolve explicit binary or multiple-choice questions about the available evidence. |
| Adaptive follow-up | Identify unresolved questions and formulate revisions to the plan. | Choose whether to continue, revise, or conclude the investigation. |
| Configuration | Accuracy | Precision | Recall | F1 | Time (Median, s) | GPU Usage (Median, GB) |
|---|---|---|---|---|---|---|
| ReDence (4B) | 0.96 (0.95) | 0.97 (0.95) | 0.98 (0.98) | 0.97 (0.96) | 43.8 (27.2) | 3.4GB |
| Mentor (27B) | 0.82 (0.82) | 0.83 (0.82) | 0.81 (0.83) | 0.82 (0.82) | 194.0 (151.2) | 33.8GB |
| URLNet | 0.41 | 0.35 | 0.19 | 0.25 | 1.2 (1.2) | 1.5GB |
| BeyondPhish | 0.72 | 0.87 | 0.70 | 0.78 | 53.7 (1.8) | 3.7GB |
| ScamFerret (GPT 4.1) | 0.79 | 0.84 | 0.81 | 0.82 | 88.5 (41.2) | API-bound |
| ScamFerret (Llama3.3:70B) | 0.71 | 0.75 | 0.67 | 0.71 | 127.4 (92.7) | 45.7GB |