DarKnight: Retrieval-Guided Agentic Framework for Breaking Dark Web CAPTCHA Gates

Binghui Wu; Yeo Zi Yi; Darren Tan; Manit Baser; Levente Csikor; Dinil Mon Divakaran

cs.CROct 14, 2026

DarKnight: Retrieval-Guided Agentic Framework for Breaking Dark Web CAPTCHA Gates

Authors: Binghui Wu, Yeo Zi Yi, Darren Tan, Manit Baser, Levente Csikor, Dinil Mon Divakaran

Organizations: National University of Singapore Singapore · A*STAR Institute of Advanced Intelligence and Compute Singapore

Abstract

Cyber Threat Intelligence (CTI) investigators who monitor dark web marketplaces for leaked credentials and breach listings must pass the diverse CAPTCHAs that operators deploy to block automated collection. Existing solvers either need unscalable per-site fine-tuning or rely on commercial Vision-Language Model (VLM) APIs, which surrender operational sovereignty: provider policies, Tor-exit blocking, government regulation, or model updates outside the investigator's control can halt or alter the solver overnight, and per-call costs recur with every session and retry. We present DarKnight, a self-improving agentic framework designed to run on a single workstation GPU inside an investigator's Tor crawler and solve CAPTCHAs with small, locally hosted open-weight VLMs that no provider decision can withdraw, without per-market training or third-party services. For each CAPTCHA, DarKnight retrieves successful and failed past attempts on similar images (dual-signal RAG), composes an image-specific preprocessing pipeline, and learns from the site's pass/fail verdict. In a case study on four real-world dark web marketplaces, agentic tool selection outperformed parameter scaling. A deployable open-weight VLM with DarKnight reached 83.2% on the most heavily defended market, 37.2 percentage points above its base model, outperforming even the frontier Claude Opus~5 (44%). DarKnight recovered 48.2% of baseline failures at a 7.6% degeneration cost, indicating that selective tool use can avoid much of the degradation caused by uniform preprocessing.

Figures & tables

Fig. 2 : Architecture Diagram of the pipeline

Features	Values
captcha_type	alphanumeric $\|$ math $\|$ chinese
char_count	Integer count of CAPTCHA characters
noise	none $\|$ color_lines : multiple coloured wavy/diagonal lines of a different hue from the text $\|$ strikethrough : one thin dark line crossing all characters $\|$ arc : dark curved or arc-shaped bands $\|$ vertical_lines : thin dark vertical strokes through characters $\|$ dots : scattered isolated dot clusters $\|$ salt_pepper : fine pixel-level random grain
background	clean $\|$ gradient $\|$ noisy $\|$ patterned
distortion	none $\|$ wave $\|$ blur $\|$ perspective
char_quality	normal $\|$ hollow $\|$ faded $\|$ thin $\|$ broken $\|$ thick

TABLE I : Visual features extracted by Stage 1 visual analysis.

Slot	Content	Capacity	Eviction Policy
correct	Successful predictions	100 records	Least-retrieved first
failed	Failed predictions	100 records	Least-retrieved first

TABLE II : RAGStore slot structure.

Slot	Similarity	Action
Correct	$\geq 0.8$	Adopt pipeline if features match
	$0.6$ – $0.8$	Verify each feature before copying
	$<0.6$	Ignore entirely
Failed	$\geq 0.6$	Avoid same tools
	$0.5$ – $0.6$	Still a negative signal
	Repeated	Treat that tool as banned

TABLE III : Planner policy for retrieved RAG records.

Tool	Description and Usage Condition
color_extract	Removes colored interference lines by isolating non-color pixels. Use when wavy or diagonal lines of a distinct color cross the text region.
sharpen	Restores character edge crispness via unsharp masking. Use when blur distortion has softened character boundaries.
denoise	Removes small connected components below an area threshold. Use to clean residual noise dots or on patterned backgrounds.
remove_line	Detects and removes a single thin dark diagonal or near-horizontal line via the Hough transform. Use when such a line crosses all characters.
binarize	Converts a faded-grayscale image to black-and-white using Otsu’s global threshold. Use only when background = "clean" .

TABLE IV : Preprocessing tools and their intended use cases.

Panel of Figure 7 — Fig. 3: Effective use of preprocessing tools.

Panel of Figure 8 — Fig. 4: Effective vs. degenerative use of preprocessing tools.

Dataset	Type	Sample size	Background	Noise	Distortion	Char Quality	Difficulty	Dark web site
$\mathcal{D}_{1}$	Alpha / Math / Chinese	500	Mixed	colored lines	Wave	Mixed	High	ChangAn
$\mathcal{D}_{2}$	Alphanumeric	500	Clean	Strikethrough line	None	Normal	Low	Freecity
$\mathcal{D}_{3}$	Alphanumeric	500	Gradient	Dots/specks	Wave	Normal	Medium	Russian market
$\mathcal{D}_{4}$	Alphanumeric	500	Patterned	None (mesh texture)	None	Thick	Medium	Mobilestore

TABLE V : Summary of dataset visual properties.

Panel of Figure 10 — Fig. 5 : Representative samples from each dataset.

Model Name	Size	Mode	Domain	Date	Source
HunyuanOCR	1B	Inst	OCR	Nov 2025	Tencent
Gemma4-E4B-it	8B	Inst	General	Apr 2026	Google
Qwen3-vl-4B	4B	Inst	General	Oct 2025	Alibaba
Qwen3-vl-8B	8B	Inst	General	Oct 2025	Alibaba
Qwen3.5-9B	9B	Think	General	Mar 2026	Alibaba
Step3-VL-10B	10B	Think	General	Jan 2026	Stepfun

TABLE VI : Baseline models evaluation: Parameter size, type, domain, and development source.

Fig. 6 : Baseline vs. DarKnight : accuracy (%) across four benchmark datasets.

Tool	N	%	C $\to$ C	C $\to$ W	W $\to$ C	W $\to$ W	Degen%	Recov%	Net
(no tool)	4406	72.8	2357	159	733	1157	6.3	38.8	$+$ 32.5
color_extract	1216	20.1	371	58	508	279	13.5	64.5	$+$ 51.0
sharpen	197	3.3	66	3	90	38	4.3	70.3	$+$ 66.0
remove_straight_line	184	3.0	111	9	53	11	7.5	82.8	$+$ 75.3
denoise	43	0.7	17	12	6	8	41.4	42.9	$+$ 1.5
binarize	3	0.0	0	0	0	3	—	0.0	—

TABLE VII : Per-tool degeneration-recovery profile across all models and datasets. Tools ordered by net effect.

	$\mathcal{D}_{1}$	$\mathcal{D}_{2}$	$\mathcal{D}_{3}$	$\mathcal{D}_{4}$
No-preproc rate	26.6%	86.9%	97.0%	56.7%
Avg baseline acc	35.7%	67.7%	50.5%	70.6%
Avg $\Delta$ with DarKnight	+34.2	+21.5	+6.5	+9.5

TABLE VIII : Planner tool-selection behavior per dataset. No-preproc rate is the fraction of decisions where the Planner selects an empty pipeline.

Fig. 7 : Ablation of retrieval memory using Qwen3-VL-4B and Qwen3-VL-8B across

\mathcal{D}_{1}-\mathcal{D}_{4}

Model	Baseline	DarKnight	$\Delta$
Qwen3-VL-8B	65.0	75.0	+10.0
Gemma4-E4B	50.8	60.8	+10.0
Qwen3.5-9B	16.5	80.4	+63.9
Qwen3.6-27B	65.0	78.0	+13.0

TABLE IX : Accuracy (%) in the cross-site mixed-stream experiment.

		$\mathcal{D}_{1}$		$\mathcal{D}_{4}$		$\mathcal{D}_{2}$		$\mathcal{D}_{3}$
Model	Params	B	DK	B	DK	B	DK	B	DK
HunyuanOCR	1B	1.82	–	0.10	–	0.07	–	0.13	–
Gemma4-E4B-IT	8B	0.156	3.81	0.154	3.25	0.137	3.12	0.157	2.97
Qwen3-VL-4B-Instruct	4B	0.122	2.72	0.102	2.21	0.120	2.57	0.108	2.34
Qwen3-VL-8B-Instruct	8B	0.170	4.26	0.164	3.72	0.273	3.89	0.165	3.67
Qwen3.5-9B	9B	0.168	5.38	0.162	4.70	0.163	5.17	0.166	4.43

TABLE X : Mean end-to-end inference time (in seconds) per CAPTCHA across dark web datasets.

Model	Params	Avg B	Avg DK	Avg $\Delta$	$t_{\mathrm{DK}}$ (s)
Qwen3-VL-4B-Instruct	4B	29.9	54.8	$+$ 24.9	2.46
Qwen3-VL-8B-Instruct	8B	36.3	48.3	$+$ 12.0	3.89
Gemma4-E4B-IT	8B	50.8	59.9	$+$ 9.1	3.29
Qwen3.5-9B	9B	64.6	80.9	$+$ 16.3	4.92
Step3-VL-10B	10B	74.2	78.8	$+$ 4.6	52.75
Gemma4-12B-IT	12B	54.9	67.4	$+$ 12.6	4.43

TABLE XI : Cross-model accuracy and latency summary. Avg B and Avg DK denote the average raw-baseline and DarKnight accuracy across the four datasets; Avg

\Delta

denotes the incremental improvement introduced by DarKnight , and

t_{\mathrm{DK}}

denotes the average local inference time per CAPTCHA.

Dataset	Samples	ESA (%)	CER (%)	Resolution
$\mathcal{D}_{1}$	500	90.4	2.90	Consensus
$\mathcal{D}_{2}$	500	99.4	0.16	Consensus
$\mathcal{D}_{3}$	500	96.4	0.70	Consensus
$\mathcal{D}_{4}$	500	94.4	1.29	Consensus

TABLE XII : Inter-annotator agreement for CAPTCHA transcription. Agreement statistics were calculated before consensus resolution.

		$\mathcal{D}_{1}$			$\mathcal{D}_{2}$			$\mathcal{D}_{3}$			$\mathcal{D}_{4}$
Model	Params	B	DK	$\Delta$	B	DK	$\Delta$	B	DK	$\Delta$	B	DK	$\Delta$
HunyuanOCR	1B	12.4	–	–	61.4	–	–	45.7	–	–	90.0	–	–
Gemma4-E4B-IT	8B	5.8	46.8	+41.0	71.4	65.8	$-$ 5.6	51.5	56.5	+5.0	74.4	70.4	$-$ 4.0
Qwen3-VL-4B-Instruct	4B	28.0	55.6	+27.6	30.8	73.4	+42.6	26.8	35.3	+8.5	33.8	54.8	+21.0
Qwen3-VL-8B-Instruct	8B	39.6	66.6	+27.0	43.4	49.0	+5.6	33.4	38.3	+4.9	28.8	39.4	+10.6
Qwen3.5-9B	9B	36.6	73.4	+36.8	78.2	94.2	+16.0	58.1	63.4	+5.3	85.4	92.4	+7.0

TABLE XIII : Accuracy (%) on dark web CAPTCHA datasets. B = baseline (raw VLM), DK = DarKnight (agentic pipeline).

\Delta

= absolute change. Best agentic accuracy per dataset in bold . Models are ordered by size. HunyuanOCR is a pure OCR system that cannot be used as a Planner and therefore has no agentic results.

Panel of Figure 21 — Fig. 8 : Comparison with other baselines.

RAG Size	$\mathcal{D}_{1}$	$\mathcal{D}_{2}$
No RAG	52.8	51.0
10	57.2	53.0
20	56.8	73.2
50	56.0	73.2
80	57.2	64.8
100	57.8	73.4

TABLE XIV : Accuracy (%) for varying RAG memory size.

Parameter	visual_analysis & plan	run_ocr
max_tokens	2048	1024
temperature	0.0	0.1
Input	image+text+RAG	processed image+text

TABLE XV : API parameters for the two VLM calls per sample.

CAPTCHA Solving

See all 8 papers

Aug 28, 2026

Breaking Darknet CAPTCHAs with general purpose LLM

Our work evaluates the effectiveness of automated methods for solving CAPTCHA challenges commonly encountered in darknet environments. These CAPTCHAs are typically designed to operate without JavaScript, resulting in distinct characteristics compared to mainstream CAPTCHA systems. Our study considers three representative challenge types: open-circle localization, rotation-based alignment, and object-selection CAPTCHAs. The experiments reveal a systematic limitation of contemporary MLLMs: while they are generally capable of identifying relevant visual structures, they frequently struggle with precise spatial localization and geometric transformations. These deficiencies can be mitigated either through task reformulation or by augmenting the models with specialized image processing tools. These deficiencies can be mitigated by task reformulation or by equipping the model with specialized image-processing tools. We therefore propose a hybrid framework in which an MLLM serves as a high-level reasoning and orchestration layer while delegating geometric computations to deterministic algorithms via the Model Context Protocol (MCP). The resulting system achieves success rates above 90% across all evaluated CAPTCHA types and demonstrates that combining the complementary strengths of MLLMs and classical computer vision yields a more accurate and efficient solver than either approach alone.

Benjamin Fehrensen, Jens Hubler

Bern University of Applied Science (BFH)

CAPTCHA Solving Large Vision-Language Models Visual Spatial Reasoning LLM Tool Use

Ask PDF

Oct 9, 2026

CaptchaAgent: Verifiable Visual Reasoning for CAPTCHA Solving

Modern reasoning CAPTCHAs require fine-grained perception, compositional relation reasoning, and grounded browser interaction. End-to-end vision-language model agents remain brittle in this setting because a missed object, incorrect count, or confused relation can propagate directly to the final action. To address this problem, we present CaptchaAgent, a structured visual reasoning framework for interactive CAPTCHA solving. Instead of predicting an action directly from a screenshot, CaptchaAgent first constructs a visual intermediate representation (V-IR) that explicitly records entities, attributes, and spatial or topological relations. A symbol-only ASCII protocol further requires the parser to commit to a concrete object set and count. Building on this global representation, localized evidence refinement revisits ambiguous regions at higher effective resolution, while counterfactual consistency verification tests whether the resulting action remains stable under task-preserving transformations. Across three proprietary backbones on Open CaptchaWorld, CaptchaAgent improves Pass@1 by 21.3-25.8 percentage points over a commercially available browser agent baseline evaluated with the same backbones. On the separate offline Oedipus-data benchmark, it further improves average Pass@1 by 9.2-12.3 percentage points over the Oedipus solver across three matched backbones. The intermediate traces show that exact counting, continuous geometry, and ordered execution remain major sources of automation difficulty for the evaluated agents, providing practical guidance for designing more resilient compositional CAPTCHAs.

Shangquan Sun, Ruoyu Chen, Zhirui Liu +4

Nanyang Technological University, Singapore. · the School of Cyber Security, University of Chinese Academy of Sciences, China. · the School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, No. 66, Gongchang Road, Guangming District, Shenzhen, Guangdong 518107, P.R. China. +2

CAPTCHA Solving Tool-Using Vision-Language Agents Visually Grounded Reasoning Visual Spatial Reasoning

Ask PDF

May 19, 2026

CaptchaMind: Training CAPTCHA Solvers via Reinforcement Learning with Explicit Reasoning Supervision

CAPTCHAs are widely deployed as human verification mechanisms and frequently block intelligent agents from completing end-to-end automation in real-world web environments. Solving modern CAPTCHAs requires robust multi-step visual reasoning and interaction capabilities, yet training-based approaches have remained absent due to the lack of large-scale training data and process-level annotations. We introduce CaptchaBench, the first CAPTCHA benchmark designed to support large-scale training, comprising 16,000 programmatically generated samples across eight task categories with detailed region and process-level annotations. Systematic evaluation on CaptchaBench reveals that existing methods fail consistently on tasks requiring fine-grained visual detail capture and region-level comparison. We therefore present CaptchaMind, an RL-based solver trained with explicit reasoning process supervision, achieving 82.9% average success rate across eight tasks and 71.0% on real-world instances, substantially outperforming all existing methods without closed-source APIs.

Pengcheng Wang, Haoxiang Liu, Yang Dai +5

1Alibaba Group · 2Southern University of Science and Technology · 3Harbin Institute of Technology (Shenzhen)

CAPTCHA Solving Visual Reasoning

Ask PDF

Abstract

CAPTCHA Solving

Explore similar work

Breaking Darknet CAPTCHAs with general purpose LLM

CaptchaAgent: Verifiable Visual Reasoning for CAPTCHA Solving

CaptchaMind: Training CAPTCHA Solvers via Reinforcement Learning with Explicit Reasoning Supervision