MapSteal: Map-based Autonomous Knowledge Stealing Attack Against Large Language Models

Zhengao Li; Chuxu Zhang; Yushun Dong

cs.CLOct 19, 2026

MapSteal: Map-based Autonomous Knowledge Stealing Attack Against Large Language Models

Authors: Zhengao Li, Chuxu Zhang, Yushun Dong

Organizations: University of Connecticut, Storrs, CT, USA · Florida State University, Tallahassee, FL, USA

Abstract

Knowledge acquisition from black-box large language models (LLMs) offers a practical approach to adapting smaller student models. However, many existing methods rely on predefined queries or student feedback to guide acquisition, leaving autonomous expansion of knowledge targets an important challenge. We propose MapSteal, a student-independent framework for structured recursive knowledge acquisition. Starting from a small set of initial roots, MapSteal represents acquisition targets using structured states and recursively expands them through teacher-proposed directions. Question realization, proposal admission, and deduplication constrain the acquisition process, while root-aware scheduling organizes exploration under a fixed QA budget. The resulting question--answer pairs are used to adapt student models without requiring student feedback during acquisition. Experiments with Qwen3.5 students at 2B, 4B, and 9B scales show competitive downstream performance across six medical and cybersecurity benchmarks. On the 4B medical evaluation, Full MapSteal improves average accuracy by 1.17 percentage points over a simplified recursive acquisition variant. Cross-domain evaluations show that gains depend on the domain and evaluation panel. These results demonstrate the feasibility of structured, student-independent recursive acquisition while highlighting the distinction between generating acquisition targets and obtaining useful training supervision. Our code is available at \href{https://github.com/LabRAI/Mapsteal} {https://github.com/LabRAI/Mapsteal}.

Figures & tables

Figure 1: Overview of MapSteal. Frozen domain roots initialize a structured acquisition map. Root-level FAIR scheduling selects pending states; a deterministic realizer converts non-root states into questions. A black-box teacher returns an answer, explanation, and typed child-state proposals in one successful interaction. Admitted children re-enter the frontier, while collected QA supervision is used to adapt the student only after acquisition. Map edges indicate acquisition provenance rather than verified domain relations.

Operator	$O$	$F$	$C$	$T,D$
NewObject	New	Empty	Empty	New
NewFacet	Keep	New	Keep	New
ConditionDepth	Keep	Keep	New	New
NewTarget	Keep	Keep	Keep	New

Table 1: Field updates for proposed states. “Keep” means inherit the parent’s field; “New” means use the proposed value. Transitions assign new

T

and

D

		Medical			Cybersecurity
Student	Method	MedMCQA	MedQA	MedXpert	CyberMetric	SecEval	CTI-Bench
Qwen3.5-2B	Pristine	45.80	36.06	10.78	79.47	71.93	60.64
	EvoKD	$40.53\pm 8.09$	$41.19\pm 8.47$	$12.39\pm 1.57$	$71.00\pm 6.19$	$69.49\pm 6.25$	$52.73\pm 9.42$
	QEDKS	$43.27\pm 0.12$	$44.59\pm 0.81$	$\mathbf{14.88\pm 0.83}$	$79.49\pm 0.23$	$77.73\pm 1.46$	$\mathbf{65.48\pm 0.16}$
	MapSteal (Ours)	$\mathbf{47.87{\pm}1.03}$	$\mathbf{45.59{\pm}0.16}$	$14.65{\pm}0.19$	$\mathbf{79.70{\pm}0.14}$	$\mathbf{78.68{\pm}0.56}$	$62.41{\pm}1.07$
Qwen3.5-4B	Pristine	56.60	62.14	16.73	83.63	80.60	65.71

Table 2: Main results across selected medical and cybersecurity benchmarks. Results are reported as accuracy (%).

Variant	MedMCQA	MedQA	MedXpert	Avg	$\boldsymbol{\Delta}$
Full MapSteal	$\mathbf{62.27\pm 0.42}$	$\mathbf{66.69\pm 0.44}$	$\mathbf{17.63\pm 0.46}$	$\mathbf{48.86\pm 0.17}$	–
w/o CONDITION_DEPTH	$60.93\pm 0.31$	$66.20\pm 0.28$	$16.30\pm 0.14$	$47.81\pm 0.04$	$-1.05$
Simplified OF	$60.87\pm 0.31$	$65.28\pm 0.36$	$16.94\pm 0.32$	$47.69\pm 0.10$	$-1.17$

Table 3: Ablation study of MapSteal on medical benchmarks.

Figure 2: Acquisition depth and expansion operator distributions of Simplified OF and Full MapSteal. Both methods execute 460 acquisition questions from the same 35 roots. Operator counts exclude root executions.

Roots	MedMCQA	MedQA	MedXpert
1	$60.87\pm 1.14$	$66.25\pm 0.67$	$16.97\pm 0.29$
10	$61.47\pm 0.31$	$65.49\pm 0.39$	$17.36\pm 0.37$
20	$61.27\pm 0.50$	$66.35\pm 0.37$	$16.87\pm 0.36$
35	$\mathbf{62.27\pm 0.42}$	$\mathbf{66.69\pm 0.44}$	$\mathbf{17.63\pm 0.46}$

Table 4: Effect of initial acquisition root count.

Metric	OF	OFCTD
Mean trajectory depth	1.67	1.87
Distinct Object strings	231	147
Sequential novelty	0.262	0.178
Parent–child cosine	0.621	0.788
Near-duplicate rate (%)	8.5	30.8

Table 5: Descriptive acquisition statistics for Simplified OF and Full MapSteal. Semantic metrics are computed from executed acquisition questions.

Figure 3: Qualitative example of structured recursive knowledge acquisition. Starting from an acute appendicitis question, MapSteal expands into related directions involving complications, patient populations, and treatment constraints, followed by context-specific questions. The figure displays 10 selected nodes from a real 14-node executed acquisition tree. Questions are shown verbatim, and arrows represent recorded parent–child links. The complete tree is provided in Appendix C.2 .

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Value
LoRA rank	16
LoRA alpha	32
LoRA dropout	0.05
Target modules	gate, up, down projections
Learning rate	$3\times 10^{-4}$
Training epochs	2

Appendix

Table 6: LoRA hyperparameters used for MapSteal Student adaptation.

Stage	Runs	Time (h)
MapSteal LoRA training	18	6.3
QEDKS LoRA training	18	6.6
QEDKS acquisition	18	$\sim$ 36.0
EvoKD online acquisition/training	18	$\sim$ 54.0
Final evaluation (all methods)	54	22.4

Appendix

Table 7: Recorded GPU time for the final experiments. Acquisition and online-training durations include Teacher API waiting time. LoRA training required 15–19 minutes per seed for 2B, 20–26 minutes for 4B, and 20–25 minutes for 9B Students. Final evaluation covered all 54 adapters, requiring 7.3, 7.7, and 7.4 GPU-hours for MapSteal, QEDKS-Adapted, and EvoKD-Adapted, respectively.

Figure 4: Acquisition depth and expansion operator distributions under different initial root counts. Each configuration contains 460 executed acquisition nodes. Depth statistics include the acquisition roots but exclude the shared cold-start QA pairs.

Metric	R1	R10	R20	R35
Recursive nodes	459	450	440	425
Mean depth	5.27	2.86	2.37	1.87
Max depth	6	5	5	4
CONDITION_DEPTH	100	133	113	108
Non-empty Condition (%)	37.8	42.8	33.0	29.1
Training QA after QC	500	498	496	495

Appendix

Table 8: Acquisition statistics under different initial root counts. All configurations use 460 successful Teacher acquisition queries.

Figure 5: Extended QA budget scaling of MapSteal on Qwen3.5-4B for Medicine, under the LoRA recipe (learning rate

2\times 10^{-4}

, three epochs, all seven LoRA target modules). The

y

-axis shows accuracy gain over the pristine Student in percentage points. Results show mean

\pm

sample standard deviation over three training seeds using one fixed nested acquisition trajectory. The budget counts successfully acquired QA supervision, including the 40-example seed set, rather than API requests. B0 is the pristine Student without an adapter, and the dashed horizontal line marks zero gain.

Figure 6: Complete acquisition forests of Simplified OF (left) and Full MapSteal (right). Both methods execute 460 acquisition questions from the same 35 roots. Nodes are colored by depth, and edges indicate recorded parent–child relationships.

Figure 7: Complete executed acquisition tree for an acute appendicitis root in Full MapSteal. All 14 executed nodes are shown with their original questions and recorded parent–child links. The selected nodes displayed in the main-text qualitative example are part of this tree.

Figure 8: Condition transitions in a real acquisition lineage rooted in multiple sclerosis. CONDITION_DEPTH introduces and later replaces the condition, while NEW_TARGET preserves the existing condition. A separate NEW_FACET branch explores another facet. All displayed nodes and edges are taken from the executed acquisition trajectory.

Domain	Roots	Admit/call	Novelty	$\boldsymbol{\Delta_{\mathrm{acq}}}$	95% CI	Seeds	Panel
Medicine	35	2.57	0.179	$+1.59$	$[+0.37,+2.82]$	3	Medical3
CS	46	1.97	0.250	$-0.57$	–	3	MMLU-CS
Law	48	1.72	0.257	$-2.35$	$[-3.57,-1.14]$	3	LawMacro
Biology	42	2.39	0.213	$+0.35$	–	1	MMLU-Bio
Business	64	–	–	$-0.20$	$[-2.31,+1.89]$	1	MMLU-Business
Chemistry	48	2.18	0.257	$+0.99$	$[-0.55,+2.47]$	3	Broad3

Appendix

Table 9: Cross-domain results with Qwen3.5-4B.

\Delta_{\mathrm{acq}}

measures the accuracy difference over the matched root-only baseline, except for Medicine, which uses Seed40-only. Acquisition statistics are computed over the first 460 successful executions, including initial roots. Confidence intervals are reported in percentage points.

Domain	Student	Primary $\Delta$ [95% CI]	Independent2 $\Delta$ [95% CI]
Chemistry	2B	$-0.89\;[-2.62,+0.81]$	$-0.36\;[-1.19,+0.47]$
	4B	$+0.99\;[-0.55,+2.47]$	$-0.36\;[-1.20,+0.47]$
	9B	$+1.79\;[+0.04,+3.49]$	$-0.77\;[-1.69,+0.12]$
Cybersecurity	2B	$+1.51\;[+0.78,+2.24]$	–
	4B	$+0.88\;[+0.27,+1.49]$	–
	9B	$+1.13\;[+0.56,+1.70]$	–

Appendix

Table 10: Incremental acquisition gains across Student scales (B500 minus root-only, pp). Primary evaluation uses Broad3 for Chemistry and CyberMacro for Cybersecurity. Independent2 is an additional Chemistry evaluation averaging ChemBench MCQ and SuperGPQA Chemistry.

	Medicine
Run	MedMCQA	MedQA	MedXpert
Seed 42	31.20	31.42	10.69
Seed 43	44.80	45.56	12.69
Seed 44	45.60	46.58	13.80
	$40.53\pm 8.09$	$41.19\pm 8.47$	$12.39\pm 1.57$
	Cybersecurity

Appendix

Table 11: Individual EvoKD runs with Qwen3.5-2B. Results are accuracy (%); the final row reports mean

\pm

sample SD over three independent runs.

Domain	Method	QA Rows	Input (K)	Output (K)	Cost ($)
Medicine	MapSteal	495	1278.6	318.3	5.10
	EvoKD	504	$132.3\pm 24.8$	$54.8\pm 8.3$	$0.70\pm 0.12$
	QEDKS	500	$439.8\pm 6.4$	$53.9\pm 1.5$	$1.31\pm 0.02$
Cybersecurity	MapSteal	500	1268.9	294.1	4.89
	EvoKD	504	$119.6\pm 2.4$	$50.3\pm 0.3$	$0.64\pm 0.01$
	QEDKS	500	$241.6\pm 0.2$	$55.1\pm 0.2$	$0.92\pm 0.00$

Appendix

Table 12: Recorded Teacher token usage and estimated acquisition costs under the nominal B500 supervision budget. Token counts are reported in thousands (K). Costs are reconstructed from archived usage records rather than verified invoices. Some EvoKD records have incomplete Teacher-call accounting.

Metric	Count	Rate
Initially incorrect	195	–
Excluded cases	16	–
Evaluated errors	179	100.0%
Errors persisted	50	27.9%
Correct after reformulation	129	72.1%

Appendix

Table 13: Error persistence under question reformulation in a preliminary Medicine diagnostic.

Knowledge Augmentation for Language Models

Knowledge augmentation for language models adds external or newly learned information to improve factuality, reasoning, and domain performance.

See all 180 papers

KA2L: A Knowledge-Aware Active Learning Framework for LLMs
Mar 18, 2026Haoxuan Yin, Chen Tang, Yangfan Wang +2Active Learning LLM Fine-Tuning
Faculty of Computing, Harbin Institute of Technology, Harbin, 150001, China · Institute for Advanced Algorithms Research, Shanghai, 201306, China · National Key Laboratory of Smart Farm Technologies and Systems, Harbin, 150001, China
Task-Specific Knowledge Distillation via Intermediate Probes
Feb 18, 2026Ryan Brown, Chris RussellLanguage Model Distillation Knowledge Distillation
Oxford Internet Institute, University of Oxford, Oxford, United Kingdom.
LittleLearner: Language Models Under Pedagogically Controlled Knowledge Exposure
Aug 13, 2026Fanfei Li, Jana Zeller, Manuel Prada-Corral +4Language Model Pretraining Knowledge Augmentation for Language Models
ETHZ

Mar 18, 2026J/K move · Enter open · S save

KA2L: A Knowledge-Aware Active Learning Framework for LLMs

Haoxuan Yin, Chen Tang, Yangfan Wang, Lian Yan+1

Faculty of Computing, Harbin Institute of Technology, Harbin, 150001, China · Institute for Advanced Algorithms Research, Shanghai, 201306, China · National Key Laboratory of Smart Farm Technologies and Systems, Harbin, 150001, China

Active Learning LLM Fine-Tuning Knowledge Augmentation for Language Models

Open paper AskPDF arXiv

Fine-tuning large language models (LLMs) with high-quality knowledge has been shown to enhance their performance effectively. However, there is a paucity of research on the depth of domain-specific knowledge comprehension by LLMs and the application of targeted active learning to improve their expertise. To address this gap, we introduce the Knowledge-Aware Active Learning (KA2L) framework. This framework assesses LLMs' mastery of specific knowledge points to aid in constructing unanswerable or unknowable questions through latent space analysis. This active learning strategy enhances training efficiency by focusing on knowledge the model has yet to master, thereby minimizing redundancy in learning already acquired information. This study innovatively employs a knowledge distribution probing technique to examine the hidden states of specific Transformer layers and identify the distribution of known and unknown knowledge within the LLM. Additionally, a hidden-state decoding method is proposed to generate numerous unknown questions in natural language from the latent knowledge space. In our experiments, we selected nine open-source LLMs to validate the effectiveness of the proposed framework. Results indicate that KA2L not only significantly reduces 50% annotation and computation costs across two open-domain and one vertical-domain dataset but also achieves better performance, offering valuable insights into active learning strategies for LLMs. The code is available at https://github.com/greenjerry/KA2L.

Abstract

Knowledge Augmentation for Language Models

Explore similar work

KA2L: A Knowledge-Aware Active Learning Framework for LLMs

Task-Specific Knowledge Distillation via Intermediate Probes

LittleLearner: Language Models Under Pedagogically Controlled Knowledge Exposure