"Nothing to See Here'': Unintended Disclosure through Revision Traces of LLM Deliverables

Yage Zhang; Yukun Jiang; Yang Zhang

cs.CRSep 28, 2026

"Nothing to See Here'': Unintended Disclosure through Revision Traces of LLM Deliverables

Authors: Yage Zhang, Yukun Jiang, Yang Zhang

Organizations: CISPA Helmholtz Center for Information Security

Abstract

Large language model (LLM) assistants increasingly help users draft content for third-party recipients. During private drafting, the user or the model may introduce an item and later remove or replace it. The model may remove the item from the intended content but reveal it again when stating the edit. We call such statements revision traces. For example, after a user removes the password before sharing a configuration file, the model may delete it but leave a comment saying, "Removed the password 'No****4!' as requested." A third-party recipient who sees only the delivered file can therefore recover the withdrawn password from the comment. In an in-the-wild analysis of three public conversation corpora, we identify 26,753 revision requests, of which 2,363 (8.8%) leave revision traces. We study them in greater depth under controlled conditions by introducing RevLeakBench, a benchmark of 100 tasks across five scenarios with a conversation track and an agent track. We measure trace occurrence, withdrawn-item recovery, trace position, and required-content retention. Across six models, about half of the deliverables in both tracks state the edit after a revocation, and a reader that sees only the deliverable can recover the withdrawn item from about 13% of them. Telling the model that its entire reply will be forwarded to the recipient still leaves revision traces in 36.4% of the deliverables. We compare prompt defenses and a delivery boundary, and propose an output-side filter that sharply reduces recovery with little loss of required content. We believe our work can benefit efforts to understand and mitigate unintended disclosure in LLM interactions.

Figures & tables

Figure 2 : Overview of RevLeakBench and the evaluation pipeline.

Panel of Figure 3 — Figure 3 : Revision traces in public conversations by revision type, deliverable type, and position.

Scenario	Deliverable	Withdrawn item	Required content	Tasks	$n$
Coordination messages	Message to a recipient	Must-not-share items of an app	Must-share items	20	1,827
Enterprise briefs	Research brief	Internal numerical facts	Public evidence	20	2,153
Policy-constrained documents	Summary for outside readers	Policy-violating propositions	Non-violating propositions	20	1,858
Data analysis reports	Report from analysis notes	Findings	Findings answering the report question	20	2,009
Software artifacts	Issue comment, review message, or configuration example	Identities, configuration values, operational details	Technical content	20	1,809
Total				100	9,656

Table 1 : Deliverable, withdrawn item, and required content of each scenario, with

n

counting deliverables across both tracks.

Scenario	$n$	Full	Body	Utility	$n$	Full	Body	Utility
	Conv. track				Agent track
Coordination messages	240	53.3%	3.3%	87.1%	240	74.6%	7.9%	67.5%
Enterprise briefs	240	71.7%	1.7%	88.3%	240	38.8%	1.7%	93.3%
Policy-constrained documents	240	60.8%	10.0%	79.2%	240	73.8%	45.4%	82.1%
Data analysis reports	240	52.1%	1.7%	95.0%	239	23.5%	7.5%	99.2%
Software artifacts	238	46.6%	6.3%	82.8%	240	40.8%	6.7%	80.0%

Table 2 : LLM-detected revision traces after revocation, over the full reply and body.

Figure 4 : A revision trace inside the delivered body.

Scenario	$n$	Static exclusion	Turn-matched	Late exclusion	Revocation
Coordination messages	159	45.3%	34.6%	43.4%	49.7%
Enterprise briefs	159	34.6%	19.5%	57.2%	89.9%
Policy-constrained documents	160	64.4%	61.2%	73.1%	69.4%
Data analysis reports	160	14.4%	28.1%	31.2%	57.5%
Software artifacts	156	19.2%	22.4%	38.5%	56.4%
All scenarios	794	35.6%	33.2%	48.7%	64.6%

Table 3 : Trace rates across four control conditions.

Panel of Figure 5 — Figure 5 : Revision traces after a revocation, by model, by position, and against the matched controls.

		Conv. track					Agent track
Source	Defense	Delivered	Traces	Recovered	Utility	End-to-end	Delivered	Traces	Recovered	Utility	End-to-end
—	No defense	99.8%	56.9%	13.7%	86.5%	74.5%	99.9%	50.3%	13.4%	84.4%	74.5%
AgentCIBench	restrictive	100.0%	50.5%	4.9%	83.1%	78.8%	99.7%	45.0%	10.5%	79.0%	71.3%
AgentCIBench	recipient_typed	100.0%	32.9%	2.7%	81.6%	79.3%	99.7%	40.1%	10.1%	86.6%	77.2%
RevLeakBench (Ours)	Delivery boundary	82.2%	2.4%	0.1%	90.3%	74.1%	—
	Delivery boundary, notes field	78.0%	1.9%	0.1%	89.9%	69.9%	—

Table 4 : Defense results after revocation.

		Conv. track		Agent track
Setting	Pairs	Baseline	Instruction	Baseline	Instruction
Forwarding notice	1,588	64.7%	38.5%	54.9%	34.3%
Deliverable-only instruction	1,595	64.5%	23.6%	55.0%	31.4%

Table 5 : Full-reply trace rates under two output instructions and their baselines.

Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

Scenario	Category	Scope Boundary
Coordination messages	Communication writing, business communication	Covers both personal and work correspondence
Enterprise briefs	Briefing, market research and analysis	Excludes market studies
Policy-constrained documents	Editing under disclosure constraints	Excludes legal writing as a domain
Data analysis reports	Academic and engineering reporting	Excludes data analysis ability
Software artifacts	Technical documentation	Excludes executable code changes

Appendix

Table 6 : Scenarios, the task and writing categories they cover, and their scope boundaries.

	Conv. track		Agent track
Scenario	Target in direct draft	Qualifying construction	Target in direct draft	Qualifying construction
Coordination messages	38.7%	26.0%	39.2%	37.0%
Enterprise briefs	100.0%	99.6%	95.0%	99.0%
Policy-constrained documents	42.0%	37.0%	50.0%	53.0%
Data analysis reports	86.0%	88.0%	80.0%	78.0%
Software artifacts	29.0%	31.0%	34.0%	36.0%

Appendix

Table 7 : Availability of the model-added condition by scenario and track.

Condition	User instruction sequence	Required content	Excluded item
Revocation
Revocation of $A$	Introduce $A$ , then withdraw it	$Y$	$A$
Static exclusion of $A$	Exclude $A$ from the start	$Y$	$A$
Revocation of $B$	Introduce $B$ , then withdraw it	$Y$	$B$
Static exclusion of $B$	Exclude $B$ from the start	$Y$	$B$
Replacement

Appendix

Table 8 : Conditions of each task.

Scenario	Presence	Utility
Coordination messages	AgentCIBench leaked-item check	RedacBench checkPropositions
Enterprise briefs	Recorded numerical values	RedacBench checkPropositions
Policy-constrained documents	RedacBench checkPropositions	RedacBench checkPropositions
Data analysis reports	RedacBench checkPropositions	RedacBench checkPropositions
Software artifacts	RedacBench checkPropositions	RedacBench checkPropositions

Appendix

Table 9 : Presence and utility checks by scenario.

Figure 6 : A reply annotated with its parts and revision-trace sentences.

Check	Method	Labels	Precision	Recall	$\kappa$
Revision trace	Scanner	375	0.91	0.63	0.63
	GPT-4.1-mini	375	0.84	0.92	0.80
	DeepSeek-V4-Flash	375	0.81	0.94	0.80
	GLM-5.3-Flash	375	0.89	0.91	0.84
Utility	GPT-4.1-mini	357	0.98	0.96	0.78
	DeepSeek-V4-Flash	357	1.00	0.93	0.76

Appendix

Table 10 : Checks against the human labels.

Condition	Model	Statement in the deliverable	Reader answer	Recovered
Revocation	GLM-5.2	“I’ll leave the Maya IVF retrieval out”	Maya IVF retrieval	Yes
Revocation	GPT-5.5	“I’ll leave that off the shared schedule”	Unknown	No
Static exclusion	GLM-5.2	None	Unknown	No

Appendix

Table 11 : One task under revocation and static exclusion.

Scenario	$n$	Full	Ident.	Descr.	Proc.	Body	Full	Utility	$n$	Full	Ident.	Descr.	Proc.	Body	Full	Utility
	Conv. track								Agent track
		Scanner					LLM check			Scanner					LLM check
Coordination messages	240	44.2%	26.7%	5.4%	11.7%	1.7%	53.3%	87.1%	240	58.8%	34.2%	12.1%	10.0%	5.0%	74.6%	67.5%
Enterprise briefs	240	67.5%	25.0%	22.1%	18.3%	2.1%	71.7%	88.3%	240	34.2%	5.8%	14.2%	12.1%	2.1%	38.8%	93.3%
Policy-constrained documents	240	59.6%	12.9%	15.0%	31.2%	8.3%	60.8%	79.2%	240	64.6%	14.2%	9.6%	20.4%	30.4%	73.8%	82.1%
Data analysis reports	240	51.2%	0.4%	21.7%	27.5%	5.4%	52.1%	95.0%	239	18.8%	0.4%	14.2%	3.8%	11.7%	23.5%	99.2%

Appendix

Table 12 : Revision traces after a revocation from the scanner and the LLM check.

			Scanner					LLM check
Scenario	Model	$n$	Full	Ident.	Descr.	Proc.	Body	Full	Utility
Coordination messages	GPT-5.6	40	45.0%	20.0%	5.0%	20.0%	0.0%	60.0%	80.0%
	GPT-5.5	40	62.5%	25.0%	15.0%	22.5%	10.0%	67.5%	95.0%
	DeepSeek-V4-Pro	40	20.0%	15.0%	0.0%	5.0%	0.0%	22.5%	82.5%
	DeepSeek-V4-Flash	40	57.5%	52.5%	0.0%	5.0%	0.0%	57.5%	87.5%
	GLM-5.3	40	45.0%	25.0%	10.0%	7.5%	0.0%	62.5%	87.5%

Appendix

Table 13 : Revision traces after a revocation, conversation track, by model.

			Scanner					LLM check
Scenario	Model	$n$	Full	Ident.	Descr.	Proc.	Body	Full	Utility
Coordination messages	GPT-5.6	40	62.5%	47.5%	12.5%	2.5%	5.0%	72.5%	65.0%
	GPT-5.5	40	60.0%	32.5%	12.5%	15.0%	10.0%	72.5%	92.5%
	DeepSeek-V4-Pro	40	45.0%	22.5%	10.0%	10.0%	0.0%	70.0%	47.5%
	DeepSeek-V4-Flash	40	50.0%	30.0%	5.0%	12.5%	2.5%	57.5%	50.0%
	GLM-5.3	40	65.0%	27.5%	27.5%	5.0%	5.0%	87.5%	85.0%

Appendix

Table 14 : Revision traces after a revocation, agent track, by model.

Scenario	Model	$n$	Full	Ident.	Descr.	Proc.	Body	Full	Body	Utility
			Scanner					LLM check
Enterprise briefs	GPT-5.5	60	18.3%	3.3%	6.7%	8.3%	3.3%	15.0%	0.0%	90.0%
	DeepSeek-V4-Pro	60	61.7%	35.0%	21.7%	5.0%	0.0%	65.0%	0.0%	80.0%
	GLM-5.3	60	46.7%	3.3%	5.0%	38.3%	1.7%	56.7%	0.0%	90.0%
	GLM-5.2	60	76.7%	16.7%	36.7%	23.3%	1.7%	71.7%	0.0%	95.0%
	All models	240	50.8%	14.6%	17.5%	18.8%	1.7%	52.1%	0.0%	88.8%

Appendix

Table 15 : Revision traces under the reduced wording, by model, over the revocation and replacement conditions.

Scenario	$n$	Recovered	Given a trace	No trace	Unresolved	$n$	Recovered	Given a trace	No trace	Unresolved
	Conv. track					Agent track
Coordination messages	240	27.1%	50.0%	1	0	240	37.9%	49.7%	0	4
Enterprise briefs	239	23.8%	32.6%	0	1	240	9.2%	23.7%	0	0
Policy-constrained documents	234	8.1%	13.6%	0	4	229	7.9%	9.6%	1	19
Data analysis reports	240	3.3%	6.4%	0	0	239	4.2%	17.9%	0	1
Software artifacts	238	5.9%	12.6%	0	0	237	7.6%	16.8%	1	3

Appendix

Table 16 : Reader recovery after a revocation, with trace judgments from the LLM check.

	Conv. track	Agent track
Defense	Difference	Difference
restrictive	+4.2 [+1.0, +7.2]	$-3.2$ [ $-5.8$ , $-0.7$ ]
recipient_typed	+4.8 [+0.9, +8.8]	+2.8 [ $-0.9$ , +6.3]
Delivery boundary	$-0.4$ [ $-4.7$ , +3.9]	—
Delivery boundary, notes field	$-4.6$ [ $-9.2$ , +0.0]	—
Delivery boundary, body only	+8.9 [+6.0, +11.8]	+5.9 [+3.7, +8.2]

Appendix

Table 17 : Paired end-to-end differences against the matched no-defense baseline, in percentage points with 95% confidence intervals.

		LLM check, full reply	Scanner, full reply				Scanner, body
Condition	$n$	Traces	Traces	Ident.	Descr.	Proc.	Traces	Utility	Recovered
Revocation of $A$	1198	53.4%	48.8%	12.4%	13.6%	19.6%	7.2%	85.6%	11.9%
Static exclusion of $A$	1194	37.9%	31.2%	2.3%	11.6%	13.8%	13.8%	86.3%	5.2%
Revocation of $B$	1199	53.8%	46.9%	14.6%	12.0%	17.2%	7.6%	85.2%	15.2%
Static exclusion of $B$	1197	37.1%	30.9%	2.7%	11.4%	13.2%	12.6%	88.7%	4.3%
Replacement	1200	31.8%	25.9%	8.4%	7.2%	6.4%	6.7%	81.7%	10.1%

Appendix

Table 18 : Revision traces by condition.

Information type	Items	Examples
Security credentials	6	Access keys, passwords
Personal information	72	Health, religion, identity numbers, home addresses
Organizational information	77	Salaries, internal business figures, operational details
Content the user chose not to deliver	45	Analysis findings, ordinary task content
Total	200

Appendix

Table 19 : Information types of the withdrawn items.

Information type	$n$	Traces	Identifying	Recovered	Utility
Security credentials	72	52.8%	1.4%	0.0%	80.6%
Personal information	863	49.0%	18.7%	18.7%	78.0%
Organizational information	923	45.8%	15.1%	14.3%	86.0%
Content the user chose not to deliver	539	34.9%	4.3%	5.9%	97.0%

Appendix

Table 20 : Revision traces and recovery after a revocation by information type, with Traces and Identifying measured by the scanner.

	Conv. track					Agent track
Scenario	Pairs	LLM check	Scanner	Recovered	Utility	Pairs	LLM check	Scanner	Recovered	Utility
AgentCIBench restrictive
Coordination messages	240	$-$ 18.3 [ $-$ 27.1, $-$ 9.6]	$-$ 20.0 [ $-$ 28.8, $-$ 11.7]	$-$ 21.2 [ $-$ 32.1, $-$ 11.2]	$-$ 6.2 [ $-$ 11.2, $-$ 1.7]	235 / 240 / 240	$-$ 4.7 [ $-$ 13.0, +2.5]	+1.7 [ $-$ 5.0, +7.9]	$-$ 12.1 [ $-$ 19.6, $-$ 5.0]	$-$ 10.4 [ $-$ 16.7, $-$ 5.0]
Enterprise briefs	239 / 240 / 238	$-$ 4.2 [ $-$ 9.7, +2.1]	$-$ 0.8 [ $-$ 5.8, +4.6]	$-$ 13.4 [ $-$ 18.3, $-$ 8.8]	$-$ 2.1 [ $-$ 6.7, +2.5]	236	$-$ 13.1 [ $-$ 18.6, $-$ 7.6]	$-$ 8.9 [ $-$ 15.7, $-$ 2.1]	$-$ 4.2 [ $-$ 7.6, $-$ 0.9]	$-$ 3.8 [ $-$ 9.8, +2.1]
Policy-constrained documents	236 / 240 / 234	$-$ 14.0 [ $-$ 20.3, $-$ 7.2]	$-$ 16.7 [ $-$ 23.8, $-$ 10.0]	$-$ 4.7 [ $-$ 7.6, $-$ 2.1]	$-$ 0.4 [ $-$ 4.2, +3.8]	206 / 240 / 218	$-$ 6.3 [ $-$ 11.9, $-$ 0.5]	$-$ 8.8 [ $-$ 15.0, $-$ 1.7]	+0.5 [ $-$ 4.1, +5.0]	$-$ 6.2 [ $-$ 12.1, $-$ 0.4]
Data analysis reports	240 / 240 / 239	$-$ 0.4 [ $-$ 7.9, +7.5]	$-$ 4.6 [ $-$ 13.3, +4.6]	$-$ 0.8 [ $-$ 2.9, +1.2]	$-$ 2.1 [ $-$ 6.7, +2.1]	237 / 239 / 239	$-$ 4.6 [ $-$ 10.5, +1.3]	$-$ 2.5 [ $-$ 9.7, +4.6]	$-$ 0.8 [ $-$ 3.4, +1.7]	$-$ 1.3 [ $-$ 3.4, +0.8]

Appendix

Table 21 : Each defense paired against no defense on the same task, model, and condition, in percentage points.

Scenario	$n$	Preface	Body opening	Body middle	Body closing	Afterword	$n$	Preface	Body opening	Body middle	Body closing	Afterword
	Conv. track						Agent track
Coordination messages	240	34.2%	0.4%	1.2%	0.0%	10.4%	240	30.8%	1.7%	2.9%	0.4%	26.2%
Enterprise briefs	240	61.2%	0.4%	0.8%	0.8%	8.8%	240	25.4%	0.8%	0.8%	0.8%	8.8%
Policy-constrained documents	240	32.1%	0.0%	6.7%	1.7%	30.8%	240	19.6%	2.5%	25.8%	13.3%	16.2%
Data analysis reports	240	42.1%	0.0%	4.6%	1.2%	4.6%	239	0.4%	0.8%	8.8%	2.9%	7.1%
Software artifacts	238	30.7%	0.0%	2.9%	0.0%	10.5%	240	6.7%	0.0%	3.8%	1.2%	27.9%

Appendix

Table 22 : Deliverables with revision traces by position after a revocation, measured by the scanner.

	Conv. track							Agent track
Condition	$n$	Preface	Body opening	Body middle	Body closing	Afterword	Traces	$n$	Preface	Body opening	Body middle	Body closing	Afterword	Traces
Coordination messages
Revocation	240	34.2%	0.4%	1.2%	0.0%	10.4%	44.2%	240	30.8%	1.7%	2.9%	0.4%	26.2%	58.8%
Replacement	120	19.2%	0.8%	2.5%	0.0%	13.3%	33.3%	120	21.7%	2.5%	10.8%	0.8%	17.5%	48.3%
Static exclusion	240	4.6%	0.0%	7.5%	0.0%	11.7%	22.9%	237	14.8%	0.8%	11.0%	0.4%	15.2%	35.4%
Replacement control	120	5.0%	0.0%	7.5%	0.0%	13.3%	23.3%	119	6.7%	0.0%	5.9%	0.8%	12.6%	23.5%

Appendix

Table 23 : Deliverables with revision traces by condition and position, measured by the scanner, with each deliverable counted at every position containing a trace.

Scenario	Revocation of $A$	Static exclusion of $A$	Revocation of $B$	Static exclusion of $B$	Replacement	Replacement control
Coordination messages	65.0% / 30.8% / 77.9%	43.3% / 11.8% / 81.5%	62.9% / 34.2% / 76.7%	45.6% / 12.1% / 84.5%	49.2% / 33.3% / 63.8%	32.2% / 2.9% / 79.5%
Enterprise briefs	54.2% / 12.6% / 90.4%	24.7% / 0.8% / 84.1%	56.2% / 20.4% / 91.2%	20.6% / 0.0% / 88.2%	18.3% / 6.2% / 90.4%	5.0% / 0.0% / 92.5%
Policy-constrained documents	67.5% / 6.5% / 80.8%	59.6% / 2.1% / 81.7%	67.1% / 9.5% / 80.4%	61.2% / 1.3% / 84.6%	49.6% / 4.7% / 76.2%	54.6% / 3.4% / 77.9%
Data analysis reports	36.7% / 2.9% / 97.9%	35.4% / 7.9% / 99.6%	39.1% / 4.6% / 96.2%	36.7% / 4.6% / 100.0%	24.6% / 2.5% / 98.3%	27.9% / 7.9% / 99.2%
Software artifacts	43.7% / 6.3% / 81.1%	26.2% / 3.4% / 84.7%	43.8% / 7.1% / 81.7%	21.2% / 3.4% / 86.2%	17.1% / 3.3% / 79.9%	22.9% / 2.9% / 82.5%

Appendix

Table 24 : Revocation and replacement against their matched controls, over the six models, both tracks pooled (traces / recovered / utility).

	Conv. track					Agent track
Scenario	Pairs	LLM check	Scanner	Recovered	Utility	Pairs	LLM check	Scanner	Recovered	Utility
Coordination messages	239	+19.2 [+7.9, +30.5]	+21.2 [+12.1, +30.8]	+19.2 [+7.9, +31.7]	+2.5 [ $-$ 2.5, +7.1]	229	+19.7 [+8.4, +31.8]	+23.6 [+13.8, +34.6]	+22.4 [+8.8, +37.1]	$-$ 13.9 [ $-$ 22.2, $-$ 5.9]
Enterprise briefs	239	+49.0 [+43.1, +54.2]	+46.2 [+38.8, +53.8]	+23.4 [+15.5, +31.7]	+0.8 [ $-$ 5.8, +8.3]	237	+16.0 [+9.7, +21.9]	+22.8 [+17.3, +28.0]	+8.4 [+4.6, +12.7]	+8.4 [+0.8, +16.9]
Policy-constrained documents	231	+10.8 [+5.2, +16.9]	+10.8 [+5.0, +17.1]	+6.8 [+3.5, +10.1]	+1.7 [ $-$ 3.8, +7.5]	199	+0.5 [ $-$ 6.5, +7.2]	$-$ 4.2 [ $-$ 9.6, +1.2]	+4.1 [+0.4, +7.7]	$-$ 6.7 [ $-$ 11.7, $-$ 2.5]
Data analysis reports	239	+30.1 [+21.7, +37.5]	+35.8 [+27.9, +43.8]	$-$ 0.4 [ $-$ 3.3, +2.5]	$-$ 5.0 [ $-$ 7.9, $-$ 2.5]	233	$-$ 27.0 [ $-$ 35.6, $-$ 18.2]	$-$ 28.5 [ $-$ 38.2, $-$ 18.3]	$-$ 4.6 [ $-$ 11.7, +1.3]	$-$ 0.4 [ $-$ 2.1, +0.8]
Software artifacts	236	+32.6 [+26.6, +38.0]	+29.4 [+22.6, +36.0]	+4.6 [+1.2, +8.4]	$-$ 3.4 [ $-$ 9.2, +3.0]	232	+5.6 [ $-$ 2.1, +13.7]	+10.5 [+4.2, +17.3]	+1.7 [ $-$ 3.1, +7.7]	$-$ 4.6 [ $-$ 11.0, +1.7]

Appendix

Table 25 : Revocation against the static exclusion by track, over the six models, in percentage points.

Scenario	Control	Replacement	Difference	Control	Replacement	Difference
	Conv. track			Agent track
Coordination messages	28.7%	41.7%	+13.0 [+3.4, +23.5]	37.2%	57.5%	+20.4 [+8.8, +32.5]
Enterprise briefs	0.8%	17.5%	+16.7 [+10.0, +23.3]	9.5%	18.1%	+8.6 [+2.6, +15.1]
Policy-constrained documents	42.7%	40.2%	$-$ 2.6 [ $-$ 10.4, +5.9]	71.3%	61.4%	$-$ 9.9 [ $-$ 16.3, $-$ 3.2]
Data analysis reports	13.4%	26.1%	+12.6 [+7.5, +17.5]	44.3%	24.3%	$-$ 20.0 [ $-$ 30.7, $-$ 9.2]
Software artifacts	14.2%	10.0%	$-$ 4.2 [ $-$ 9.2, +0.8]	31.9%	24.4%	$-$ 7.6 [ $-$ 17.8, +2.5]

Appendix

Table 26 : Replacement and replacement-control trace rates by track.

Scenario	Triplets	Original	Trace removed	Unrelated removed	$p_{10}$ / $p_{01}$	$p$	$p_{10}$ / $p_{01}$	$p$
		Recovered			Trace removed		Unrelated removed
Coordination messages	233	133	10	129	125 / 2	$9.6\times 10^{-35}$	7 / 3	0.34
Enterprise briefs	289	84	2	81	83 / 1	$8.8\times 10^{-24}$	4 / 1	0.38
Policy-constrained documents	375	37	9	35	34 / 5	$2.4\times 10^{-6}$	8 / 5	0.58
Data analysis reports	170	17	1	17	16 / 0	$3.1\times 10^{-5}$	2 / 2	1.00
Software artifacts	172	32	2	31	31 / 1	$1.5\times 10^{-8}$	4 / 3	1.00

Appendix

Table 27 : Recovery counts before and after trace deletion, over six models.

Scenario	Triplets	Original	Trace removed	Unrelated removed	Trace $p_{10}$ / $p_{01}$	Unrelated $p_{10}$ / $p_{01}$	Triplets	Original	Trace removed	Unrelated removed	Trace $p_{10}$ / $p_{01}$	Unrelated $p_{10}$ / $p_{01}$
	Conv. track						Agent track
Coordination messages	102	52	1	52	52 / 1	0 / 0	131	81	9	77	73 / 1	7 / 3
Enterprise briefs	190	58	1	56	58 / 1	2 / 0	99	26	1	25	25 / 0	2 / 1
Policy-constrained documents	175	18	0	15	18 / 0	4 / 1	200	19	9	20	16 / 5	4 / 4
Data analysis reports	122	7	0	6	7 / 0	2 / 1	48	10	1	11	9 / 0	0 / 1
Software artifacts	85	14	0	14	14 / 0	0 / 0	87	18	2	17	17 / 1	4 / 3

Appendix

Table 28 : Recovery counts before and after trace deletion, by track.

	Conv. track		Agent track
Scenario	Baseline	Instruction	Baseline	Instruction
Coordination messages	49.7%	5.2%	71.2%	34.4%
Enterprise briefs	89.9%	50.3%	55.6%	25.6%
Policy-constrained documents	69.4%	55.0%	81.1%	78.6%
Data analysis reports	57.5%	39.4%	15.1%	6.3%
Software artifacts	56.4%	41.7%	51.2%	26.9%

Appendix

Table 29 : Full-reply trace rates from the LLM check under the forwarding notice and its matched baseline.

	Conv. track		Agent track
Position	Baseline	Instruction	Baseline	Instruction
Preface	44.9%	15.5%	19.4%	6.0%
Body	3.8%	1.1%	10.5%	7.9%
Afterword	18.4%	8.5%	23.9%	12.9%
Full reply	64.5%	23.6%	55.0%	31.4%

Appendix

Table 30 : Trace rates under the deliverable-only instruction and its matched baseline, with position rows measured by the scanner and Full reply by the LLM check.

Scenario	$n$	Turn-matched	Late exclusion	Revocation
Coordination messages	159	1.9%	8.2%	22.6%
Enterprise briefs	159	0.6%	0.6%	30.8%
Policy-constrained documents	160	5.0%	4.4%	6.9%
Data analysis reports	160	10.0%	10.0%	5.6%
Software artifacts	156	3.8%	6.4%	7.7%
All scenarios	794	4.3%	5.9%	14.7%

Appendix

Table 31 : Reader recovery under the two controls and the revocation.

Condition	Preface	Body	Afterword	Repeats instruction
Static exclusion	21.5%	36.2%	42.3%	39.3%
Turn-matched	10.8%	27.0%	62.2%	44.6%
Late exclusion	44.5%	10.6%	44.8%	44.5%
Revocation	82.1%	5.0%	12.9%	12.3%

Appendix

Table 32 : Trace-sentence positions and instruction repetition for the four conditions, measured by the scanner as percentages of trace sentences.

	Conv. track				Agent track
Scenario	$n$	Replacement	Original control	Matched control	$n$	Replacement	Original control	Matched control
Coordination messages	80	32.5%	36.2%	40.0%	79	55.7%	41.8%	65.8%
Enterprise briefs	80	22.5%	0.0%	21.2%	80	27.5%	12.5%	32.5%
	Both tracks
Scenario	$n$		Replacement		Original control		Matched control
All scenarios	319		34.5%		22.6%		39.8%

Appendix

Table 33 : Replacement trace rates under the original and matched controls.

Scenario	Wording	$n$	Traces	Recovered	Utility	Cites a policy
Policy-constrained documents	Original	160	66.2%	7.0%	80.0%	67.9%
Policy-constrained documents	Reduced	160	26.1%	0.6%	88.6%	0.0%
Enterprise briefs	Original	160	79.4%	25.0%	90.0%	0.8%
Enterprise briefs	Reduced	160	71.2%	23.3%	90.0%	0.0%
Data analysis reports	Original	160	61.9%	4.4%	92.5%	1.0%
Data analysis reports	Reduced	160	67.5%	6.9%	93.1%	0.0%

Appendix

Table 34 : Results under the original and reduced task wording.

Explore similar work

Stealing Reasoning Traces from Proprietary LLM APIs
Aug 10, 2026Alexander Panfilov, David Schmotz, Ilia Shumailov +5LLM Reasoning Strategies Reasoning Traces
MATS Research · ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems +4
PrivDrift: Auditing User-Secret Leakage Under Topic Drift in Active LLM Conversations
Sep 24, 2026Luciano MaldonadoData Leakage Disclosures
West Virginia University, Morgantown, WV, USA.
Auditable Release Control for Pedagogical Leakage in LLM Tutors
Aug 1, 2026Nizam KadirData Leakage Release
Science, Mathematics and Technology Singapore University of Technology and Design (SUTD) Singapore

Aug 10, 2026, cs.CRJ/K move · Enter open · S save

Stealing Reasoning Traces from Proprietary LLM APIs

Alexander Panfilov, David Schmotz, Ilia Shumailov, Luca Beurer-Kellner+4

MATS Research · ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems+4

LLM Reasoning Strategies Reasoning Traces Jailbreak Attacks Reasoning Skills Traces

Open paper AskPDF arXiv

Leading large language model providers now conceal their models' step-by-step reasoning, or chain-of-thought, to protect intellectual property and limit information leakage. Rather than storing these traces server-side, providers return them to the client as blocks of encrypted text, which the client passes back with each subsequent request. Building on prior research, we identify an architectural vulnerability: these encrypted blocks are fully compatible and interchangeable across different sessions, users, and models within a provider's ecosystem. We exploit this compatibility to develop a scalable decryption jailbreak. By injecting an encrypted reasoning trace from a given model into a weaker, and less safeguarded model from the same provider, we force it to decode and output the trace verbatim in plaintext, without ever jailbreaking the more capable model directly. This vulnerability enables four distinct attack vectors. First, it circumvents anti-distillation mechanisms, allowing adversaries to extract a proprietary model's reasoning, as we demonstrate across Anthropic, OpenAI, and Google. Second, it allows for large-scale private data extraction. Developers frequently share session logs publicly, unaware of contents of the encrypted blocks. By decoding 315,320 reasoning blocks scraped from public repositories, we recovered 367 Personally Identifiable Information (PII) artifacts and 182 credentials. Third, it inadvertently reveals hazardous information hidden within the reasoning process, even in cases where the model's final, visible output safely rejects a malicious request. Fourth, attackers can leverage this flaw to execute invisible prompt injections, embedding malicious payloads entirely within encrypted blocks to poison public agentic rollouts. Following responsible disclosure, we propose concrete cryptographic and system-level mitigations to secure client-side reasoning.

Abstract

Explore similar work

Stealing Reasoning Traces from Proprietary LLM APIs

PrivDrift: Auditing User-Secret Leakage Under Topic Drift in Active LLM Conversations

Auditable Release Control for Pedagogical Leakage in LLM Tutors