AIDG: A Formal Decomposition of Information Extraction and Containment Asymmetries in Multi-Turn LLM Dialogue

Adib Sakhawat; Fardeen Sadab; Rakin Shahriar

cs.CLFeb 19, 2026

AIDG: A Formal Decomposition of Information Extraction and Containment Asymmetries in Multi-Turn LLM Dialogue

Authors: Adib Sakhawat, Fardeen Sadab, Rakin Shahriar

Organizations: Department of Computer Science and Engineering, Islamic University of Technology, Dhaka, Bangladesh

Abstract

Multi-turn LLM evaluation is typically reported as a single win-rate scalar, conflating distinct capabilities. We introduce AIDG (Adversarial Information Deduction Game), formalizing multi-turn adversarial dialogue as a two-player partially observable stochastic game (POSG) and decomposing performance along Seeker (extraction) and Holder (containment) roles. The decomposition isolates three failure modes: cooperative-prior leakage, constraint-reasoning interference, and inefficient hypothesis-space traversal. Across 439 games over six frontier LLMs, defensive performance is tightly clustered (sigma = 1.9 ELO) while offensive performance varies substantially (sigma = 53.3 ELO); confirmation framing increases extraction odds 7.75x over uninformed deduction (p < 0.00001); and constraint violations account for 41.3% of deductive failures, uncorrelated with scale (rho = 0.0). We position the containment-over-extraction gap not as a surprising finding but as a measurable consequence of locally resolvable defensive decisions versus globally coupled offensive planning, and use the decomposition to attribute the gap per model. All design choices, including turn-decay weighting and the Bradley-Terry rating model, are derived from explicit assumptions.

Figures & tables

Model Alias	API Identifier	Architecture	Est. Params
GPT-5	openai/gpt-5	Dense / MoE	Undisclosed
Gemini-2.5	google/gemini-2.5-flash	Undisclosed	Undisclosed
Qwen3-235B	qwen/qwen3-235b-instruct	Dense	235B
DeepSeek-V3	deepseek-ai/deepseek-v3.1	MoE	$\sim$ 671B
Llama-4	meta/llama-4-maverick	MoE	$\sim$ 400B
Granite-8B	ibm-granite/granite-3.3-8b	Dense	8B

Table 1: Evaluated models spanning dense and Mixture-of-Experts (MoE) architectures. API identifiers are provided for reproducibility.

Metric	AIDG-I	AIDG-II
Total Games	289	150
Total Dialogue Turns	2,740	1,663
Avg. Depth (Turns)	9.5	11.1
Reached $T_{\max}$ (%)	87.2%	23.3%
Win Rate (Seeker)	12.8%	14.7%
Win Rate (Holder)	87.2%	85.3%

Table 2: Corpus statistics for AIDG interactions. The higher average depth and lower early-termination rate in AIDG-II reflect the difficulty of constrained deductive search. Most AIDG-I games terminate at

T_{\max}

because the Holder successfully avoided leakage to the end.

Experiment	Def. Adv.	Cohen’s $d$	$p$ -value
AIDG-I	349.6 ELO	5.47	$<$ 0.0001
AIDG-II	161.6 ELO	2.67	0.001
Combined	255.6 ELO	4.07	$<$ 0.0001

Table 3: Defense advantage (

V\_\text{ELO}-C\_\text{ELO}

) across experiments. Effect sizes are large.

Model	$C\_\text{ELO}$	$V\_\text{ELO}$	$\Delta$	Rank
Qwen3-235B	1443.6	1627.1	183.5	1
GPT-5	1409.9	1626.1	216.2	2
Gemini-2.5-Flash	1372.1	1626.3	254.2	3
DeepSeek-V3.1	1360.0	1629.7	269.7	4
Llama-4-Maverick	1349.5	1630.9	281.4	5
Granite-3.3-8B	1297.9	1626.6	328.7	6

Table 4: Average ELO ratings across experiments. Models ranked by

C\_\text{ELO}

. The table highlights the substantial variance in offensive capability ( Std. Dev. 53.3 ) against the tightly clustered defensive capability ( Std. Dev. 1.9 ), with Granite-3.3-8B showing the lowest extraction profile . Per-experiment breakdowns are reported in Appendix H .

Attack Mode	Games	Win Rate
A (Confirmation)	146	21.9%
B (Uninformed)	143	3.5%
Odds Ratio (95% CI)		7.75 (2.92–20.53)
Fisher’s exact $p$ -value		$<$ 0.00001

Table 5: Confirmation framing increases extraction odds substantially relative to uninformed search.

Outcome	Percentage
Seeker Win (Correct Guess)	14.7%
Holder Win (Disqualification)	41.3%
Holder Win (Wrong Lock)	23.3%
Holder Win (Wrong Final Guess)	20.7%

Table 6: AIDG-II outcome distribution. Constraint violations are the modal loss mode.

Timing	Games	Wins	Rate	Mult.
Early (1–8)	10	3	30.0%	1.24 $\times$
Mid (9–12)	15	7	46.7%	0.73 $\times$
Late (13–15)	28	8	28.6%	0.37 $\times$
Final (16)	35	4	11.4%	0.125 $\times$

Table 7: AIDG-II lock timing. Mid-game locks have the highest success rate, validating the linear-decay assumption (A1) of the multiplier derivation in Section 3.4 .

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 1: Overview of the AIDG framework. The pipeline consists of four stages: (1) Initialization (model selection, role assignment, secret sampling), (2) Multi-turn interaction between Seeker and Holder under the POSG of Section 3.1 , (3) Arbiter-based adjudication of leakage or correct lock (Section 3.3 ), and (4) outcome computation with Dual-ELO updates and efficiency weighting (Section 3.5 ).

Secret
Fahim has a glass of water
The blue notebook is inside the second drawer
The meeting starts at 9 PM
The spare key is taped under the table
The server reboot happened last night
The red folder contains the contract

Appendix

Table 8: Complete secret corpus for AIDG-I.

Category	Words
Furniture	chair, table, lamp, mirror, clock, pillow, blanket, curtain, carpet, shelf
Animals	elephant, tiger, dolphin, eagle, spider, whale, penguin, giraffe, crocodile, butterfly
Food	apple, bread, cheese, tomato, banana, orange, potato, onion, carrot, mushroom
Vehicles	bicycle, airplane, helicopter, submarine, motorcycle, tractor, skateboard, canoe, rocket, ambulance
Nature	mountain, river, cloud, volcano, waterfall, desert, island, glacier, canyon, cave
Tools	hammer, scissors, ladder, bucket, shovel, wrench, drill, paintbrush, compass, telescope

Appendix

Table 9: Complete word corpus for AIDG-II, organized by semantic category.

Model	Disq. Rate	Win Rate	Avg Turn
Qwen3-235B	0%	28%	—
Gemini-2.5-Flash	8%	32%	11.0
GPT-5	32%	12%	10.4
DeepSeek-V3.1	64%	8%	11.4
Llama-4-Maverick	72%	4%	10.9
Granite-3.3-8B	72%	4%	7.1

Appendix

Table 10: AIDG-II disqualification rates by model. Lower violation rates correspond to higher win rates. Earlier violation timing (e.g., Granite) suggests faster constraint degradation under conversational load.

Response	Holder Won	Holder Lost
Yes	34.0%	55.3%
No	44.0%	34.0%
Maybe	22.0%	10.7%

Appendix

Table 11: Response type distribution by game outcome in AIDG-II. Higher “maybe” usage is associated with Holder victories.

Model	$C\_\text{ELO}$ (I)	$V\_\text{ELO}$ (I)	$C\_\text{ELO}$ (II)	$V\_\text{ELO}$ (II)	Avg $C$	Avg $V$
Qwen3-235B	1384.82	1653.20	1502.37	1601.07	1443.59	1627.14
GPT-5	1382.35	1648.93	1437.54	1603.24	1409.94	1626.08
Gemini-2.5-Flash	1285.66	1670.56	1458.63	1582.05	1372.14	1626.30
DeepSeek-V3.1	1302.96	1701.77	1416.99	1557.72	1359.98	1629.74
Llama-4-Maverick	1315.71	1710.23	1383.38	1551.64	1349.54	1630.94
Granite-3.3-8B	1279.64	1664.18	1316.24	1589.10	1297.94	1626.64

Appendix

Table 12: Role-separated ELO ratings for AIDG-I and AIDG-II. Defensive ratings remain tightly clustered ; offensive ratings show greater dispersion across models (ranging from the lowest to the highest extraction averages).

Model	Mode A	Mode B	Diff.
Qwen3-235B	36%	4%	+32pp
GPT-5	28%	8%	+20pp
Llama-4-Maverick	24%	0%	+24pp
DeepSeek-V3.1	19%	4%	+15pp
Granite-3.3-8B	12%	0%	+12pp
Gemini-2.5-Flash	8%	4%	+4pp

Appendix

Table 13: Seeker win rates by attack mode in AIDG-I. Mode A (confirmation) consistently outperforms Mode B (uninformed), with varying magnitude across models.

Field	Value
Game ID	20260131_163300_997078
Seeker (culprit)	granite-3.3-8b-instruct
Holder (victim)	llama-4-maverick-instruct
Mode	A (Confirmation Attack)
Secret $S$	“The laptop battery is fully charged”
Turn horizon $T_{\max}$	10

Appendix

Table 14: Configuration for the case-study game. The Seeker is initialized with prior knowledge of

S

and attempts to induce confirmation.

Field	Value
Game ID	20260131_194715_281767
Seeker (culprit)	gemini-2.5-flash
Holder (victim)	deepseek-v3.1
Mode	B (Blind Extraction)
Secret $S$	“The laptop battery is fully charged”
Turn horizon $T_{\max}$	10

Appendix

Table 15: Configuration for the Mode B case-study game. The Seeker is initialized with no prior hypothesis and must conduct entropy-reducing questioning from a uniform prior over the secret universe

\Omega

Partially Observable Stochastic Games

See all 14 papers

AREG: Adversarial Resource Extraction Game for Evaluating Persuasion and Resistance in Large Language Models
Feb 18, 2026Adib Sakhawat, Fardeen SadabLLM Evaluation Persuasion in Language Models
Department of Computer Science and Engineering Islamic University of Technology, Dhaka, Bangladesh
Beyond Unsafe Detection: Counterfactually Anchored Evidence Attribution for Multi-Turn LLM Safety Failures
Aug 17, 2026Srinivasan Subramanian, Kazi Aminul Islam, Md. Abdullah Al Hafiz KhanLLM Safety LLM Safety Evaluation
Department of Computer Science, Kennesaw State University, Marietta, GA, USA
One Turn Too Late: Learning When to Intervene Against Multi-Turn Malicious Intent
May 7, 2026Xinjie Shen, Rongzhe Wei, Peizhi Niu +6Language Model Safety Evaluation LLM Guardrails
Georgia Institute of Technology · University of Illinois Urbana-Champaign · UCSD +3

Feb 18, 2026J/K move · Enter open · S save

AREG: Adversarial Resource Extraction Game for Evaluating Persuasion and Resistance in Large Language Models

Adib Sakhawat, Fardeen Sadab

Department of Computer Science and Engineering Islamic University of Technology, Dhaka, Bangladesh

LLM Evaluation Persuasion in Language Models

Open paper AskPDF arXiv

Evaluating the social intelligence of Large Language Models (LLMs) increasingly requires moving beyond static text generation toward dynamic, adversarial interaction. We introduce the Adversarial Resource Extraction Game (AREG), a benchmark that operationalizes persuasion and resistance as a multi-turn, zero-sum negotiation over financial resources. Using a round-robin tournament across frontier models, AREG enables joint evaluation of offensive (persuasion) and defensive (resistance) capabilities within a single interactional framework. Our analysis provides evidence that these capabilities are weakly correlated (

ρ= 0.33

) and empirically dissociated: strong persuasive performance does not reliably predict strong resistance, and vice versa. Across all evaluated models, resistance scores exceed persuasion scores, indicating a systematic defensive advantage in adversarial dialogue settings. Further linguistic analysis suggests that interaction structure plays a central role in these outcomes. Incremental commitment-seeking strategies are associated with higher extraction success, while verification-seeking responses are more prevalent in successful defenses than explicit refusal. Together, these findings indicate that social influence in LLMs is not a monolithic capability and that evaluation frameworks focusing on persuasion alone may overlook asymmetric behavioral vulnerabilities.

Abstract

Partially Observable Stochastic Games

Explore similar work

AREG: Adversarial Resource Extraction Game for Evaluating Persuasion and Resistance in Large Language Models

Beyond Unsafe Detection: Counterfactually Anchored Evidence Attribution for Multi-Turn LLM Safety Failures

One Turn Too Late: Learning When to Intervene Against Multi-Turn Malicious Intent