Game of Phones: Harnessing Game Theory and LLMs in 'Bot Wars' to Counteract Phone Scams

Nardine Basta; Robin Carpentier; Benjamin Zi Hao Zhao; Dali Kaafar

cs.CROct 19, 2026

Game of Phones: Harnessing Game Theory and LLMs in 'Bot Wars' to Counteract Phone Scams

Authors: Nardine Basta, Robin Carpentier, Benjamin Zi Hao Zhao, Dali Kaafar

Organizations: Macquarie University, NSW, Australia

Abstract

Phone scams impose substantial and growing costs on individuals and financial systems worldwide, yet existing countermeasures impose no resource cost on perpetrators. Strategic scam-baiting, in which automated agents engage scammers to deplete their operational capacity, offers a proactive alternative but requires dynamic strategy adaptation that static systems cannot provide. We introduce a game-theoretic framework enabling LLMs to learn effective counter-strategies through in-context experience without fine-tuning. The framework models scammer-baiter interactions as a two-player game with empirically grounded strategies and utility functions, using Nash equilibrium as a formal benchmark for evaluating learned behaviour. Across 300 simulated dialogues and three LLM architectures, adaptive strategy selection achieves 28.9-79.7% closer alignment to Nash equilibrium than non-adaptive baselines. The code and an annotated dataset of 300 dialogues with turn-level strategy labels and utility scores are released at https://github.com/nardinebasta/GameofPhones_/ .

Figures & tables

Table 1: Overview of strategies. Full descriptions are provided in appendix 0.B

Panel of Figure 1 — Figure 1: System architecture and information flow during a turn. Data from the current game and 20 most recent games are processed to select a strategy. Message Generators translate selected strategies into contextually appropriate dialogue. The Dialogue Analyzer extracts PII, computes utilities, and stores the data. Dashed arrows indicate feedback loops enabling learning across games.

Model	# Games	Total # Turns	Avg # Turns	Scammer Wins
GPT-4	100	3,500	35.0	49%
Mixtral	100	2,900	29.0	88%
DeepSeek	100	4,200	42.0	41%
Total	300	10,247	34.2	59.3%

Table 2: Dataset statistics by model.

$t$	$S_{i}$	$B_{j}$	PII Exposed	Baiter Response	FPII	$U_{S}$
0	$S_{8}$	$B_{9}$	Name	“Can you verify which account?”	0.00	0.25
5	$S_{1}$	$B_{1}$	Name, Other	“I’m driving. Can I pull over?”	0.00	6.34
20	$S_{9}$	$B_{8}$	Name, Card, Expiry	“Expiry is 05/25. Why the code?”	0.50	49.90
23	$S_{1}$	$B_{3}$	Name, Card, Expiry	“My phone is at 1% now.”	0.50	61.77
24	$S_{9}$	$B_{8}$	Full FPII	“I think it’s 2-8-3?”	1.00	85.92

Table 3: Annotated excerpt from a representative simulated dialogue. FPII tracks cumulative financial credential exposure;

U_{S}

denotes cumulative scammer utility.

Model	Static	Adaptive†	Static	Adaptive†	Static	Adaptive†
	Avg. # Turns		Full FPII		Scammer Wins
GPT-4	35	35	38%	11%	N/A	49%
Mixtral	21	29	42%	24%	N/A	88%
DeepSeek	23	42	49%	13%	N/A	41%

Table 4: Static prompting baseline [ 3 ] (estimated from their Figure 2 and Table 3) vs. our adaptive framework(†). Full FPII denotes the proportion of games where complete FPII has been acquired by the scammer. Scammer wins denotes the proportion of games won by the scammer (N/A for [ 3 ] as not using game theory).

	Baiter Strategy (%)										Scammer Strategy (%)
	$B_{1}$	$B_{2}$	$B_{3}$	$B_{4}$	$B_{5}$	$B_{6}$	$B_{7}$	$B_{8}$	$B_{9}$	$B_{10}$	$S_{1}$	$S_{2}$	$S_{3}$	$S_{4}$	$S_{5}$	$S_{6}$	$S_{7}$	$S_{8}$	$S_{9}$	$S_{10}$
GPT-4	34	5.6	11	4.3	14	0.9	2.7	25	2.8	0.1	19	3.9	2.7	0.1	5.3	2.8	22	3.9	39	<0.1
Mixtral	27	6.1	8.4	9.2	9.1	8.7	8.2	7.9	8.6	7.1	1.7	30	0.7	—	2.0	0.2	60	4.6	—	<0.1
DeepSeek	1.1	0.3	4.3	70	0.9	0.1	0.2	0.9	22	0.1	1.1	1.1	—	0.1	4.9	25	64	1.8	1.7	<0.1

Table 5: Empirical strategy selection frequencies (%) by model.

	Baiter Nash Strategy (%)										Scammer Nash Strategy (%)
	$B_{1}$	$B_{2}$	$B_{3}$	$B_{4}$	$B_{5}$	$B_{6}$	$B_{7}$	$B_{8}$	$B_{9}$	$B_{10}$	$S_{1}$	$S_{2}$	$S_{3}$	$S_{4}$	$S_{5}$	$S_{6}$	$S_{7}$	$S_{8}$	$S_{9}$	$S_{10}$
GPT-4	12	8.6	2.5	5.7	54	—	—	10	6.7	—	—	—	—	—	19	—	64	—	—	18
Mixtral	29	0.7	0.4	11	24	1.4	—	19	5.8	8.3	—	—	—	—	—	—	100	—	—	<0.1
DeepSeek	13	—	—	72	—	—	3.8	—	11	—	100	—	—	—	—	—	—	—	—	<0.1

Table 6: Nash equilibrium strategy distributions (%) by model.

Model	Method	JS Div. $\downarrow$	Cosine Sim. $\uparrow$
GPT-4	Random Uniform	0.296	0.546
	Majority ( $B_{1}$ )	0.726	0.206
	Adaptive†	0.206	0.566
Mixtral	Random Uniform	0.222	0.706
	Majority ( $B_{1}$ )	0.505	0.645
	Adaptive†	0.158	0.845

Table 7: Proximity of strategy selection methods to Nash equilibrium. JS divergence and cosine similarity are computed against the equilibrium distributions of table 6 . † Our method. Best results in bold .

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Scammer Strategies
S1: Urgency	Apply artificial time constraints to force expedited decision-making processes.
S2: Authority	Assert false power or authority to compel compliance through institutional intimidation.
S3: Emotional Manipulation	Deliberate evocation of emotional responses to influence cognitive processing and decision-making.
S4: Incentive	Present purported rewards or benefits to motivate cooperative behavior.
S5: Information Extraction	Systematic acquisition of information through a series of seemingly innocuous requests.
S6: Technical Jargon	Use of complex terminology and concepts to impair victim critical thinking capabilities.

Appendix

Table 8: Strategy spaces for scammer and baiter. Strategies are derived from established literature on scam techniques and persuasion tactics.

Game-Playing Agents

Game-playing agents use search, learning, and reasoning to choose actions in games with strategic, uncertain, or changing environments.

See all 206 papers

PreScam: A Benchmark for Predicting Scam Progression from Early Conversations
May 12, 2026Weixiang Sun, Shang Ma, Yiyang Li +5LLM Evaluation Cybersecurity
University of Notre Dame · Arizona State University
Anatomy of a Scam Call: What 10,000 real scam and spam calls reveal about how phone scammers operate
Aug 25, 2026Ethan Traister, Ankit Raj, Jiaqi Gan +7Financial Fraud Detection
Used Car Salesbots? Honesty and Credulity of LLMs as Bargaining Agents under Partial Information
May 29, 2026Antonio Valerio Miceli-Barone, Vaishak Belle, Shay B. CohenMulti-Agent LLM Systems Multi-Agent Negotiation
University of Edinburgh

May 12, 2026J/K move · Enter open · S save

PreScam: A Benchmark for Predicting Scam Progression from Early Conversations

Weixiang Sun, Shang Ma, Yiyang Li, Tianyi Ma+4

University of Notre Dame · Arizona State University

LLM Evaluation Cybersecurity

Open paper AskPDF arXiv

Conversational scams, such as romance and investment scams, are emerging as a major form of online fraud. Unlike one-shot scam lures such as fake lottery or unpaid toll messages, they unfold through multi-turn conversations in which scammers gradually manipulate victims using evolving psychological techniques. However, existing research mainly focuses on static scam detection or synthetic scams, leaving open whether language models can understand how real-world scams progress over time. We introduce PreScam, a benchmark for modeling scam progression from early conversations. Built from user-submitted scam reports, PreScam filters and structures 177,989 raw reports into 11,573 conversational scam instances spanning 20 scam categories. Each instance is hierarchically structured according to the scam lifecycle defined by the proposed scam kill chain, and further annotated at the turn level with scammer psychological actions and victim responses. We benchmark models on two tasks: real-time termination prediction, which estimates whether a conversation is approaching the termination stage, and scammer action prediction, which forecasts the scammer's subsequent actions. Results show a clear gap between surface-level fluency and progression modeling: supervised encoders substantially outperform zero-shot LLMs on real-time termination prediction, while next-action prediction remains only moderately successful even for strong LLMs. Taken together, these results show that current models can capture some scam-related cues, yet still struggle to track how risk escalates and how manipulation unfolds across turns.

Abstract

Game-Playing Agents

Explore similar work

PreScam: A Benchmark for Predicting Scam Progression from Early Conversations

Anatomy of a Scam Call: What 10,000 real scam and spam calls reveal about how phone scammers operate

Used Car Salesbots? Honesty and Credulity of LLMs as Bargaining Agents under Partial Information