PolyDebate: A Game-Orchestrated Multimodal System for Debate Skills Practice and Evaluation
Authors: Jianing Yin, Weng Pan Kuan, Xiaoyun Liu, Zhiyuan Wen, Yuxuan Li, Milos Stojmenovic, Jiannong Cao
Organizations: Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China · Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong SAR, China · Institute for Higher Education Research and Development, The Hong Kong Polytechnic University, Hong Kong SAR, China · Department of Computer Science and Electrical Engineering, Singidunum University, Belgrade, Serbia
Debate is a structured form of persuasive communication that trains argument construction, rebuttal, oral delivery, and audience awareness. These skills are valued in education, language learning, and professional communication. Recent AI debate systems and LLM-based judges have advanced argument generation and debate evaluation, but most remain text-centered and rarely support learners through a complete multimodal practice experience. We introduce PolyDebate, a game-orchestrated multimodal system for English debate practice and evaluation. PolyDebate guides learners through staged one-on-one (1v1) debates with an AI opponent, while skill cards, props, and coins make persuasive strategies explicit and turn practice into a game-like interaction. During each session, the system captures learner speech and visual delivery evidence, generates context-aware opponent responses, and produces rubric-informed stage-level and overall feedback. PolyDebate is available as both an immersive Unity 3D game version and a web platform version that share the same workflow and evaluation services. Four studies covering AI opponent quality, evaluation coverage, AI judge feedback, and user perception show that PolyDebate brings debate interaction, gamified scaffolding, multimodal assessment, and structured feedback together in a practical workflow for debate skills practice. The demonstration video is available at https://youtu.be/mHwBG1_8Ebk.
Figures & tables
Figure 1: PolyDebate overview for gamified multimodal debate practice and rubric-aligned feedback.
Figure 2: Demonstration walkthrough of a complete 1v1 PolyDebate session in the Unity 3D game version.
Figure 3: Representative key interfaces of the PolyDebate web platform version.
Subjective quality
Objective quality
Skill usage
Variant
EA
AC
AA
TR
Avg.
FA
LV
Avg.
Pres.
Corr.
Vis.
Avg.
Overall
Generic LLM opponent
3.2
4.0
3.2
3.8
3.6
3.6
3.9
3.8
2.0
1.8
2.6
2.1
3.1
Stage-only opponent
3.7
4.0
3.7
4.5
4.0
3.7
3.9
3.8
2.8
2.6
3.2
2.9
3.6
PolyDebate opponent
3.8
4.0
4.3
4.6
4.2
3.8
4.0
3.9
4.2
3.5
4.1
3.9
4.0
Table 1: AI opponent next-turn response quality. Scores use a 1–5 scale.
Modality
Analysis
Persuasiveness
Clarity
Appropriacy
Feedback output
Framework
T
A
V
Arg.
Opp.
Pathos
Strat.
Org.
Pron.
Gram.
Task
Ethos
Tone
Str.
Weak.
Rec.
Debatrix
✓
✓
△
△
△
△
△
△
△
InspireScore
✓
✓
△
✓
Debatable Intelligence
✓
△
△
△
△
△
MA Debate Chatbot (CT)
✓
✓
△
△
△
△
△
△
PolyDebate
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
Table 2: Comparison of representative debate and debate-context assessment frameworks in terms of modality, assessed aspects, and learner-facing feedback output.
Variant
W.Cov.
W.F1
Spec.
Act.
Grd.
Generic LLM feedback judge
46.7
80.6
4.0
4.1
4.1
Full w/o rubrics
51.3
64.3
4.4
4.5
4.7
Full w/o skill card
81.7
59.6
4.4
4.4
4.6
Full w/o multimodal evidence
71.7
32.2
4.2
4.2
4.1
Full w/o feedback schema
76.2
74.8
4.8
4.8
4.8
Full PolyDebate judge
99.2
85.9
4.9
4.9
4.9
Table 3: AI judge feedback quality and component ablation. Coverage and weakness F1 are percentages; specificity, actionability, and groundedness use a 1–5 scale.
Figure 4: User-perception results for the Unity 3D and web platform versions of PolyDebate. Bars report mean 5-point Likert ratings from ten students across six questionnaire dimensions.
Multi-agent debate can improve large language model reasoning by eliciting diverse hypotheses and critiques, yet its performance is often constrained by weak moderation. Common pipelines rely on fixed budgets, agreement-based stopping, or untrained judges, leading to redundant deliberation and unreliable evidence aggregation. We cast moderation as a meta-cognitive process, monitoring debate utility, controlling deliberation, and adjudicating a final answer, and introduce Meta-Moderator, a learnable framework that dynamically regulates debate and decides when to finalize an answer. Meta-Moderator is trained independently of the debaters via outcome-driven policy optimization, making debate regulation an explicit capability rather than an incidental effect of prompting. Across five benchmarks, Meta-Moderator outperforms widely used decision layers and transfers across tasks and system configurations. Further analyses show that it allocates debate more selectively and reduces mis-aggregation after informative hypotheses appear.
Wentao Hu, Zhuoyue Wan, Jinhao Shen +3
The Hong Kong Polytechnic University · Sichuan University
Debate adjudication requires tracking how arguments develop through interaction, yet existing datasets rarely combine fine-grained debate transcripts with professional judgments collected during real competitions under a shared rubric. We introduce a dataset and benchmark for evaluating large language models' understanding of competitive Chinese-language debate at the match, stage, and speaker levels. We organized 182 matches and recruited 120 professional judges, with each match independently adjudicated by three judges using a predefined rubric. After excluding matches with incomplete records, the dataset contains 148 matches, 2,698 stages, and 20,542 exchange units, with manually verified transcripts and segmentation. It preserves original stage scores, match votes, best-debater ballots, and adjudication rationales. We define three tasks: winner-tendency prediction, stage-score prediction, and best-debater prediction. Zero-shot evaluation of multiple large language models yields a highest winner-prediction accuracy of 66.2%, a highest Pearson correlation of 0.250 between model stage scores and mean human ratings, and a highest best-debater prediction accuracy of 56.8%. The dataset and benchmark provide a testbed for studying large language models' understanding of interactive argumentation and their agreement with professional judges.
Zongrui Yang, Haoyuan Li, Zhongsheng Wang +5
University of Auckland, New Zealand · South China Agricultural University, China · Sanming University, China
Multi-agent debate systems are typically evaluated only on whether the final answer is correct, overlooking the quality of the intermediate reasoning that debate is designed to produce. This paper studies the relationship between three signals in multi-agent debate: token-level log-probability distributions over reasoning tokens, LLM-as-judge rubric scores assigned to those tokens, and final task accuracy. We examine whether internal confidence signals predict externally evaluated reasoning quality, and whether either signal aligns with task correctness, across three domains: rubric-based scoring, mathematical reasoning, and factual question answering. Our framework pairs a two-agent debate architecture -- a Constructor and an Auditor -- with an LLM-as-judge that scores each agent's reasoning along instruction following, justification quality, and evidence grounding, together with a critical-failure flag. Experiments in the rubric-scoring domain reveal a consistent four-phase confidence trajectory and a substantial role asymmetry: confidence aligns with judged reasoning quality roughly twice as strongly for the Constructor as for the Auditor, and confidence-based detection of critical reasoning failures is markedly more reliable for the Constructor (AUROC 0.804) than for the Auditor (0.634). These findings motivate the broader cross-domain investigation proposed in this paper.