PolyDebate: A Game-Orchestrated Multimodal System for Debate Skills Practice and Evaluation
Authors: Jianing Yin, Weng Pan Kuan, Xiaoyun Liu, Zhiyuan Wen, Yuxuan Li, Milos Stojmenovic, Jiannong Cao
Organizations: Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China · Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University, Hong Kong SAR, China · Institute for Higher Education Research and Development, The Hong Kong Polytechnic University, Hong Kong SAR, China · Department of Computer Science and Electrical Engineering, Singidunum University, Belgrade, Serbia
Debate is a structured form of persuasive communication that trains argument construction, rebuttal, oral delivery, and audience awareness. These skills are valued in education, language learning, and professional communication. Recent AI debate systems and LLM-based judges have advanced argument generation and debate evaluation, but most remain text-centered and rarely support learners through a complete multimodal practice experience. We introduce PolyDebate, a game-orchestrated multimodal system for English debate practice and evaluation. PolyDebate guides learners through staged one-on-one (1v1) debates with an AI opponent, while skill cards, props, and coins make persuasive strategies explicit and turn practice into a game-like interaction. During each session, the system captures learner speech and visual delivery evidence, generates context-aware opponent responses, and produces rubric-informed stage-level and overall feedback. PolyDebate is available as both an immersive Unity 3D game version and a web platform version that share the same workflow and evaluation services. Four studies covering AI opponent quality, evaluation coverage, AI judge feedback, and user perception show that PolyDebate brings debate interaction, gamified scaffolding, multimodal assessment, and structured feedback together in a practical workflow for debate skills practice. The demonstration video is available at https://youtu.be/mHwBG1_8Ebk.
Figures & tables
Figure 1: PolyDebate overview for gamified multimodal debate practice and rubric-aligned feedback.
Figure 2: Demonstration walkthrough of a complete 1v1 PolyDebate session in the Unity 3D game version.
Figure 3: Representative key interfaces of the PolyDebate web platform version.
Subjective quality
Objective quality
Skill usage
Variant
EA
AC
AA
TR
Avg.
FA
LV
Avg.
Pres.
Corr.
Vis.
Avg.
Overall
Generic LLM opponent
3.2
4.0
3.2
3.8
3.6
3.6
3.9
3.8
2.0
1.8
2.6
2.1
3.1
Stage-only opponent
3.7
4.0
3.7
4.5
4.0
3.7
3.9
3.8
2.8
2.6
3.2
2.9
3.6
PolyDebate opponent
3.8
4.0
4.3
4.6
4.2
3.8
4.0
3.9
4.2
3.5
4.1
3.9
4.0
Table 1: AI opponent next-turn response quality. Scores use a 1–5 scale.
Modality
Analysis
Persuasiveness
Clarity
Appropriacy
Feedback output
Framework
T
A
V
Arg.
Opp.
Pathos
Strat.
Org.
Pron.
Gram.
Task
Ethos
Tone
Str.
Weak.
Rec.
Debatrix
✓
✓
△
△
△
△
△
△
△
InspireScore
✓
✓
△
✓
Debatable Intelligence
✓
△
△
△
△
△
MA Debate Chatbot (CT)
✓
✓
△
△
△
△
△
△
PolyDebate
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
✓
Table 2: Comparison of representative debate and debate-context assessment frameworks in terms of modality, assessed aspects, and learner-facing feedback output.
Variant
W.Cov.
W.F1
Spec.
Act.
Grd.
Generic LLM feedback judge
46.7
80.6
4.0
4.1
4.1
Full w/o rubrics
51.3
64.3
4.4
4.5
4.7
Full w/o skill card
81.7
59.6
4.4
4.4
4.6
Full w/o multimodal evidence
71.7
32.2
4.2
4.2
4.1
Full w/o feedback schema
76.2
74.8
4.8
4.8
4.8
Full PolyDebate judge
99.2
85.9
4.9
4.9
4.9
Table 3: AI judge feedback quality and component ablation. Coverage and weakness F1 are percentages; specificity, actionability, and groundedness use a 1–5 scale.
Figure 4: User-perception results for the Unity 3D and web platform versions of PolyDebate. Bars report mean 5-point Likert ratings from ten students across six questionnaire dimensions.