Public speaking is an essential skill in academic and professional contexts, but it often causes anxiety. Although several AI-based speech coaching tools exist, they typically provide generic feedback and lack model speeches for learning. This study addresses these gaps by developing a system that incorporates speech context into evaluation criteria, enabling more tailored feedback. It also generates improved model speeches in both text and audio formats to support example-based learning. The system was evaluated in a pilot study with seven undergraduate students over one to three weeks. Participants used the system repeatedly and practiced the same speech at least three times. Results showed improvements in public speaking performance for all participants. Filler word usage decreased, and anxiety levels dropped by 15.3% to 34.4%. Participants found the context-aware feedback and revised speeches useful and confidence-building. These findings suggest that context-aware feedback and model speeches can enhance public speaking skills and reduce anxiety. Future studies could explore video-based analysis of physical behavior during speeches.
Figures & tables
Figure 1: Speech Analysis Pipeline
Participant
Sessions
Initial Overall Score (%)
Final Overall Score (%)
Relative Change (%)
P1
4
57
67
+17.5%
P2
5
46
60
+30.4%
P3
4
62
66
+6.5%
P4
3
62
72
+16.1%
P5
4
30
66
+120.0%
P6
5
57
63
+10.5%
Table 1: CSSEF Overall Score Changes
Participant
Sessions
Initial Filler Words (%)
Final Filler Words (%)
Absolute Change (pp)
Relative Change (%)
P1
4
2.4%
1.0%
−1.4
−58.3 %
P2
5
0.7%
0.0%
−0.7
−100.0 %
P3
4
0.7%
0.0%
−0.7
−100.0 %
P4
3
3.4%
1.3%
−2.1
−61.8 %
P5
4
6.3%
1.7%
−4.6
−73.0 %
P6
5
4.0%
2.7%
−1.3
−32.5 %
Table 2: Change in Filler Word Usage by Participant
Survey Question
Average Rating
The revised text version helped me see how to improve my structure.
4.71
The AI-generated audio version helped me understand pacing and tone.
4.14
Comparing my speech with the AI version improved my reflection.
4.14
The improved versions increased my confidence to try again.
4.57
I learned more effectively by listening to the AI-generated examples.
4.43
Overall, the improved versions accelerated my learning.
4.71
Table 3: Participants Ratings on Usefulness of Revised Speeches
Participant
Initial PRPSA
PSA Level (Pre)
Final PRPSA
PSA Level (Post)
Absolute Change
Percentage Change (%)
P1
115
Moderate
77
Low
−38
−33.0 %
P2
101
Moderate
67
Low
−34
−33.7 %
P3
131
Moderate
86
Low
−45
−34.4 %
P4
86
Low
61
Low
−25
−29.1 %
P5
126
Moderate
87
Low
−39
−31.0 %
P6
124
Moderate
105
Moderate
−19
−15.3 %
Table 4: Individual PRPSA Score (Pre vs Post Intervention)
PSA Level (Pre)
n
Mean PRPSA (Pre)
Mean PRPSA (Post)
Mean Reduction
Percentage Reduction (%)
Low
1
86.0
61.0
−25.0
−29.1 %
Moderate
6
116.5
83.0
−33.5
−28.8 %
Table 5: Grouped PRPSA Results by PSA Level (Pre vs. Post)