Key Point Analysis (KPA) aims to identify a concise set of key points that summarize a collection of arguments together with their prevalence. We argue that KPA is fundamentally a structured prediction problem that requires recovering semantic groupings, generating representative key points, ensuring coverage, and estimating prevalence. Under this formulation, we show that existing KPA benchmarks suffer from limitations in grouping quality, redundancy, coverage, and argument-key point mappings, causing ceiling violation and selection failure in reference-based evaluation. To support future research on true KPA, we introduce a structure-aware, distribution-sensitive benchmark built via a human-in-the-loop re-annotation. Human and LLM evaluations consistently show that the resulting structures yield more coherent groupings, higher-quality key points, better coverage, and more reliable prevalence estimates than existing annotations. We further release several annotation resources to support research on KPA evaluation, argument-key point matching, explainable KPA, and LLM-as-a-judge methodologies, and outline a research agenda for true KPA.
Figures & tables
Topic
Global Precision
Abstraction Score
GT
LLM
GT
LLM
Topic 0
0.591
0.717
2.78
3.93
Topic 1
0.609
0.899
3.50
3.86
Topic 2
0.832
0.930
3.86
3.57
Overall
0.683
0.857
3.45
3.73
Table 1: Results for semantic grouping and KP quality.
Figure 1: Example of redundancy annotation. KPs A1, A3, and A4 (blue) express similar reasoning and are consolidated under A2 (“saving lives”), while A5 represents a distinct semantic aspect.
Figure 2: Two independently sampled argument subsets from the same topic produce different KPA structures. Blue nodes denote themes shared across both subsets, while red and green nodes represent distribution-specific themes.
Judge
ArgKP21
Reannotated
GLM 5
8.53
19.80
DeepSeek V3.2
9.53
19.40
Table 2: Average overall scores across all five subsets.
Structure
Grouping
KP Quality
Coverage
ArgKP21-derived
3.00
3.07
2.60
Reannotated
4.73
4.60
4.80
Table 3: Average human evaluation scores across all subsets.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Judge
Structure
Grouping
KP Gen.
Coverage
Prevalence
Overall
GLM
ArgKP21
2.07
2.87
1.53
2.07
8.53
Reannotated
5.00
5.00
4.80
5.00
19.80
DeepSeek V3.2
ArgKP21
2.67
2.73
1.93
2.20
9.53
Reannotated
4.93
4.93
4.60
4.93
19.40
Appendix
Table 4: Average dimension-level LLM evaluation scores across all five subsets.
Subset
GLM (ArgKP21)
GLM (Reannotated)
DeepSeek (ArgKP21)
DeepSeek (Reannotated)
Subset 0
8.33
19.67
8.67
19.33
Subset 1
9.00
19.67
9.67
19.67
Subset 2
8.67
20.00
10.00
18.67
Subset 3
8.33
19.67
10.00
19.33
Subset 4
8.33
20.00
9.33
20.00
Appendix
Table 5: Overall LLM evaluation scores for individual subsets. Both judges preferred the reannotated structures on all five subsets.
Semantic Grouping
KP Quality
Coverage
Subset
ArgKP21
Reann.
ArgKP21
Reann.
ArgKP21
Reann.
Subset 0
2.67
4.67
2.67
4.33
2.00
5.00
Subset 0-2
3.33
4.33
2.67
4.67
2.67
5.00
Subset 1
3.33
4.67
3.00
4.33
2.67
5.00
Subset 1-2
3.33
4.33
2.67
4.67
2.67
5.00
Subset 2
2.00
4.33
3.00
4.33
2.33
4.33
Appendix
Table 6: Detailed human evaluation results for each subset.
Statistic
Value
Topics
3
Subsets
15
Avg. Arguments per Subset
39.27
Median Arguments per Subset
40
Min Arguments per Subset
32
Max Arguments per Subset
48
Appendix
Table 7: Summary statistics of the benchmark. Statistics are reported at the subset level, which is the unit of evaluation in our benchmark.
Figure 3: Distribution of semantic group sizes in the benchmark. A semantic group consists of a set of semantically related arguments summarized by a key point. Most semantic groups contain between three and six arguments, while a small number of larger groups contain up to twelve arguments.
Statistic
Value
Examples
126
Topics
3
Avg. Candidate Arguments per Example
4.79
Median Candidate Arguments per Example
4
Min Candidate Arguments per Example
1
Max Candidate Arguments per Example
13
Appendix
Table 8: Summary statistics of the semantic group verification dataset. Selected arguments correspond to candidate arguments judged to belong to the semantic group represented by the key point.
Stance
Examples
Pro
63
Con
63
Appendix
Table 9: Key-point stance distribution of the semantic group verification dataset.
Quality
Count
Excellent
84
Good
37
Fair
2
Poor
3
Appendix
Table 10: Distribution of key-point quality labels in the semantic group verification dataset.
Statistic
Value
Examples
125
Topics
3
Assigned to at least one key point
100
No suitable key point ( none )
25
Match Rate (%)
80.0
Appendix
Table 11: Statistics of the unmatched argument reassignment dataset.
Model
Identifier
Qwen 3 235B
qwen/qwen3-235b-a22b-2507
GLM 5
z-ai/glm-5
DeepSeek V3.2
deepseek/deepseek-v3.2
Appendix
Table 12: Identifier for the models we use in the prompting experiments.