To fully capture the meaning of a sentence, semantic representations should encode aspect, which describes the internal temporal structure of events. In graph-based meaning representation frameworks such as Uniform Meaning Representations (UMR), aspect lets one know how events unfold over time, including distinctions such as states, activities, and completed events. Despite its importance, aspect remains sparsely annotated across semantic meaning representation frameworks. This has, in turn, hindered not only current manual annotation, but also the development of automatic systems capable of predicting aspectual information. In this paper, we introduce a new dataset of English sentences annotated with UMR aspect labels over Abstract Meaning Representation (AMR) graphs that lack the feature. We describe the annotation scheme and guidelines used to label eventive predicates according to the UMR aspect lattice, as well as the annotation pipeline used to ensure consistency and quality across annotators through a multi-step adjudication process. To demonstrate the utility of our dataset for future automation, we present baseline experiments using three modeling approaches. Our results establish initial benchmarks for automatic UMR aspect prediction and provide a foundation for integrating aspect into semantic meaning representations more broadly.
Figures & tables
Figure 2: UMR aspect lattice with aspectual values utilized in our English annotation highlighted in yellow.
Per-class
Label
Kα∗
Count
None
0.735
514
State
0.706
385
Performance
0.716
360
Process
0.547
100
Habitual
0.704
56
Table 1: Aspect label distribution and inter-annotator agreement metrics. *Per-class Krippendorff’s alpha computed as one-vs-rest. † Fleiss’ κ restricted to the 333 events that received first-pass tie-breaks.
Sentence
Event
Main label
2nd label
(a) <Architect> now above that place red blocks on the grid.
Place
Performance
Process
(b) "Clap your hands, one against the other," the conceited man now directed him.
Clap
Endeavor
Performance
(c) Greed is when you are wealthy and lobby your representatives for special tax breaks because you are over 60 years of age.
Age
State
N/A
Table 2: Selected examples with ambiguous aspect; some annotated with secondary labels.
Type
Model
Acc.
Macro F1
Wtd. F1
Precision
Recall
LLM
LLaMA-3.1-8B-Instruct (zero-shot)
0.31
0.19
0.27
0.29
0.24
LLaMA-3.1-8B-Instruct (3-shot)
0.25
0.16
0.22
0.32
0.21
GPT-5mini (zero-shot)
0.56
0.49
0.56
0.69
0.49
GPT-5mini (3-shot)
0.56
0.46
0.60
0.49
0.46
Neural
Feedforward MLP
0.45
0.27
0.44
0.29
0.32
Symbolic
AutoAspect †
0.39
0.23
0.40
—
—
Table 3: Baseline results on the test split (254 events, 72 sentences). Human performance reflects first-pass annotator accuracy against adjudicated gold labels. Precision and Recall are macro averages across classes. † AutoAspect results are taken from the original publication and are not directly comparable.
Aspect
Little Prince
Minecraft
BOLT DF
WB
Pear Story
Lorelei
UMR 1.0
Total
State
63
20
119
45
121
0
62
430
Habitual
1
0
17
4
28
0
2
52
Process
2
0
5
5
44
1
1
58
Activity
9
0
31
14
57
0
21
132
Performance
35
2
43
18
159
3
57
317
Endeavor
0
0
0
0
2
0
14
16
Table 4: Aspect label distribution from existing UMR data before any additional annotation was done.
Aspect
Little Prince
Minecraft
BOLT DF
WB
Existing Labels
Total
State
172
14
101
14
430
731
Habitual
41
0
4
0
52
97
Process
31
0
38
8
58
135
Activity
15
2
10
3
132
162
Performance
163
43
69
32
317
624
Endeavor
15
0
4
0
16
35
Table 5: Label distribution by corpus and annotated aspect. We report Fleiss’ Kappa between the two initial annotators and do not include disagreements in the reported total.
Category
Metric
Value
Accuracy
State
0.82
Habitual
0.63
Activity
0.52
Performance
0.80
Endeavor
0.11
Overall Accuracy
0.74
Table 6: Practice Annotation Results: Overall Accuracy is the ratio of the total number of correct annotations over the total number of predicates annotated. Perfect Accuracy is the ratio of predicates that were correctly annotated by all annotators. No occurrences of Process aspects were present in the practice set.
Split
Sentences
Events
Train
333
,0 999
Dev
71
,0 220
Test
72
,0 254
Total
476
1,473
Table 7: Stratified 70/15/15 train/dev/test split, divided at the sentence level using dominant aspect label for stratification.
Gold label
Error rate
Error Label
Activity
6.7%
Performance
Endeavor
12.2%
Performance
Habitual
6.8%
Performance
None
6.0%
State
Performance
3.9%
None
Process
9.1%
None
Table 8: Final gold labels and the percent for each that was incorrectly annotated then fixed during adjudication out of the total number of annotations for that label. The error label is the most common incorrect annotation.
Figure 3: Flow diagram illustrating the annotation and adjudication phases, and how many labels were completed at each stage of the process.
Comparison and evaluation of graph-based representations of sentence meaning is a challenge because competing representations of the same sentence may have different number of nodes, and it is not obvious which nodes should be compared to each other. Existing approaches favor node mapping that maximizes F1 score over node relations and attributes, regardless whether the similarity is intentional or accidental; consequently, the identified mismatches in values of node attributes are not useful for any detailed error analysis. We propose a node-matching algorithm that allows comparison of multiple Uniform Meaning Representations (UMR) of one sentence and that takes advantage of node-word alignments, inherently available in UMR. We compare it with previously used approaches, in particular smatch (the de-facto standard in AMR evaluation), and argue that sensitivity to word alignment makes the comparison of meaning representations more intuitive and interpretable, while avoiding the NP-hard search problem inherent in smatch. A script implementing the method is freely available.
Daniel Zeman, Federica Gamba
Charles University Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics (ÚFAL) Prague, Czechia
In this paper, we present AILS-NTUA system for Track-A of SemEval-2026 Task 3 on Dimensional Aspect-Based Sentiment Analysis (DimABSA), which encompasses three complementary problems: Dimensional Aspect Sentiment Regression (DimASR), Dimensional Aspect Sentiment Triplet Extraction (DimASTE), and Dimensional Aspect Sentiment Quadruplet Prediction (DimASQP) within a multilingual and multi-domain framework. Our methodology combines fine-tuning of language-appropriate encoder backbones for continuous aspect-level sentiment prediction with language-specific instruction tuning of large language models using LoRA for structured triplet and quadruplet extraction. This unified yet task-adaptive design emphasizes parameter-efficient specialization across languages and domains, enabling reduced training and inference requirements while maintaining strong effectiveness. Empirical results demonstrate that the proposed models achieve competitive performance and consistently surpass the provided baselines across most evaluation settings.
Stavros Gazetas, Giorgos Filandrianos, Maria Lymperaiou +3
School of Electrical and Computer Engineering, AILS Laboratory National Technical University of Athens
Aspect Sentiment Quadruple Prediction (ASQP) is fundamentally a relational reasoning problem: identifying aspect-opinion-category-sentiment quadruples requires modeling structured dependencies, not merely sequential co-occurrence. Existing LLM-based approaches incorporate syntactic information indirectly, via retrieval-based demonstration selection or multi-stage training, rather than as unified, single-pass generation-time context. We present DRIFT, which linearizes dependency parse trees and injects them directly into LLM prompts as explicit relational context, enabling the model to reason over aspect-opinion dependencies rather than infer them from token proximity alone. DRIFT further integrates retrieval-augmented in-context learning and a syntax-aware LLM-based adjudication mechanism that complements exact-match evaluation under span-boundary variation. On SemEval Rest15, Rest16, and ACOS, DRIFT achieves the strongest performance among compared generative ASQP methods, reaching 54.47% and 64.10% F1 on Rest15 and Rest16, with consistent gains across short- and medium-distance aspect-opinion configurations; adjudication further shows exact-match metrics systematically underestimate semantically correct predictions.