CrisisKD: Five-Stage Knowledge Distillation for Aspect-Level Sentiment and Emotion Analysis in Crisis Discourse
Authors: Marko Haralović, Onat Akca, Salih Eren Yücetürk, Minsi Li, Mariët Theune
Organizations: Faculty of Electrical Engineering, Mathematics and Computer Science, University of Twente · Faculty of Electrical Engineering and Computing, University of Zagreb · Faculty of Behavioural, Management and Social Sciences, University of Twente
Identifying the target of emotional words or phrases in crisis situations, especially health-related ones, is important for understanding public concerns across cultural and linguistic contexts. We propose CrisisKD, a five-stage teacher--student knowledge distillation framework for aspect-level sentiment and emotion analysis on unannotated social media data. A teacher LLM generates aspect-level labels and reasoning traces that supervise a smaller student model across aspect extraction, syntactic parsing, opinion extraction, sentiment classification, and emotion classification. Using this framework, we construct and release a dataset containing 50,615 aspect-level labels, together with the annotation and fine-tuning scripts as open-source resources. The resulting student supports end-to-end ABSA and emotion detection at substantially lower inference cost than the teacher. On a manually annotated 500-tweet gold set, the 5-task Qwen2.5-7B student improves over the untuned model by 7.9 F1 points on aspect extraction, 17.0 points on emotion accuracy, and 6.5 points on sentiment accuracy. On the external ABEA benchmark, CrisisKD improves the same-model Qwen2.5-7B ICL baseline by 2.8 F1 points on ATE and 3.8 F1 points on joint ATE+AEC.
Figures & tables
Figure 1: System overview of the proposed teacher–student framework. Each input sentence is parsed by spaCy into CoNLL-U format and, together with the raw sentence, is fed to a frozen teacher LLM ( θT∗ ). The teacher produces multi-task reasoning traces R1 – R5 , which serve as knowledge-distillation targets for the student LLM (LLaMA-3-8B-Instruct or Qwen2.5-7B-Instruct, fine-tuned with LoRA). Task-specific generative losses L1…L5 are averaged into LG .
Model
Scenario
Aspect Extraction
Emotion Cls.
Sentiment Cls.
Prec.
Rec.
F1
Prec.
Rec.
F1
Acc.
Prec.
Rec.
F1
Acc.
Llama-3-8B-Instruct
No fine-tuning
0.341
0.515
0.411
0.566
0.635
0.515
0.733
0.876
0.909
0.891
0.911
Fine-tune (3 tasks, w/o opinions and syntax)
0.654
0.643
0.648
0.699
0.656
0.669
0.850
0.935 ∗
0.912 ∗
0.920 ∗
0.929
Fine-tune (4 tasks, w/o opinion extraction)
0.653
0.626
0.639
0.710
0.640
0.649
0.836
0.933
0.906
0.918
0.927
Fine-tune (4 tasks, w/o syntactic parsing)
0.629
0.646
0.638
0.724 ∗
0.657
0.675
0.852
0.933
0.899
0.915
0.926
Fine-tune (5 tasks)
0.649
0.622
0.635
0.698
0.640
0.638
0.858
0.933
0.906
0.918
0.927
Table 1: Student–teacher fidelity across training scenarios. We report precision, recall, F1, and accuracy for sentiment and emotion classification, and precision, recall, and F1 for aspect extraction for each model against teacher outputs. Bold values indicate the best performance for a model, and ∗ denotes the best performance overall.
Task
Model
Acc.
F1
Aspect Extraction
Qwen2.5-7B Base
–
0.549
Qwen2.5-7B FT (3)
–
0.583
Qwen2.5-7B FT (5)
–
0.628
Teacher: Qwen2.5-32B
–
0.709
Emotion Classification
Majority class
0.551
0.051
Uniform random
0.305
0.077
Table 2: Performance on the manually annotated gold set. We report accuracy and macro-F1 where applicable. Majority-class and uniform-random baselines are included for the classification tasks.
Task
Model
Rec.
Prec.
F1
ATE
Qwen3-14B ICL
0.468
0.427
0.447
Qwen2.5-7B ICL baseline
0.303
0.529
0.385
Ours: Qwen2.5-7B
0.359
0.486
0.413
ATE + AEC
Qwen3-14B ICL
0.358
0.327
0.342
Qwen2.5-7B ICL baseline
0.179
0.313
0.228
Ours: Qwen2.5-7B
0.231
0.313
0.266
Table 3: Performance on the ABEA test set. ATE denotes Aspect Term Extraction, AEC denotes Aspect Emotion Classification, and ICL denotes in-context learning. We compare our Qwen2.5-7B model against a same-size ICL baseline and a larger Qwen3-14B ICL model as an upper-bound reference.
Emotion
Top aspects (log-enrichment)
Proud
doctor (4.04), people (1.69), vaccine (1.48), quarantine (1.47), shopping (1.45)
Trustful
doctor (3.24), hand sanitizer (2.53), World Health Organization (2.52), mask (2.38), government response (1.89)
Optimistic
shopping (3.14), health (1.75), cure (1.74), food (1.68), price (1.66)
Thankful
doctor (3.13), health (2.48), news (2.26), grocery/supermarket/store (1.98), shopping (1.82)
Empathetic
doctor (3.06), people (2.43), food (2.15), health (2.10), shopping (1.78)
Hopeful
cure (3.04), vaccine (2.48), news (1.48), shopping (1.30), world (1.08)
Table 4: Top aspects per emotion ranked by log-enrichment, defined as log(P(emotion∣aspect)/P(emotion)) , over the top 50 aspects (minimum support n≥200 ). Positive values indicate that the emotion is more common for that aspect than in the dataset overall, negative values indicate underrepresentation, and 0 indicates baseline-level prevalence. Rows are ordered by the strongest top-ranked log-enrichment value in each emotion category.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: Reasoning example for aspect-based sentiment analysis and emotion detection on a single sentence via our teaching framework.
Figure 3: Example reasoning traces for two COVID-19 tweets demonstrating the complete annotation pipeline. Left: sarcastic reference requiring contextual sentiment disambiguation. Right: anxious expression of genuine health concern with negative sentiment.
Figure 4: Illustrative reconstruction of a real tweet. Display name, profile image, username, timestamp, and engagement metadata are fictionalized. Reasoning trace across multiple aspects for a COVID-19 tweet. The model identifies two distinct aspects and performs the complete annotation pipeline for each, resulting in shared negative sentiment but divergent emotions: angry toward media censorship versus scared regarding the outbreak itself.
Metric
Value
Total sentences
21,937
Total aspects
50,615
Average aspects per sentence
2.307
Appendix
Table 5: Corpus summary based on Qwen2.5-32B annotations.
Sentiment
Count
Percent
Neutral
26,249
51.87%
Negative
18,288
36.13%
Positive
6,078
12.01%
Appendix
Table 6: Aspect-level sentiment distribution ( N=50,615 ).
Emotion
Count
Percent
Neutral
21,608
42.70%
Annoyed
10,488
20.73%
Anxious
8,877
17.54%
Hopeful
1,640
3.24%
Scared
1,552
3.07%
Empathetic
1,436
2.84%
Appendix
Table 7: Aspect-level emotion distribution ( N=50,615 ). The no emotion label indicates that the teacher did not assign an emotion to the aspect.
Figure 5: Emotion distribution across the 50 most frequent aspects. Each bar represents the normalized distribution of emotion categories for a given aspect.
Crisis sentiment analysis is especially challenging for low-resource languages such as Bangla, where language, context, and public reaction shift rapidly. We introduce UNRESTSENT200K, a Bangla crisis sentiment dataset with approximately 200K Facebook and YouTube comments from the July-August 2024 Bangladesh uprising. The dataset covers five event-aligned phases, from early escalation and internet blackout to regime transition and a later flood crisis. Each comment is linked to its parent post, enabling evaluation with and without discourse context. All comments are annotated through a fully human process involving 14 native Bangla-speaking annotators and senior validation, achieving substantial agreement (kappa = 0.73, alpha = 0.71) and 94.2% blind-audit agreement. We benchmark fine-tuned encoders, prompted LLMs, and LoRA-tuned LLMs. Results show that parent-post context consistently improves performance, while temporal shift across phases causes large performance drops. Strong LLMs perform well, but still struggle with sarcasm, implicit political references, and phase-dependent meaning. UNRESTSENT200K provides a benchmark for studying context-aware and temporally robust sentiment analysis in low-resource crisis discourse. UNRESTSENT200K is available at https://sami0055.github.io/UNRESTSENT200K/
Md. Samiul Alim, Mahir Shahriar Tamim, Tanvir Ahmed Khan +4
North South University, Dhaka, Bangladesh · Charles Sturt University, Australia
Emotion dynamics are critical for understanding crisis-support conversations, yet most computational work treats emotion as static utterance-level labels. We introduce EMPATH, a framework for understanding affective dynamics in mental health dialogues across three granularities: turn-level labels, transition probabilities, and global conversation archetypes. Applying EMPATH to text-based crisis conversations with self-identified Black texters discussing grief, we find persistent negative affect, gradual hope-ward transitions, distinct texter-volunteer emotional roles, and heterogeneous recovery trajectories. These results highlight the informative patterns that emerge from computationally understanding crisis support and expressions of grief as dynamic processes within conversations, as well as the overall value of emotion-dynamic analysis for analyzing and comparing affect in dialogues.
Semi-supervised learning approaches have been investigated as a means to enhance the analysis of social media data in disaster management contexts. In this work, we present the first empirical evaluation of large language model (LLM) guided semi-supervised learning for crisis related tweet classification. We compare two recent LLM assisted semi-supervised methods, VerifyMatch and LLM guided Co-Training ( LG-CoTrain), against established semi-supervised baselines. Our results show that LG-CoTrain significantly outperforms classical semi-supervised approaches in low resource settings with 5, 10 and 25 labeled examples per class, achieving the highest averaged Macro F1 across events. VerifyMatch achieves competitive performance while also demonstrating strong calibration properties. As the number of labeled examples increases, the performance gap narrows and Self Training emerges as a strong baseline. We further observe that compact semi-supervised models can, in some cases, outperform very large LLMs operating in zero-shot settings. This finding highlights the potential of transferring knowledge from LLMs into smaller and more deployable models through LLM guided semi-supervised learning, offering a practical pathway for real world disaster response applications. Our project repository on Github is here.
Jacob Ativo, Bharaneeshwar Balasubramaniyam, Anh Tran +4
Department of Computer Science University of Illinois at Chicago