LLM-as-an-Annotator: Training Lightweight Models with LLM-Annotated Examples for Aspect Sentiment Tuple Prediction

Nils Constantin Hellwig; Jakob Fehle; Udo Kruschwitz; Christian Wolff

cs.CLMar 2, 2026

LLM-as-an-Annotator: Training Lightweight Models with LLM-Annotated Examples for Aspect Sentiment Tuple Prediction

Authors: Nils Constantin Hellwig, Jakob Fehle, Udo Kruschwitz, Christian Wolff

Organizations: Media Informatics Group, University of Regensburg, Regensburg, Germany · Information Science Group, University of Regensburg, Regensburg, Germany

Abstract

Training models for Aspect-Based Sentiment Analysis (ABSA) tasks requires manually annotated data, which is expensive and time-consuming to obtain. This paper introduces LA-ABSA, a novel approach that leverages Large Language Model (LLM)-generated annotations to fine-tune lightweight models for complex ABSA tasks. We evaluate our approach on five datasets for Target Aspect Sentiment Detection (TASD) and Aspect Sentiment Quad Prediction (ASQP). Our approach outperformed previously reported augmentation strategies and achieved competitive performance with LLM-prompting in low-resource scenarios, while providing substantial energy efficiency benefits. For example, using 50 annotated examples for in-context learning (ICL) to guide the annotation of unlabeled data, LA-ABSA achieved an F1 score of 49.85 for ASQP on the SemEval Rest16 dataset, closely matching the performance of ICL prompting with Gemma-3-27B (51.10), while requiring significantly lower computational resources.

Figures & tables

Figure 1: Illustration of LLM-as-an-Annotator (LA-ABSA) . A LLM (Gemma-3-27B) is prompted to annotate training examples, which are subsequently used to fine-tune lightweight state-of-the-art models for Target Aspect Sentiment Detection (TASD) and Aspect Sentiment Quad Prediction (ASQP).

Dataset	Train		Test
	TASD	ASQP	TASD	ASQP
Rest15	1,120	834	582	537
Rest16	1,708	1,264	587	544
FlightABSA	1,351	1,351	387	387
Coursera	1,400	1,400	400	400
Hotels	1,400	1,400	400	400

Table 1: Dataset statistics overview. Distribution of annotated examples for TASD and ASQP tasks across five domain-specific datasets, showing training and test set sizes.

Approach	# Train		Rest15		Rest16		FlightABSA		Coursera		Hotels		AVG
Approach	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP
Scenario: 0 annotated examples given
Gemma-3-27B (0-shot)	0	0	30.36	24.73	45.51	28.96	51.81	42.37	29.50	13.36	38.97	23.02	39.23	26.49
LA-ABSA
w/ Paraphrase	full	full	27.70	18.54	47.73	26.07	52.10	42.58	30.92	13.41	40.74	26.27	39.84	25.37
w/ DLO	full	full	25.82	18.88	47.41	26.97	51.75	42.82	30.60	13.38	40.70	24.97	39.26	25.40

Table 2: F1 scores of LA-ABSA . Results are evaluated against EDA-based data augmentation methods, QAIE (FT), DS 2 -ABSA (FT), and prompting baselines (0, 10, and 50 annotated examples) as reported by Hellwig et al. (2025a) , as well as fully supervised models including DLO ( Hu et al., 2022b ) (FT) and Paraphrase ( Zhang et al., 2021 ) (FT). The highest F1 scores within each annotation regime (0, 10, 50 or all examples) are shown in bold; the best overall scores across all settings are underlined.

Figure 2: Energy consumption analysis . Comparative evaluation of energy usage (kilowatt-hours, kWh) for ASQP and TASD tasks across different settings (0, 10 or 50 annotated examples given). Results are shown for LLM-prompting (Gemma-3-27B) and LA-ABSA methods, using either DLO ( Hu et al., 2022b ) or Paraphrase ( Zhang et al., 2021 ) for fine-tuning. Each line represents the average energy usage to predict up to 100,000 examples per method and task across the five datasets. LA-ABSA approaches generally require much lower energy due to their smaller underlying base model T5-base.

Method	0-shot		10-shot		50-shot
Method	TASD	ASQP	TASD	ASQP	TASD	ASQP
LA-ABSA (Paraphrase)	2.47 ± 0.53	3.17 ± 0.31	1.96 ± 0.40	2.66 ± 0.28	1.83 ± 0.38	2.41 ± 0.46
LA-ABSA (DLO)	11.16 ± 0.72	13.06 ± 0.89	10.68 ± 2.11	13.19 ± 1.62	8.95 ± 0.85	11.65 ± 2.23
Gemma-3-27B ICL	703.10 ± 63.97	912.25 ± 72.67	481.39 ± 46.11	843.68 ± 356.29	1688.76 ± 255.30	2963.65 ± 1360.88

Table 3: Average energy consumption analysis . Energy usage (milliwatt-hours, mWh) per sample of LA-ABSA and LLM-prompting (Gemma-3-27B) across few-shot settings. Results show mean ± standard deviation across the five datasets for TASD and ASQP tasks. We performed 18 pairwise comparisons between DLO, Paraphrase, and Prompting (3 comparisons

\times

2 tasks

\times

3 shot settings) using paired t-tests with Bonferroni-Holm correction (

p_{\text{adj}}\leq 0.05

), and all differences were found to be significant.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Distribution of aspect categories and sentiments across datasets for TASD and ASQP tasks. Each subplot shows the top 10 aspect categories (sorted by total frequency), with stacked bars representing positive (green), neutral (yellow), and negative (red) sentiments. The ’Others’ category aggregates the remaining aspects. Results are aggregated across five datasets: Rest15 ( Zhang et al., 2021 ; Pontiki et al., 2015 ) , Rest16 ( Zhang et al., 2021 ; Pontiki et al., 2016 ) , FlightABSA ( Hellwig et al., 2025a ) , Coursera ( Chebolu et al., 2024 ) , and Hotels ( Chebolu et al., 2024 ) . This visualization highlights the imbalances in aspect-level sentiment annotations, showing varying distributions of polarities and aspect categories across datasets.

Approach	# Aug	# Train		Rest15		Rest16		FlightABSA		Coursera		Hotels		AVG
Approach	# Aug	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP
Scenario: 10 annotated examples given
EDA w/ Paraphrase	2	30	30	13.15	1.74	7.67	11.41	10.79	4.42	17.97	6.61	22.05	3.47	14.33	5.53
EDA w/ Paraphrase	5	60	60	23.20	10.42	16.35	12.30	19.26	10.99	23.62	9.47	25.85	6.15	21.66	9.86
EDA w/ Paraphrase	10	110	110	26.07	10.52	16.82	12.68	21.83	13.46	25.67	10.35	24.95	5.84	23.07	10.57
EDA w/ Paraphrase	15	160	160	25.53	10.08	18.34	12.32	21.57	12.39	25.60	12.33	24.94	7.03	23.20	10.83

Appendix

Table 4: Results of ablation study for EDA-based data augmentation. F1 scores are reported across different augmentation ratios (2, 5, 10, 15) and fine-tuning approaches (DLO, Paraphrase) for scenarios with 10 and 50 annotated examples. Results demonstrate that 10 augmentations combined with DLO achieves optimal performance across most datasets. Bold values indicate the highest F1 scores within a scenario assuming either 10 or 50 given annotated examples.

Figure 4: Prediction time analysis . Prediction time in minutes for ASQP and TASD tasks across different settings (0, 10, or 50 annotated examples given). Results are shown for LLM-prompting (Gemma-3-27B) and LA-ABSA using DLO ( Hu et al., 2022b ) and Paraphrase ( Zhang et al., 2021 ) for fine-tuning. Each line represents the average time required to predict up to 100,000 examples per method and task across the five datasets. LA-ABSA approaches generally require substantially less time due to their smaller underlying base model T5-base.

Approach	# Train		Rest15		Rest16		FlightABSA		Coursera		Hotels		AVG
Approach	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP
Scenario: 0 annotated examples given
Gemma-3-27B (0-shot)	0	0	29.41	23.35	44.49	27.75	47.55	39.70	26.95	11.95	37.12	22.88	37.11	25.13
LA-ABSA
w/ Paraphrase	full	full	26.91	17.09	45.93	24.00	48.95	39.41	28.32	11.68	38.53	25.41	37.73	23.52
w/ DLO	full	full	24.44	17.21	45.68	24.62	47.41	39.06	27.52	11.48	37.98	23.83	36.61	23.24

Appendix

Table 5: Precision scores of LA-ABSA . Results are evaluated against EDA-based data augmentation methods, QAIE, DS 2 -ABSA, and prompting baselines (0, 10, and 50 annotated examples) as reported by Hellwig et al. (2025a) , as well as fully supervised models including DLO ( Hu et al., 2022b ) and Paraphrase ( Zhang et al., 2021 ) . The highest precision scores within each annotation regime (0, 10, 50 or all examples) are shown in bold; the best overall scores across all settings are underlined.

Approach	# Train		Rest15		Rest16		FlightABSA		Coursera		Hotels		AVG
Approach	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP
Scenario: 0 annotated examples given
Gemma-3-27B (0-shot)	0	0	31.36	26.29	46.57	30.29	56.90	45.42	32.58	15.14	41.02	23.16	41.69	28.06
LA-ABSA
w/ Paraphrase	full	full	28.54	20.25	49.68	28.54	55.69	46.31	34.06	15.74	43.22	27.18	42.24	27.60
w/ DLO	full	full	27.36	20.91	49.27	29.81	56.98	47.39	34.47	16.02	43.85	26.21	42.38	28.07

Appendix

Table 6: Recall scores of LA-ABSA . Results are evaluated against EDA-based data augmentation methods, QAIE, DS 2 -ABSA, and prompting baselines (0, 10, and 50 annotated examples) as reported by Hellwig et al. (2025a) , as well as fully supervised models including DLO ( Hu et al., 2022b ) and Paraphrase ( Zhang et al., 2021 ) . The highest recall scores within each annotation regime (0, 10, 50 or all examples) are shown in bold; the best overall scores across all settings are underlined.

Approach	# Train		Rest15		Rest16		FlightABSA		Coursera		Hotels		AVG
Approach	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP	TASD	ASQP
Scenario: 0 annotated examples given
Gemma-3-27B (0-shot)	0	0	33.68	23.64	45.94	23.64	55.24	39.50	33.78	13.23	29.70	18.49	39.67	23.70
LA-ABSA
w/ Paraphrase	full	full	22.88	19.93	35.84	26.20	45.95	33.61	11.85	4.09	16.20	12.44	26.54	19.25
w/ DLO	full	full	24.12	21.02	36.12	29.16	45.37	36.32	12.10	3.70	17.18	13.10	26.98	20.66

Appendix

Table 7: Macro-averaged F1 scores of LA-ABSA. Results are evaluated against EDA-based data augmentation methods, QAIE, DS 2 -ABSA, and prompting baselines (0, 10, and 50 annotated examples) as reported by Hellwig et al. (2025a) , as well as fully supervised models including DLO ( Hu et al., 2022b ) and Paraphrase ( Zhang et al., 2021 ) . The highest F1 macro scores within each annotation regime (0, 10, 50 or all examples) are shown in bold; the best overall scores across all settings are underlined.

Aspect-Based Sentiment Analysis

Aspect-Based Sentiment Analysis (ABSA) identifies what people express opinions about and the sentiment or emotion attached to each aspect.

See all 51 papers

Prompting Is All You Need: Multi-view Prompting Large Language Models for Aspect-Based Sentiment Analysis
May 27, 2026Nils Constantin Hellwig, Niklas Donhauser, Jakob Fehle +2LLM Prompting Few-Shot Prompting
1Media Informatics Group, University of Regensburg, Germany · 2Information Science Group, University of Regensburg, Germany
AnnoABSA: A Web-Based Annotation Tool for Aspect-Based Sentiment Analysis with Retrieval-Augmented Suggestions
Mar 2, 2026Nils Constantin Hellwig, Jakob Fehle, Udo Kruschwitz +1Aspect-Based Sentiment Analysis LLM-Assisted Annotation
Media Informatics Group, University of Regensburg, Regensburg, Germany · Information Science Group, University of Regensburg, Regensburg, Germany
Zero-Shot to Full-Resource: Cross-lingual Transfer Strategies for Aspect-Based Sentiment Analysis
Apr 29, 2026Jakob Fehle, Nils Constantin Hellwig, Udo Kruschwitz +1Multilingual Language Model Evaluation Aspect-Based Sentiment Analysis
Media Informatics Group, University of Regensburg, Regensburg, Germany · Information Science Group, University of Regensburg, Regensburg, Germany

May 27, 2026J/K move · Enter open · S save

Prompting Is All You Need: Multi-view Prompting Large Language Models for Aspect-Based Sentiment Analysis

Nils Constantin Hellwig, Niklas Donhauser, Jakob Fehle, Udo Kruschwitz+1

1Media Informatics Group, University of Regensburg, Germany · 2Information Science Group, University of Regensburg, Germany

LLM Prompting Few-Shot Prompting Aspect-Based Sentiment Analysis

Open paper AskPDF arXiv

Recent work explored the capabilities of Large Language Models (LLMs) in Aspect-Based Sentiment Analysis (ABSA) through few-shot prompting, requiring substantially fewer annotated examples while achieving notable improvements over zero-shot baselines. However, a performance gap remained compared to models fine-tuned on hundreds of examples, and the computational costs of LLM inference present practical barriers to deployment. We introduce LLM-based Multi-View Prompting (LLM-MvP), which adapts the multi-view principle of considering multiple element orderings to LLM prompting. By combining schema-constrained decoding with a context-free grammar and prefix batching, LLM-MvP achieves performance competitive or superior to fine-tuned approaches while substantially reducing computational overhead. Extensive experiments across five benchmark datasets demonstrate that LLM-MvP closes the gap between few-shot prompting and fine-tuned models, offering a practical and efficient solution for ABSA.

Abstract

Aspect-Based Sentiment Analysis

Explore similar work

Prompting Is All You Need: Multi-view Prompting Large Language Models for Aspect-Based Sentiment Analysis

AnnoABSA: A Web-Based Annotation Tool for Aspect-Based Sentiment Analysis with Retrieval-Augmented Suggestions

Zero-Shot to Full-Resource: Cross-lingual Transfer Strategies for Aspect-Based Sentiment Analysis