Label-Consistent Data Generation for Aspect-Based Sentiment Analysis Using LLM Agents

Mohammad H. A. Monfared; Lucie Flek; Akbar Karimi

cs.CLFeb 18, 2026

Label-Consistent Data Generation for Aspect-Based Sentiment Analysis Using LLM Agents

Authors: Mohammad H. A. Monfared, Lucie Flek, Akbar Karimi

Organizations: Bonn-Aachen International Center for Information Technology, University of Bonn, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence, Germany

Abstract

We propose an agentic data augmentation method for Aspect-Based Sentiment Analysis (ABSA) that uses iterative generation and verification to produce high quality synthetic training examples. To isolate the effect of agentic structure, we also develop a closely matched prompting-based baseline using the same model and instructions. Both methods are evaluated across three ABSA subtasks (Aspect Term Extraction (ATE), Aspect Sentiment Classification (ATSC), and Aspect Sentiment Pair Extraction (ASPE)), four SemEval datasets, and two encoder-decoder models: T5-Base and Tk-Instruct. Our results show that the agentic augmentation outperforms raw prompting in label preservation of the augmented data, especially when the tasks require aspect term generation. In addition, when combined with real data, agentic augmentation provides higher gains, consistently outperforming prompting-based generation. These benefits are most pronounced for T5-Base, while the more heavily pretrained Tk-Instruct exhibits smaller improvements. As a result, augmented data helps T5-Base achieve comparable performance with its counterpart.

Figures & tables

Figure 1: Overview of the agentic data augmentation workflow. A generator agent first extracts a style policy and produces candidate sentences, which are then evaluated by an evaluator agent. Only validated examples are saved, forming a high-quality synthetic dataset.

Raw Prompting

Agentic Approach

You are a critic who can generate comments on the specified aspect and sentiment. We would like you to complete a sentence generation task. Please follow these requirements: - You need to use the sentiment, the aspect mentioned in the prompt - Domain: Restaurants - Your response must include: 1. The sentence. 2. A line that starts with Terms= followed by the list of aspect terms used. 3. A line that starts with Polarity= followed by the matching polarity list. - ALL aspect terms must appear as actual aspects in the sentence with intended polarities - Generated sentence must have the writing style and grammar structure and length of this sentence: {sent} - The sentence should not have aspect words not specified in the prompt - DO NOT repeat the input text in the output - PRINT ONLY THE ANSWER TEXT — no explaining, nothing else - Make sure to use aspect words in the output Good Examples: [’prices’] [’negative’] The prices were too high for this type of restaurant [’Gnocchi’, ’cheesecake’] [’positive’, ’negative’] The Gnocchi was perfectly cooked and delicious, but the cheesecake was dry and flavorless. Bad Example: [’soup’], [’positive’] The udon soup was rich and flavorful. (term incorrect) Make sure your output exactly follows this format.

Generate Sentence Prompt: You are a critic who can generate comments on the specified aspect and sentiment. We would like you to complete a sentence generation task. Please follow these requirements: - Generate a sentence using this aspect term: {aspect_term} with the following polarities: {polarity} - Write in the style: {writing_style}, and use a {grammar_structure} grammatical structure and {sentence_length} sentence length. - Domain: Restaurants - Include: 1. The sentence. 2. A line that starts with Terms= followed by the list of aspect terms used. 3. A line that starts with Polarity= followed by the matching polarity list. - Follow the exact structure shown in the examples. - Do not include explanations. Use plain apostrophes (’) — do not escape with backslashes. Label Verifier Prompt: You are an expert in linguistic evaluation. Check if the given aspect terms and polarities are correct for the provided sentence. - If all aspect terms appear as actual aspects in the sentence with intended polarities, respond only with: OK - If any term is missing, incorrect, or has the wrong polarity, respond only with: NOT_OK Do not provide explanations or any other text. Example Input: The food was lousy… Terms=[’food’] Polarity=[’negative’] → OK The udon soup was rich and flavorful. Terms=[’soup’] Polarity=[’positive’] → NOT_OK

Table 1: Side-by-side comparison of raw prompting and agentic prompt structures. Both methods share identical task instructions and output constraints; the agentic approach differs only by decomposing generation and verification into separate steps, enabling automatic rejection of label-inconsistent outputs.

Model	Data Source	Training Strategy	Ratio	ASPE	ATE	ATSC
T5-Base	Original	Original	–	78.84	88.15	86.65
T5-Base	Agentic	Gen	1.0	39.16	54.56	81.72
T5-Base	Agentic	Gen	2.0	39.45	54.10	82.26
T5-Base	Agentic	Mixed	1.0	79.6	88.66	88.10
T5-Base	Agentic	Mixed	2.0	78.26	88.58	87.37
T5-Base	Prompting	Gen	1.0	31.29	49.97	80.53

Table 2: F 1 scores of training strategies averaged over data sets and runs.

Data Type	ATE	ATSC	ASPE
Agentic	78.17%	80.37%	33.89%
Prompting	43.89%	72.94%	18.33%

Table 3: Label consistency of augmented data from Rest16 dataset, measured by Tk-Instruct .

Figure 2: Average F 1 score across all ABSA tasks for each dataset, comparing original vs. generated-only training, signaling the clear gap in quality between real and synthetic data.

Model	Data Type	Laptop14	Rest14	Rest15	Rest16
T5-Base	Original Data	93.32	95.12	80.90	83.28
	Mixed Data - Agentic
	Mixed x1	93.82 ↑0.50	94.63 ↓0.49	80.98 ↑0.08	85.21 ↑1.93
	Mixed x2	93.50 ↑0.18	94.26 ↓0.86	82.22 ↑1.32	84.37 ↑1.09
	Mixed Data - Prompting
	Mixed x1	92.83 ↓0.49	94.53 ↓0.59	80.43 ↓0.47	83.83 ↑0.55

Table 4: ATE F 1 scores across datasets for each model trained on original data and mixed datasets combining original and synthetic examples. Mixed x1 and Mixed x2 denote synthetic-to-original data ratios of 1:1 and 2:1, respectively. Agentic augmentation provides clearer gains, while prompting-based augmentation yields limited or negative impact.

Figure 3: $\Delta$ F 1 between baseline and added agentic data, across both T5-Base and Tk-Instruct . Each bar represents mean F 1 scores averaged over ATE, ATSC, and ASPE tasks for each dataset. This plot shows the clear difference of the augmentation effectiveness on these models.

Figure 4: Agentic data augmentation narrows the F 1 gap between T5-Base and Tk-Instruct . Grey bars indicate the original model gap; blue bars show the performance gain of T5-Base after agentic augmentation (Mixed x1).

Figure 5: Average F 1 score change ( $\Delta F\textsubscript{1}$ ) for agentic and prompting augmentation, computed as the difference between the Mixed x1 setup and original-only baseline. Scores are averaged across three ABSA subtasks and two model architectures for each dataset.

Figure 6: Comparison of food-related term frequency and sentiment polarity in Rest 16 across original and mixed data sets, suggesting that augmentation mitigates the inherent biases.

Figure 7: Average change in F 1 score across all datasets and both models for each ABSA task (ATE, ATSC, ASPE) under Mixed x1 conditions. This figure points out the effectiveness of augmentation for each task.

ID	raw_text	aspectTerms	aspectCategories
2383	Bottom line: B+ for the food, F for the service.	[{term: food , polarity: positive }, {term: service , polarity: negative }]	[{category: food , polarity: positive }, {category: service , polarity: negative }]
766	We always go there on the weekends and leave extremely full and satisfied.	[{term: noaspectterm , polarity: none }]	[{category: anecdotes , polarity: positive }]
1419	We actually gave 10% tip (which we have never done despite mediocre food and service), because we felt totally ripped off.	[{term: food , polarity: neutral }, {term: service , polarity: neutral }, {term: tip , polarity: negative }]	[{category: food , polarity: neutral }, {category: service , polarity: neutral }, {category: price , polarity: negative }]
1700	The food was good.	[{term: food , polarity: positive }]	[{category: food , polarity: positive }]
1892	Service is great, takeout is good too.	[{term: Service , polarity: positive }, {term: takeout , polarity: positive }]	[{category: food , polarity: positive }, {category: service , polarity: positive }]

Table 5: Sample entries from SemEval-2014 Restaurant dataset

Model	Data Type	Laptop14	Rest14	Rest15	Rest16
T5-Base	Original Data	93.32	95.12	80.90	83.28
	Generated Data - Agentic
	Gen. x1	46.18	66.53	50.82	54.73
	Gen. x2	46.83	65.09	50.19	54.30
	Generated Data - Prompting
	Gen. x1	41.46	58.85	48.45	51.13

Table 6: ATE F 1 scores across datasets for each model trained on original data and on synthetic data only. Gen. x1 and Gen. x2 indicate synthetic data generated at 1:1 and 2:1 ratios relative to the original training set. Agentic generation consistently outperforms prompting, while synthetic-only training shows lower performance.

Model	Data Type	Laptop14	Rest14	Rest15	Rest16
T5-Base	Original Data	79.78	86.25	88.74	91.86
	Generated Data - Agentic
	Gen. x1	77.42	79.64	81.54	88.29
	Gen. x2	78.05	80.00	81.73	89.26
	Generated Data - Prompting
	Gen. x1	75.86	76.85	81.84	87.57

Table 7: ATSC F 1 scores across datasets for each model trained on original data and on synthetic data only. Gen. x1 and Gen. x2 indicate synthetic data generated at 1:1 and 2:1 ratios relative to the original training set. Agentic generation consistently outperforms prompting, while synthetic-only training shows lower performance.

Model	Data Type	Laptop14	Rest14	Rest15	Rest16
T5-Base	Original Data	81.19	81.34	73.38	79.47
	Generated Data - Agentic
	Gen. x1	28.42	48.79	37.94	41.49
	Gen. x2	28.81	48.88	37.58	42.56
	Generated Data - Prompting
	Gen. x1	23.13	38.15	28.58	35.30

Table 8: ASPE F 1 scores across datasets for each model when trained on original data and on synthetic data only. Gen. x1 and Gen. x2 denote synthetic datasets generated at ratios of 1:1 and 2:1 relative to the original training set size. The performance gap between synthetic-only and original data is most pronounced for ASPE, highlighting the difficulty of learning joint aspect–sentiment extraction from generated data alone, even with agentic generation.

Model	Data Type	Laptop14	Rest14	Rest15	Rest16
T5-Base	Original Data	79.78	86.25	88.74	91.86
	Mixed Data - Agentic
	Mixed x1	83.38 ↑3.60	86.78 ↑0.53	88.92 ↑0.18	93.33 ↑1.47
	Mixed x2	81.34 ↑1.56	86.42 ↑0.17	88.56 ↓0.18	93.17 ↑1.31
	Mixed Data - Prompting
	Mixed x1	79.93 ↑0.15	86.96 ↑0.71	88.37 ↓0.37	93.82 ↑1.96

Table 9: ATSC F 1 scores across datasets for each model trained on original data and mixed datasets combining original and synthetic examples. Mixed x1 and Mixed x2 denote synthetic-to-original data ratios of 1:1 and 2:1; Agentic augmentation yields the largest and most consistent gains for T5-Base, whereas Tk-Instruct shows mixed or negative changes under both augmentation methods.

Model	Data Type	Laptop14	Rest14	Rest15	Rest16
T5-Base	Original Data	81.19	81.34	73.38	79.47
	Mixed Data - Agentic
	Mixed x1	82.37 ↑1.18	80.31 ↓1.03	76.00 ↑2.62	79.72 ↑0.25
	Mixed x2	78.84 ↓2.35	81.52 ↑0.18	73.89 ↑0.51	78.79 ↓0.68
	Mixed Data - Prompting
	Mixed x1	80.68 ↓0.51	80.42 ↓0.92	72.92 ↓0.46	79.02 ↓0.45

Table 10: ASPE F 1 scores across datasets for each model trained on original data and mixed datasets combining original and synthetic examples. Mixed x1 and Mixed x2 denote synthetic-to-original data ratios of 1:1 and 2:1, respectively; Agentic augmentation yields modest and dataset-dependent improvements. Prompting-based augmentation generally leads to performance drops for both models.

Synthetic Data Augmentation

Synthetic data augmentation creates or modifies training examples to improve learning when real data are scarce, imbalanced, costly, or sensitive.

See all 448 papers

LLM-as-an-Annotator: Training Lightweight Models with LLM-Annotated Examples for Aspect Sentiment Tuple Prediction
Mar 2, 2026Nils Constantin Hellwig, Jakob Fehle, Udo Kruschwitz +1Aspect-Based Sentiment Analysis LLM-Assisted Annotation
Media Informatics Group, University of Regensburg, Regensburg, Germany · Information Science Group, University of Regensburg, Regensburg, Germany
Prompting Is All You Need: Multi-view Prompting Large Language Models for Aspect-Based Sentiment Analysis
May 27, 2026Nils Constantin Hellwig, Niklas Donhauser, Jakob Fehle +2LLM Prompting Few-Shot Prompting
1Media Informatics Group, University of Regensburg, Germany · 2Information Science Group, University of Regensburg, Germany
DiffuSent: Towards a Unified Diffusion Framework for Aspect-Based Sentiment Analysis
May 31, 2026Shu Long, Yanglei Gan, Xuchuan ZhouAspect-Based Sentiment Analysis Conditional Diffusion Models
University of Electronic Science and Technology of China · Southwest Petroleum University · Southwest Minzu University

Mar 2, 2026J/K move · Enter open · S save

LLM-as-an-Annotator: Training Lightweight Models with LLM-Annotated Examples for Aspect Sentiment Tuple Prediction

Nils Constantin Hellwig, Jakob Fehle, Udo Kruschwitz, Christian Wolff

Media Informatics Group, University of Regensburg, Regensburg, Germany · Information Science Group, University of Regensburg, Regensburg, Germany

Aspect-Based Sentiment Analysis LLM-Assisted Annotation

Open paper AskPDF arXiv

Training models for Aspect-Based Sentiment Analysis (ABSA) tasks requires manually annotated data, which is expensive and time-consuming to obtain. This paper introduces LA-ABSA, a novel approach that leverages Large Language Model (LLM)-generated annotations to fine-tune lightweight models for complex ABSA tasks. We evaluate our approach on five datasets for Target Aspect Sentiment Detection (TASD) and Aspect Sentiment Quad Prediction (ASQP). Our approach outperformed previously reported augmentation strategies and achieved competitive performance with LLM-prompting in low-resource scenarios, while providing substantial energy efficiency benefits. For example, using 50 annotated examples for in-context learning (ICL) to guide the annotation of unlabeled data, LA-ABSA achieved an F1 score of 49.85 for ASQP on the SemEval Rest16 dataset, closely matching the performance of ICL prompting with Gemma-3-27B (51.10), while requiring significantly lower computational resources.

Abstract

Synthetic Data Augmentation

Explore similar work

LLM-as-an-Annotator: Training Lightweight Models with LLM-Annotated Examples for Aspect Sentiment Tuple Prediction

Prompting Is All You Need: Multi-view Prompting Large Language Models for Aspect-Based Sentiment Analysis

DiffuSent: Towards a Unified Diffusion Framework for Aspect-Based Sentiment Analysis