cs.CLAug 21, 2026

STAR-OPD: Structured Aspect-Cascade-Aware On-Policy Reward Distillation for ABSA Quadruple Extraction

Authors: Tong Sun, Mingyang Ma, Jiayang Yu

Organizations: Alibaba International Digital Commerce Group

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Aspect-based sentiment analysis (ABSA) quadruple extraction requires jointly predicting target, aspect, opinion, and sentiment over reviews that often contain multiple fine-grained sentiment tuples. While large chain-of-thought (CoT) models perform well on this task, distilling them into smaller deployable models remains difficult. We identify a task-specific failure mode in distilled ABSA extraction: student errors at the target-aspect interface create structurally invalid states, such as broken target-aspect bindings and hallucinated targets, which then corrupt downstream predictions. Conventional off-policy distillation is poorly suited to this setting because it trains only on teacher-generated trajectories and provides little supervision on the student-induced structural states that dominate inference. To address this mismatch, we propose STAR-OPD (STructured Aspect-cascade-aware On-Policy Reward Distillation), which builds on generic on-policy distillation and instantiates it for ABSA quadruple extraction with cascade-aware, set-structured rewards. STAR-OPD trains on student rollouts and applies set-structured rewards that directly target binding consistency, target grounding, and fine-grained aspect disambiguation. Experiments on E-ABSA20K and SemEval-2014 show that STAR-OPD consistently outperforms off-policy and general on-policy baselines, reduces target hallucination, and substantially improves performance on structurally hard cases. With Qwen3-4B, STAR-OPD substantially narrows the student-teacher gap while improving inference efficiency, highlighting the importance of on-policy structural correction for distilled ABSA extraction.

Figures & tables

Figure 1: Motivation for STAR-OPD . Left: Errors at the target–aspect interface create structurally invalid tuples and corrupt downstream prediction. Center: Off-policy distillation trains only on teacher trajectories and misses student-induced cascade failures. Right: On-policy training exposes these states and uses cascade-aware rewards to correct them.
Figure 2: Overview of STAR-OPD . We first prepare a high-quality CoT teacher and generate filtered pseudo-labels for quadruple extraction. We then train the student on-policy under its own rollouts, using cascade-aware rewards to correct target–aspect binding errors and hallucinated targets.
 E-ABSA20KSemEval-2014
MethodWomenBagsDressesMakeupFurnitureRestaurantLaptop
Teacher0.7560.7620.7570.8120.7460.516
Student: Qwen3-4B
Direct-SFT0.6850.6910.6920.7200.6850.470
SeqKD0.6870.6950.6990.7240.7150.468
GKD0.6890.6990.7040.7270.7190.471
Table 1: Main results on E-ABSA20K and SemEval-2014 (Micro F1). Best student in bold ; runner-up underlined . †\dagger : significantly better than MiniLLM ( p<0.05p{<}0.05 , paired bootstrap).
Figure 3: Reducing structured aspect cascade on E-ABSA20K Test. (a) Performance at increasing levels of structural completeness: T-F1, TA-F1, TAS-F1, and Quad-F1. (b) Target hallucination rate (T-Hall; lower is better). STAR-OPD improves all stages of structural correctness, with the largest gain at the target–aspect interface, while reducing hallucinated targets.
VariantF1Δ\Delta F1
STAR-OPD (full)0.747—
On-policy mechanism
w/o On-Policy0.702−- 0.045
w/o Rewards0.716−- 0.031
Reward decomposition
Collapsed scalar reward0.734−- 0.013
Table 2: Ablation of on-policy design choices and cascade-aware rewards on E-ABSA20K Test. F1 denotes micro Quad-F1 over the four E-ABSA20K test domains. Δ\Delta F1 is relative to full STAR-OPD .
MethodPPSarc.Cont.Impl.
Teacher0.6870.7170.7220.723
Direct-SFT0.6320.6640.6570.679
SeqKD0.6330.6780.6710.680
GKD0.6350.6780.6720.683
MiniLLM0.6430.6890.6870.699
G-OPD0.6440.6920.6860.709
Table 3: Hard-sample F1 on E-ABSA20K-Hard. PP: product-part mentions; Sarc.: colloquial sarcasm; Cont.: contrastive sentences; Impl.: implicit opinion expressions.
Qualitative case: product-part (womenbags)
ReviewThe bag looks elegant, but the zipper got stuck on the second day.
Gold( PRODUCT:bag , Design Aesthetic , looks elegant , pos ); ( PRODUCT_PART:zipper , Hardware Quality , got stuck , neg )
SeqKD( PRODUCT_PART:zipper , Ease of Use , got stuck , neg ) [binding drift]
MiniLLM( PRODUCT_PART:handle , Hardware Quality , got stuck , neg ) [hallucinated target]
STAR-OPD( PRODUCT_PART:zipper , Hardware Quality , got stuck , neg ) [correct grounding + binding]
Figure 4: Representative qualitative case from E-ABSA20K-Hard. SeqKD preserves the grounded target but drifts in aspect binding, whereas MiniLLM generates a plausible yet non-grounded part target. STAR-OPD restores both correct grounding and part-specific aspect binding. Additional cases are given in Appendix G .
Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Field32B CoT4B Direct
Target agreement97.5%94.2%
Aspect agreement93.8%81.6%
Opinion agreement94.1%88.3%
Sentiment agreement95.3%86.7%
Overall κ\kappa0.920.81
Appendix
Table 4: Teacher quality against human expert annotations. κ\kappa denotes Cohen’s Kappa.
Reward termDefaultPRODUCT_PART case
λbase\lambda_{\text{base}}0.200.15
λbind\lambda_{\text{bind}}0.300.35
λhall\lambda_{\text{hall}}0.250.30
λcat\lambda_{\text{cat}}0.200.15
λfmt\lambda_{\text{fmt}}0.050.05
Appendix
Table 5: Reward weights used in STAR-OPD. Default weights are used unless the input is identified as a likely PRODUCT_PART case based on teacher pseudo-labels.
Figure 5: Training dynamics of the main reward components and corresponding validation structural metrics during STAR-OPD optimization. Left: RbindR_{\text{bind}} and validation TA-F1. Right: RhallR_{\text{hall}} and validation T-Hall. The reward curves and validation metrics move in directions consistent with improved structural behavior and do not show obvious pathological late-stage divergence.
ModelLatencyReviews/sSpeedup
Teacher †2.1190.471.0 ×\times
STAR-OPD (4B) ‡0.9711.032.18 ×\times
STAR-OPD (1.7B) ‡0.2903.457.31 ×\times
Appendix
Table 6: Inference efficiency on E-ABSA20K Test under the same decoding configuration. All models are evaluated on a single A100 80G GPU.
 L=LrKL+βLref\mathcal{L}=\mathcal{L}_{\mathrm{rKL}}+\beta\mathcal{L}_{\mathrm{ref}} 
Appendix
Algorithm 1 STAR-OPD : On-Policy Cascade-Aware Distillation
(a) product-part (womenbags)
ReviewThe bag looks elegant, but the zipper got stuck on the second day.
Gold( PRODUCT:bag , Design Aesthetic , looks elegant , pos ); ( PRODUCT_PART:zipper , Hardware Quality , got stuck , neg )
SeqKD( PRODUCT_PART:zipper , Ease of Use , got stuck , neg ) [binding drift]
MiniLLM( PRODUCT_PART:handle , Hardware Quality , got stuck , neg ) [hallucinated target]
STAR-OPD( PRODUCT_PART:zipper , Hardware Quality , got stuck , neg ) [correct grounding + binding]
Appendix
Figure 6: Additional qualitative cases: product-part and contrastive reviews.
(c) Sarcasm (makeup)
ReviewLove how this “long-lasting” lipstick vanishes before I finish my coffee.
Gold( PRODUCT:lipstick , Longevity , vanishes before I finish my coffee , neg )
SeqKD( PRODUCT:lipstick , Longevity , long-lasting , pos ) [surface cue imitation]
MiniLLM( PRODUCT:lipstick , Overall , vanishes before I finish my coffee , neg ) [coarse aspect fallback]
STAR-OPD( PRODUCT:lipstick , Longevity , vanishes before I finish my coffee , neg ) [fine-grained recovery]
Appendix
Figure 7: Additional qualitative cases: sarcasm and implicit opinion reviews.
ThemeAspect Categories
OverallOverall Satisfaction
MaterialMaterial Type, Material Quality,
 Material Accuracy, Water Resistance,
 Odor, Cleaning & Maintenance
AppearanceDesign Aesthetic, Color Accuracy,
 Color Preference, Appearance
Appendix
Table 7: Aspect taxonomy for women’s bags (34 categories).
More in this topic

RL for Knowledge Distillation

See all 12 papers

Explore similar work

CardsList
Sep 28, 2026

Decide, Don't Generate: Competitive Dimensional ABSA with Jev's Typed Decisions

Aspect-based sentiment analysis (ABSA) has largely turned to text generation. We show that competitive dimensional ABSA does not need it. Using Jev, a frozen model that answers typed questions with rubric scores, label probabilities, and yes/no judgments, we decompose all three tasks of SemEval-2026 Task III Track A into such decisions and align them with the annotation scheme through 488 coefficients fitted on CPU, with no text generation and no backbone tuning. On valence-arousal regression over ten corpora in six languages, the system reaches 1.0645 RMSE, the lowest aggregate error of any participating system. On triplet and quadruplet extraction, it reaches 52.09 and 44.06 continuous F1, above fine-tuned Llama-3.3-70B and GPT-OSS-120B baselines. Analyses and ablations show where the accuracy comes from: supervised calibration roughly halves the raw regression error, exact valence-arousal would add only 4.5 F1 to extraction, and the learned combination of span-boundary evidence, not any single signal, carries the extraction systems.

Yiqun Zhang, Peidong Wang, Zihan Wang +1

Northeastern University, China

Sentiment AnalysisAspect-Based Sentiment Analysis
AskPDF
May 31, 2026

DiffuSent: Towards a Unified Diffusion Framework for Aspect-Based Sentiment Analysis

Aspect-Based Sentiment Analysis (ABSA) encompasses seven distinct subtasks, each focusing on different extracted elements. Despite the proven success of generative models in unified aspect sentiment analysis, existing approaches often rely on auto-regressive token-by-token generation without grasping the whole information of the aspect and opinion terms, resulting in boundary insensitivity, particularly in context of multi-word aspect and opinion terms. To address these issues, we present DiffuSent, a non-auto-regressive diffusion framework that systematically formulates all ABSA subtasks as boundary denoising diffusion processes, progressively refining boundaries over noisy states. Furthermore, we introduce a contrastive denoising training strategy which effectively address duplicate predictions with subtle variations introduced by diffusion process. Extensive experiments across 28 settings (7 subtasks x 4 datasets) demonstrate that DiffuSent achieves delivers consistent improvements over the strongest generative and span-based systems. DiffuSent exhibits notable gains on multi-word triplets, achieving an average improvement of +2.48 F1, and maintains robust extraction accuracy in sentences containing multiple sentiment triplets. Moreover, the non-auto-regressive decoding enables substantial efficiency benefits, reaching up to 181 times faster inference than auto-regressive generative baselines

Shu Long, Yanglei Gan, Xuchuan Zhou

University of Electronic Science and Technology of China · Southwest Petroleum University · Southwest Minzu University

Aspect-Based Sentiment AnalysisConditional Diffusion Models
AskPDF
Mar 14, 2026

Generate Then Correct: Single Shot Global Correction for Aspect Sentiment Quad Prediction

Aspect-based sentiment analysis (ABSA) extracts aspect-level sentiment signals from user-generated text, supports product analytics, experience monitoring, and public-opinion tracking, and is central to fine-grained opinion mining. A key challenge in ABSA is aspect sentiment quad prediction (ASQP), which requires identifying four elements: the aspect term, the aspect category, the opinion term, and the sentiment polarity. However, existing studies usually linearize the unordered quad set into a fixed-order template and decode it left-to-right. With teacher forcing training, the resulting training-inference mismatch (exposure bias) lets early prefix errors propagate to later elements. The linearization order determines which elements appear earlier in the prefix, so this propagation becomes order-sensitive and is hard to repair in a single pass. To address this, we propose a method, Generate-then-Correct (G2C): a generator drafts quads and a corrector performs a single-shot, sequence-level global correction trained on LLM-synthesized drafts with common error patterns. On the Rest15 and Rest16 datasets, G2C outperforms strong baseline models.

Shidong He, Haoyu Wang, Wenjie Luo

School of Cyber Security and Computer, Hebei University, Baoding, China. · Faculty of Computer Science, Dalhousie University, Halifax, Canada.

Exposure BiasAspect-Based Sentiment Analysis
AskPDF