STAR-OPD: Structured Aspect-Cascade-Aware On-Policy Reward Distillation for ABSA Quadruple Extraction

Tong Sun; Mingyang Ma; Jiayang Yu

cs.CLAug 21, 2026

STAR-OPD: Structured Aspect-Cascade-Aware On-Policy Reward Distillation for ABSA Quadruple Extraction

Authors: Tong Sun, Mingyang Ma, Jiayang Yu

Organizations: Alibaba International Digital Commerce Group

Abstract

Aspect-based sentiment analysis (ABSA) quadruple extraction requires jointly predicting target, aspect, opinion, and sentiment over reviews that often contain multiple fine-grained sentiment tuples. While large chain-of-thought (CoT) models perform well on this task, distilling them into smaller deployable models remains difficult. We identify a task-specific failure mode in distilled ABSA extraction: student errors at the target-aspect interface create structurally invalid states, such as broken target-aspect bindings and hallucinated targets, which then corrupt downstream predictions. Conventional off-policy distillation is poorly suited to this setting because it trains only on teacher-generated trajectories and provides little supervision on the student-induced structural states that dominate inference. To address this mismatch, we propose STAR-OPD (STructured Aspect-cascade-aware On-Policy Reward Distillation), which builds on generic on-policy distillation and instantiates it for ABSA quadruple extraction with cascade-aware, set-structured rewards. STAR-OPD trains on student rollouts and applies set-structured rewards that directly target binding consistency, target grounding, and fine-grained aspect disambiguation. Experiments on E-ABSA20K and SemEval-2014 show that STAR-OPD consistently outperforms off-policy and general on-policy baselines, reduces target hallucination, and substantially improves performance on structurally hard cases. With Qwen3-4B, STAR-OPD substantially narrows the student-teacher gap while improving inference efficiency, highlighting the importance of on-policy structural correction for distilled ABSA extraction.

Figures & tables

Figure 1: Motivation for STAR-OPD . Left: Errors at the target–aspect interface create structurally invalid tuples and corrupt downstream prediction. Center: Off-policy distillation trains only on teacher trajectories and misses student-induced cascade failures. Right: On-policy training exposes these states and uses cascade-aware rewards to correct them.

Figure 2: Overview of STAR-OPD . We first prepare a high-quality CoT teacher and generate filtered pseudo-labels for quadruple extraction. We then train the student on-policy under its own rollouts, using cascade-aware rewards to correct target–aspect binding errors and hallucinated targets.

	E-ABSA20K				SemEval-2014
Method	WomenBags	Dresses	Makeup	Furniture	Restaurant	Laptop
Teacher	0.756	0.762	0.757	0.812	0.746	0.516
Student: Qwen3-4B
Direct-SFT	0.685	0.691	0.692	0.720	0.685	0.470
SeqKD	0.687	0.695	0.699	0.724	0.715	0.468
GKD	0.689	0.699	0.704	0.727	0.719	0.471

Table 1: Main results on E-ABSA20K and SemEval-2014 (Micro F1). Best student in bold ; runner-up underlined .

\dagger

: significantly better than MiniLLM (

p{<}0.05

, paired bootstrap).

Figure 3: Reducing structured aspect cascade on E-ABSA20K Test. (a) Performance at increasing levels of structural completeness: T-F1, TA-F1, TAS-F1, and Quad-F1. (b) Target hallucination rate (T-Hall; lower is better). STAR-OPD improves all stages of structural correctness, with the largest gain at the target–aspect interface, while reducing hallucinated targets.

Variant	F1	$\Delta$ F1
STAR-OPD (full)	0.747	—
On-policy mechanism
w/o On-Policy	0.702	$-$ 0.045
w/o Rewards	0.716	$-$ 0.031
Reward decomposition
Collapsed scalar reward	0.734	$-$ 0.013

Table 2: Ablation of on-policy design choices and cascade-aware rewards on E-ABSA20K Test. F1 denotes micro Quad-F1 over the four E-ABSA20K test domains.

\Delta

F1 is relative to full STAR-OPD .

Method	PP	Sarc.	Cont.	Impl.
Teacher	0.687	0.717	0.722	0.723
Direct-SFT	0.632	0.664	0.657	0.679
SeqKD	0.633	0.678	0.671	0.680
GKD	0.635	0.678	0.672	0.683
MiniLLM	0.643	0.689	0.687	0.699
G-OPD	0.644	0.692	0.686	0.709

Table 3: Hard-sample F1 on E-ABSA20K-Hard. PP: product-part mentions; Sarc.: colloquial sarcasm; Cont.: contrastive sentences; Impl.: implicit opinion expressions.

Qualitative case: product-part (womenbags)
Review	The bag looks elegant, but the zipper got stuck on the second day.
Gold	( PRODUCT:bag , Design Aesthetic , looks elegant , pos ); ( PRODUCT_PART:zipper , Hardware Quality , got stuck , neg )
SeqKD	( PRODUCT_PART:zipper , Ease of Use , got stuck , neg ) [binding drift]
MiniLLM	( PRODUCT_PART:handle , Hardware Quality , got stuck , neg ) [hallucinated target]
STAR-OPD	( PRODUCT_PART:zipper , Hardware Quality , got stuck , neg ) [correct grounding + binding]

Figure 4: Representative qualitative case from E-ABSA20K-Hard. SeqKD preserves the grounded target but drifts in aspect binding, whereas MiniLLM generates a plausible yet non-grounded part target. STAR-OPD restores both correct grounding and part-specific aspect binding. Additional cases are given in Appendix G .

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Field	32B CoT	4B Direct
Target agreement	97.5%	94.2%
Aspect agreement	93.8%	81.6%
Opinion agreement	94.1%	88.3%
Sentiment agreement	95.3%	86.7%
Overall $\kappa$	0.92	0.81

Appendix

Table 4: Teacher quality against human expert annotations.

\kappa

denotes Cohen’s Kappa.

Reward term	Default	PRODUCT_PART case
$\lambda_{\text{base}}$	0.20	0.15
$\lambda_{\text{bind}}$	0.30	0.35
$\lambda_{\text{hall}}$	0.25	0.30
$\lambda_{\text{cat}}$	0.20	0.15
$\lambda_{\text{fmt}}$	0.05	0.05

Appendix

Table 5: Reward weights used in STAR-OPD. Default weights are used unless the input is identified as a likely PRODUCT_PART case based on teacher pseudo-labels.

Figure 5: Training dynamics of the main reward components and corresponding validation structural metrics during STAR-OPD optimization. Left: $R_{\text{bind}}$ and validation TA-F1. Right: $R_{\text{hall}}$ and validation T-Hall. The reward curves and validation metrics move in directions consistent with improved structural behavior and do not show obvious pathological late-stage divergence.

Model	Latency	Reviews/s	Speedup
Teacher †	2.119	0.47	1.0 $\times$
STAR-OPD (4B) ‡	0.971	1.03	2.18 $\times$
STAR-OPD (1.7B) ‡	0.290	3.45	7.31 $\times$

Appendix

Table 6: Inference efficiency on E-ABSA20K Test under the same decoding configuration. All models are evaluated on a single A100 80G GPU.

\mathcal{L}=\mathcal{L}_{\mathrm{rKL}}+\beta\mathcal{L}_{\mathrm{ref}}

Appendix

Algorithm 1 STAR-OPD : On-Policy Cascade-Aware Distillation

(a) product-part (womenbags)
Review	The bag looks elegant, but the zipper got stuck on the second day.
Gold	( PRODUCT:bag , Design Aesthetic , looks elegant , pos ); ( PRODUCT_PART:zipper , Hardware Quality , got stuck , neg )
SeqKD	( PRODUCT_PART:zipper , Ease of Use , got stuck , neg ) [binding drift]
MiniLLM	( PRODUCT_PART:handle , Hardware Quality , got stuck , neg ) [hallucinated target]
STAR-OPD	( PRODUCT_PART:zipper , Hardware Quality , got stuck , neg ) [correct grounding + binding]

Appendix

Figure 6: Additional qualitative cases: product-part and contrastive reviews.

(c) Sarcasm (makeup)
Review	Love how this “long-lasting” lipstick vanishes before I finish my coffee.
Gold	( PRODUCT:lipstick , Longevity , vanishes before I finish my coffee , neg )
SeqKD	( PRODUCT:lipstick , Longevity , long-lasting , pos ) [surface cue imitation]
MiniLLM	( PRODUCT:lipstick , Overall , vanishes before I finish my coffee , neg ) [coarse aspect fallback]
STAR-OPD	( PRODUCT:lipstick , Longevity , vanishes before I finish my coffee , neg ) [fine-grained recovery]

Appendix

Figure 7: Additional qualitative cases: sarcasm and implicit opinion reviews.

Theme	Aspect Categories
Overall	Overall Satisfaction
Material	Material Type, Material Quality,
	Material Accuracy, Water Resistance,
	Odor, Cleaning & Maintenance
Appearance	Design Aesthetic, Color Accuracy,
	Color Preference, Appearance

Appendix

Table 7: Aspect taxonomy for women’s bags (34 categories).

RL for Knowledge Distillation

See all 12 papers

Decide, Don't Generate: Competitive Dimensional ABSA with Jev's Typed Decisions
Sep 28, 2026Yiqun Zhang, Peidong Wang, Zihan Wang +1Sentiment Analysis Aspect-Based Sentiment Analysis
Northeastern University, China
DiffuSent: Towards a Unified Diffusion Framework for Aspect-Based Sentiment Analysis
May 31, 2026Shu Long, Yanglei Gan, Xuchuan ZhouAspect-Based Sentiment Analysis Conditional Diffusion Models
University of Electronic Science and Technology of China · Southwest Petroleum University · Southwest Minzu University
Generate Then Correct: Single Shot Global Correction for Aspect Sentiment Quad Prediction
Mar 14, 2026Shidong He, Haoyu Wang, Wenjie LuoExposure Bias Aspect-Based Sentiment Analysis
School of Cyber Security and Computer, Hebei University, Baoding, China. · Faculty of Computer Science, Dalhousie University, Halifax, Canada.

Sep 28, 2026J/K move · Enter open · S save

Decide, Don't Generate: Competitive Dimensional ABSA with Jev's Typed Decisions

Yiqun Zhang, Peidong Wang, Zihan Wang, Shi Feng

Northeastern University, China

Sentiment Analysis Aspect-Based Sentiment Analysis

Open paper AskPDF arXiv

Aspect-based sentiment analysis (ABSA) has largely turned to text generation. We show that competitive dimensional ABSA does not need it. Using Jev, a frozen model that answers typed questions with rubric scores, label probabilities, and yes/no judgments, we decompose all three tasks of SemEval-2026 Task III Track A into such decisions and align them with the annotation scheme through 488 coefficients fitted on CPU, with no text generation and no backbone tuning. On valence-arousal regression over ten corpora in six languages, the system reaches 1.0645 RMSE, the lowest aggregate error of any participating system. On triplet and quadruplet extraction, it reaches 52.09 and 44.06 continuous F1, above fine-tuned Llama-3.3-70B and GPT-OSS-120B baselines. Analyses and ablations show where the accuracy comes from: supervised calibration roughly halves the raw regression error, exact valence-arousal would add only 4.5 F1 to extraction, and the learned combination of span-boundary evidence, not any single signal, carries the extraction systems.

Abstract

RL for Knowledge Distillation

Explore similar work

Decide, Don't Generate: Competitive Dimensional ABSA with Jev's Typed Decisions

DiffuSent: Towards a Unified Diffusion Framework for Aspect-Based Sentiment Analysis

Generate Then Correct: Single Shot Global Correction for Aspect Sentiment Quad Prediction