Beyond Correlation: Refutation-Validated Aspect-Based Sentiment Analysis for Explainable Energy Market Returns
Authors: Wihan van der Heever, Keane Ong, Ranjan Satapathy, Erik Cambria
Organizations: College of Computing and Data Science, Nanyang Technological University, 50 Nanyang Ave, Singapore, 639798, Singapore · College of Design and Engineering, National University of Singapore, 9 Engineering Drive 1, Singapore, 117575, Singapore · Institute of High Performance Computing, Agency for Science, Technology and Research, Fusionopolis Way, #16-16 Connexis, Singapore, 138632, Singapore
This paper proposes a refutation-validated framework for aspect-based sentiment analysis in financial markets, addressing the limitations of correlational studies that cannot distinguish genuine associations from spurious ones. Using X data for the energy sector, we test whether aspect-level sentiment signals show robust, refutation-validated relationships with equity returns. Our pipeline combines net-ratio scoring with z-normalization, OLS with Newey West HAC errors, and refutation tests including placebo, random common cause, subset stability, and bootstrap. Across six energy tickers, only a few associations survive all checks, while renewables show aspect and horizon specific responses. While not establishing causality, the framework provides statistically robust, directionally interpretable signals, with limited sample size (six stocks, one quarter) constraining generalizability and framing this work as a methodological proof of concept.
Figures & tables
Figure 1: Directed acyclic graph (DAG) representing the assumed causal structure for sentiment–return analysis. The target estimand is βi , the effect of lagged aspect sentiment za,t−l on returns ri,t . Observed controls Xt (lagged returns, sentiment activity) are included in the OLS specification. Dashed elements represent unobserved confounders U (e.g., private information flows, algorithmic trading patterns) whose influence refutation tests help assess sensitivity to. Refutation testing cannot eliminate confounding but provides bounded confidence that estimates are not purely artifactual.
Ticker
AspectLag
RT1
RT2
RT3
RT4
Validated associations (pass all tests):
BP
economy1
✓
✓
✓
✓
Shell
economy1
✓
✓
✓
✓
NextEra
market2
✓
✓
✓
✓
NextEra
inflation3
✓
✓
✓
✓
Clearway
investors2
✓
✓
✓
✓
Table 1: Refutation Test (RT) results for sentiment–return associations. Top panel shows validated associations passing all four tests; bottom panel illustrates how specific test failures filter potentially spurious signals.
Figure 2: Dot-and-whisker plot of bootstrap 95% confidence intervals for top signals. Blue: associations passing all four refutation tests; grey: filtered associations failing ≥1 test. Coefficient labels in basis points (×100). The dashed red line marks zero.
Figure 3: Stem plot of market sentiment coefficients across lags 0–3 for NextEra. Only the lag-2 coefficient (blue) survives all four refutation tests, with temporal decay evident at lag 3.
Figure 4: Heatmap of validated sentiment–return associations across aspects and lags. Colored cells show refutation-validated coefficients ( β^×100 , basis points per standard deviation); gray cells indicate associations failing at least one refutation test. The clustering of positive effects at lags 1–2 with a negative inflation effect at lag 3 suggests aspect-specific temporal dynamics in information incorporation.
Figure 5: Comparison of correlational effect sizes (red, Pearson ∣r∣ ) versus refutation-validated regression coefficients (green, ∣β^∣ in daily return units). Raw correlations range from 0.45 to 0.73, while validated effects are an order of magnitude smaller (0.034–0.048), illustrating the substantial “deflation” that occurs when spurious associations are filtered through systematic robustness testing.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure A6: End-to-end workflow for refutation-validated sentiment analysis. Data inputs (top) flow through signal construction and OLS modelling to a four-part refutation suite. Only associations passing all refutation tests proceed to interpretation; spurious signals are discarded.
Financial NLP has a standard workflow: validate a sentiment tool against human labels, then trust it to extract market signal. This assumes the two evaluations measure the same thing. We test that assumption in a setting where both can be measured at once: a corpus of securities class actions (2002-2025) linking 70,500 X messages to abnormal stock returns, with a single-annotator human labelled gold sample. Running five instruments (VADER, Loughran-McDonald, FinBERT, Twitter-RoBERTa, and an LLM annotator) through one identical pipeline, we find that the relationship between construct and predictive validity depends on the sampling convention and score representation. Under conventional method-specific sampling, human agreement aligns more closely with graded same-day associations than with one-day leads. On a fixed-n panel, however, agreement has similar graded rank correlations at both horizons, while the coarse ordering remains weak. Benchmark agreement therefore establishes semantic validity but does not by itself determine predictive rankings. In a conversation that is 17.6% spam, message volume predicts neither market damage nor settlement size.
AS Aravinthakshan, Laven Srivastava, Harsh Nandwani
Financial sentiment classifiers are commonly evaluated against human labels, but strong linguistic performance does not necessarily imply economically useful return predictability. This study separates these questions through two experiments. First, we construct a unified three-class benchmark from five financial text datasets and compare TF--IDF Naive Bayes, off-the-shelf FinBERT and Financial-RoBERTa encoders, zero-shot Qwen2.5-7B, and QLoRA-adapted Qwen2.5-7B, LLaMA3-8B, and Mistral-7B models. Mistral-7B achieves the best test accuracy (0.8840) and macro-F1 (0.8771), while QLoRA raises Qwen2.5's macro-F1 from 0.7274 to 0.8615. An inverse-frequency class-weighted loss does not improve Qwen2.5. Second, we evaluate economic validity on a temporally separate 2019 Benzinga sample containing 10,637 unique headlines and 13,115 headline--stock observations for a fixed S&P~100 universe. Model probabilities are converted into continuous sentiment scores, aggregated by stock and signal date, and aligned with next-session returns over one-, two-, three-, and five-day horizons. All seven downstream models produce positive but small mean rank information coefficients at the one-day horizon; the largest is 0.0143 for FinBERT. None of the 28 model--horizon tests remains significant after Newey--West inference and false-discovery-rate correction. Portfolio results likewise fail to establish a robust advantage for the best-performing classifiers. The findings show that QLoRA is effective for financial sentiment adaptation, while also documenting a clear gap between classification accuracy and tradable cross-sectional signals.
We present the SemEval-2026 shared task on Dimensional Aspect-Based Sentiment Analysis (DimABSA), which improves traditional ABSA by modeling sentiment along valence-arousal (VA) dimensions rather than using categorical polarity labels. To extend ABSA beyond consumer reviews to public-issue discourse (e.g., political, energy, and climate issues), we introduce an additional task, Dimensional Stance Analysis (DimStance), which treats stance targets as aspects and reformulates stance detection as regression in the VA space. The task consists of two tracks: Track A (DimABSA) and Track B (DimStance). Track A includes three subtasks: (1) dimensional aspect sentiment regression, (2) dimensional aspect sentiment triplet extraction, and (3) dimensional aspect sentiment quadruplet extraction, while Track B includes only the regression subtask for stance targets. We also introduce a continuous F1 (cF1) metric to jointly evaluate structured extraction and VA regression. The task attracted more than 400 participants, resulting in 112 final submissions and 42 system description papers. We report baseline results, discuss top-performing systems, and analyze key design choices to provide insights into dimensional sentiment analysis at the aspect and stance-target levels. All resources are available on our GitHub repository.
Liang-Chih Yu, Jonas Becker, Shamsuddeen Hassan Muhammad +14
Yuan Ze University · University of Göttingen · Imperial College London +10