SenseAI: A Human-in-the-Loop Dataset for RLHF-Aligned Financial Sentiment Reasoning

Berny Kabalisa

cs.CLApr 6, 2026

SenseAI: A Human-in-the-Loop Dataset for RLHF-Aligned Financial Sentiment Reasoning

Authors: Berny Kabalisa

Organizations: RizqSpark | SenseAI

Abstract

We introduce SenseAI, a human-in-the-loop (HITL) validated financial sentiment dataset designed to capture not only model outputs but the full reasoning process behind them. Unlike existing resources, SenseAI incorporates reasoning chains, confidence scores, human correction signals, and real-world market outcomes, providing a structure aligned with Reinforcement Learning from Human Feedback (RLHF) paradigms. The dataset consists of 1,439 labelled data points across 40 US-listed equities and 13 financial data categories, enabling direct integration into modern LLM fine-tuning pipelines. Through analysis, we identify several systematic patterns in model behavior, including a novel failure mode we term Latent Reasoning Drift, where models introduce information not grounded in the input, as well as consistent confidence miscalibration and forward projection tendencies. These findings suggest that LLM errors in financial reasoning are not random but occur within a predictable and correctable regime, supporting the use of structured HITL data for targeted model improvement. We discuss implications for financial AI systems and highlight opportunities for applying SenseAI in model evaluation and alignment.

Figures & tables

Field	Description
Ticker	Equity symbol of the company referenced in the news item
Timestamp	Precise date and time of the news event (Eastern Time)
AI News Headline	Financial news headline as processed by the language model
AI Sentiment	Model-generated classification (Bullish / Slightly Bullish / Neutral / Slightly Bearish / Bearish)
AI Reasoning	Full chain-of-thought reasoning paragraph supporting the classification
AI Confidence Score	Model self-assessed certainty expressed as a percentage

Table 1: SenseAI Dataset Schema

Category	Label	Description
0	No correction	AI classification accepted as-is
1	Minor correction	Small sentiment adjustment (e.g., Slightly Bullish $\to$ Bullish)
2	Moderate correction	Meaningful sentiment shift (e.g., Neutral $\to$ Slightly Bullish)
3	Complete reversal	Sentiment direction inverted (e.g., Bullish $\to$ Bearish)

Table 2: Edit Type Taxonomy

Sentiment Category	Count	Proportion
Slightly Bullish	882	61.3%
Neutral	417	29.0%
Bearish	69	4.8%
Slightly Bearish	32	2.2%
Bullish	32	2.2%
Total	1,439	100.0%

Table 3: AI Sentiment Distribution (

n=1{,}439

)

Panel of Figure 1 — Figure 1: AI Sentiment Distribution ( $n=1{,}439$ ). The dominance of Slightly Bullish (61.3%) relative to unhedged Bullish (2.2%) directly validates Finding 1: systematic sentiment hypersensitivity to linguistic qualifiers.

Category	Description	Count	Proportion
0	No correction (accepted)	707	49.1%
1	Minor correction	715	49.7%
2	Moderate correction	19	1.3%
3	Complete reversal	0	0.0%
	Total	1,439	100.0%

Table 4: Edit Type Distribution (

n=1{,}439

)

Panel of Figure 2 — Figure 2: Edit type distribution. The near-symmetry between Category 0 (no edit, 49.1%) and Category 1 (minor edit, 49.7%), combined with the complete absence of Category 3 reversals, defines the Goldilocks Zone of correctable model error.

Confidence Band	Count	Proportion
50–59%	130	9.0%
60–69%	1,022	71.0%
70–79%	288	20.0%
80%+	14	1.0%
Total	1,439	100.0%
Mean		65.6%

Table 5: Confidence Score Distribution by Band

Panel of Figure 3 — Figure 3: Confidence score distribution. The dominant concentration in the 60–69% band (71% of all observations) directly validates Finding 2: systematic confidence over-hedging regardless of signal clarity. No score exceeds 85%.

Model Version	Approx. Data Points	Proportion
GPT-5.2	1,122	78%
GPT-5 mini	216	15%
GPT-5.1	101	7%
Total	1,439	100%

Table 6: Model Version Distribution

Dimension	SenseAI	FinancialPhraseBank	FiQA	FLUE
Dataset size	1,439+ (growing)	4,840 (static)	1,174 (static)	Multiple tasks
Sentiment granularity	5 categories	3 categories	Continuous score	3 categories
Reasoning chain	✓ Full CoT	✗	✗	✗
RLHF alignment	✓ Designed for	✗	✗	✗
Continuous collection	✓ Daily	✗	✗	✗
Market outcome validation	✓ 4h price	✗	✗	✗

Table 7: Comparative Analysis: SenseAI vs. Existing Financial Sentiment Benchmarks

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Field	Value
Ticker	MSFT – Microsoft
Timestamp	2025-12-01, 2:23 AM ET
AI News Headline	Microsoft doubles down on AI and cloud — expanding infrastructure and earning fresh accolades
AI Sentiment	Slightly Bullish
AI Reasoning	Microsoft’s expansion in AI and cloud infrastructure demonstrates strong long-term commitment. The recognition it is receiving underlines growing confidence from the market and industry peers. Its investments in regions such as Indonesia underline its global footprint and reinforce long-term demand for cloud and AI offerings. On the flip side, rising costs tied to heavy AI investment, along with reports about margin pressure, suggest that profitability could face headwinds in the near term. The strength of its growth potential outweighs those near-term risks, but exposure to execution and cost concerns tempers full bullishness.
AI Confidence Score	75%

Appendix

Table 8: Representative SenseAI Data Point (MSFT, December 1, 2024)

Explore similar work

Unlocking Data Value in Finance: A Study on Distillation and Difficulty-Aware Training
Mar 7, 2026Chuxue Cao, Honglin Lin, Zhanping Zhong +5LLM Training Language Modeling
Shanghai Artificial Intelligence Laboratory, OpenDataLab, OpenDataArena · Hong Kong University of Science and Technology
FinReasoning: A Hierarchical Benchmark for Reliable Financial Research Reporting
Feb 25, 2026Yiyun Zhu, Yidong Jiang, Ziwen Xu +4LLM Evaluation LLM Reliability
School of Computer Science and Technology, Tongji University, Shanghai, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China
Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements
Jul 22, 2026Xinke Tong, Xuanming Zhang, Tianyi Tang +10Numerical Reasoning in Language Models LLM Evaluation
Qwen Team, Alibaba Group · Tsinghua University · Alibaba Group

Mar 7, 2026J/K move · Enter open · S save

Unlocking Data Value in Finance: A Study on Distillation and Difficulty-Aware Training

Chuxue Cao, Honglin Lin, Zhanping Zhong, Xin Gao+4

Shanghai Artificial Intelligence Laboratory, OpenDataLab, OpenDataArena · Hong Kong University of Science and Technology

LLM Training Language Modeling Language Model Distillation Synthetic Data Generation for Language Models RL for Language Models

Open paper AskPDF arXiv

Large Language Models (LLMs) have demonstrated strong general capabilities, yet their deployment in finance remains challenging due to dense domain-specific terminology, stringent numerical reasoning requirements, and low tolerance for factual errors. We conduct a controlled empirical study showing that in specialized vertical domains, performance is largely determined by the quality and difficulty/verifiability profile of post-training data. We introduce ODA-Fin-SFT-318k, constructed via multi-stage distillation and verification to produce high-quality Chain-of-Thought supervision, and ODA-Fin-RL-12k, curated for hard-but-verifiable tasks that balance reward precision and task diversity. Using standard SFT and RL pipelines, we show that high-quality CoT distillation establishes a robust foundation during SFT, while difficulty- and verifiability-aware sampling improves RL generalization. Evaluated on nine benchmarks spanning general financial tasks, sentiment analysis, and numerical reasoning, our ODA-Fin-RL-8B consistently surpasses open-source state-of-the-art (SOTA) financial LLMs of comparable size. We release our ODA-Fin-SFT-318k and ODA-Fin-RL-12k datasets, along with trained models to advance data-centric financial AI research.

Abstract

Explore similar work

Unlocking Data Value in Finance: A Study on Distillation and Difficulty-Aware Training

FinReasoning: A Hierarchical Benchmark for Reliable Financial Research Reporting

Are the Financial Reasoning from LLMs Credible? A Real World Test over Long-Horizon Statements