STQA: A Benchmark for Stock-Focused Tabular Question Answering over Historical and Forecasted Data

Baoxu An; Wenmian Yang; Zhensheng Wang; Weijia Jia

cs.CLSep 5, 2026

STQA: A Benchmark for Stock-Focused Tabular Question Answering over Historical and Forecasted Data

Authors: Baoxu An, Wenmian Yang, Zhensheng Wang, Weijia Jia

Organizations: School of Artificial Intelligence, Beijing Normal University, Beijing, PR China · Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Zhuhai, PR China · Beijing Normal-Hong Kong Baptist University, Zhuhai, PR China

Abstract

Stock market analysis inherently requires composite reasoning over historical records and future projections, yet existing benchmarks remain fragmented across isolated tasks. We introduce STQA (Stock-focused Tabular Question Answering), an end-to-end benchmark designed to systematically evaluate natural-language question answering over historical data, numerical forecasts, and forecast-based reasoning. Built on a large-scale financial dataset, STQA covers 4,417 stocks and contains 31,400 question-answer pairs derived from expert-crafted templates, accompanied by fine-grained intent and slot annotations. To operationalize this benchmark, we present SQFRS (Stock Query-Forecast-Reasoning System), an agent-based unified framework that orchestrates SQL retrieval and time-series forecasting tools. Experiments demonstrate that while current large language models perform well on historical queries, forecast-based reasoning poses a substantial challenge, revealing critical bottlenecks in tool coordination and reasoning under uncertainty. The dataset and code are available at https://github.com/xuxubaobaoan/STQA_Project. STQA thus serves as a rigorous testbed for future research on trustworthy, tool-augmented financial agents.

Figures & tables

Dataset	QA Pairs	Tables	Answer format	Forecast Tasks	Multi-scenario Tasks
FinDER 10.1145/3768292.3770361	5,703	490	Text	×	×
FinTextQA Chen_2024	1262	-	Text	×	×
TQABench qiu2024tqabenchevaluatingllmsmultitable	26,260	1,200	Text/SQL	×	×
WikiSQL zhong2017seq2sqlgeneratingstructuredqueries	80,654	24,241	SQL	×	×
WikiTableQuestion kweon-etal-2023-open	22,033	2,108	Text	×	×
Open-WikiTable kweon-etal-2023-open	67,023	24,680	Text/SQL	×	×

Table 1: Dataset comparison, where “-” indicates data that is currently inaccessible.

Figure 1: The overall framework of SQFRS. Given an input query, the SLU module predicts its intent and extracts slots. In the illustrated example, the Dynamic Routing Agent identifies the query as a forecast-based reasoning problem and selects the corresponding FR workflow to generate the final answer.

		QR		NF			FR		WF
Model	Method	F1	Acc	MSE	MAE	MRE	F1	Acc	Acc
Qwen3 30B	SQFRS	0.7000	0.6924	81.3165	2.4583	0.0617	0.5536	0.5529	99.67%
Qwen3 30B	Vanilla	0.6755	0.6406	128.6482	2.8026	0.0744	0.5111	0.4672	-
GPT-oss-20B	SQFRS	0.7186	0.7152	81.3120	2.4561	0.0616	0.6234	0.5929	99.93%
GPT-oss-20B	Vanilla	0.6259	0.6174	126.2028	2.7814	0.0742	0.5202	0.4706	-
GPT-oss-120B	SQFRS	0.7309	0.7178	80.5744	2.4140	0.0590	0.6319	0.6171	100%

Table 2: Overall performance comparison between the SQFRS framework and the Vanilla baseline, with boldface indicating the best result achieved by each method. “–” indicates that WF is not applicable.

Model	Method	QR	NF	FR
Model	Method	Acc	MRE	Acc
Qwen3 30B	SQFRS	0.6924	0.0617	0.5529
Qwen3 30B	W/O SLU	0.6486	0.0704	0.5029
GPT-oss-20B	SQFRS	0.7152	0.0616	0.5929
GPT-oss-20B	W/O SLU	0.6705	0.0704	0.5509
GPT-oss-120B	SQFRS	0.7178	0.0590	0.6171

Table 3: Ablation Study Results for the SLU Module.

Figure 2: The impact of different GT labels.

		QR		NF			FR
Model	Method	F1	Acc	MAE	MRE	MSE	F1	Acc
Qwen3 30B	Template	0.7000	0.6924	2.4583	0.0617	81.3165	0.5536	0.5529
Qwen3 30B	Human	0.5303	0.5000	2.7960	0.0743	127.5079	0.4857	0.4750
GPT-oss-20B	Template	0.7186	0.7152	2.4561	0.0616	81.3120	0.6234	0.5929
GPT-oss-20B	Human	0.5962	0.5333	2.7714	0.0731	125.5648	0.4970	0.4667
GPT-oss-120B	Template	0.7309	0.7178	2.4140	0.0590	80.5744	0.6319	0.6171

Table 4: Result of Real-world Questions

Model	Setting	FR Acc (%)
Human	Stockholder	69.17
Human	Non-stockholder	63.33
Qwen3 30B	SQFRS	59.17
Qwen3 30B	DR	54.17
GPT-oss-20B	SQFRS	62.50
GPT-oss-20B	DR	50.88

Table 5: FR Accuracy of Humans and Models under SQFRS vs DR.

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

	History SQL
Model	ECR	Pass@1
Qwen3 30B	100%	99.05%
GPT-oss-20B	100%	99.25%
GPT-oss-120B	100%	99.60%
GEMMA 4-31B	100%	99.70%

Appendix

Table 6: History SQL ECR and Pass@1.

Model	Setting	QR				NF			FR
Model	Setting	P	R	F1	Acc	MSE	MAE	MRE	P	R	F1	Acc
Qwen3 30B	SQFRS	0.7006	0.7003	0.7000	0.6924	81.3165	2.4583	0.0617	0.5539	0.5532	0.5536	0.5529
Qwen3 30B	W/O SLU	0.6571	0.6523	0.6529	0.6486	116.9249	2.7243	0.0704	0.5123	0.5166	0.5145	0.5029
GPT-oss-20B	SQFRS	0.7236	0.7188	0.7186	0.7152	81.3120	2.4561	0.0616	0.5959	0.6535	0.6234	0.5929
GPT-oss-20B	W/O SLU	0.6771	0.6730	0.6734	0.6705	116.8996	2.7237	0.0704	0.5531	0.5800	0.5762	0.5509
GPT-oss-120B	SQFRS	0.7314	0.7316	0.7309	0.7178	80.5744	2.4140	0.0590	0.6160	0.6596	0.6319	0.6171

Appendix

Table 7: Experimental Results of the SLU Module.

Model	Method	QR	NF	FR
Model	Method	Acc	MRE	Acc
Qwen3 30B	SQFRS	0.6924	0.0617	0.5529
Qwen3 30B	LLM-SLU	0.6810	0.0685	0.5400
GPT-oss-20B	SQFRS	0.7152	0.0616	0.5929
GPT-oss-20B	LLM-SLU	0.7029	0.0678	0.5886
GPT-oss-120B	SQFRS	0.7178	0.0590	0.6171

Appendix

Table 8: Robustness analysis via in - context learning.

Model	Threshold	Extreme Value	Trend Comparison
Qwen3-30B	76.0%	44.5%	54.8%
GPT-oss-20B	66.0%	56.0%	55.2%
GPT-oss-120B	80.0%	76.0%	55.6%
GEMMA 4-31B	87.0%	82.5%	52.8%

Appendix

Table 9: Fine-grained results on different types of forecast-based reasoning questions.

Price Fields	Forecast-15days	TimeXer	I-Transformer	TimeMixer	TimesNet	GPT-oss-120B	Time-MoE
open	MSE	252.2331	474.3625	147.4776	202.2346	202.2304	145.3490
	MAE	4.1762	5.5079	3.4352	3.8685	4.0777	3.3174
	MRE	0.0868	0.1154	0.0727	0.0810	0.0876	0.0686
close	MSE	216.6427	289.5159	364.7470	673.4907	258.6725	202.9997
	MAE	4.0625	4.7266	5.0233	7.0094	4.4143	3.7858
	MRE	0.0863	0.0998	0.1050	0.1464	0.0929	0.0799

Appendix

Table 10: Experimental Results of the Time Series Forecasting Model Based on Raw Data.

Model	Method	FR				NF
Model	Method	P	R	F1	Acc	MSE	MAE	MRE
Qwen3 30B	TimeXer	0.4768	0.4839	0.4803	0.4743	178.0094	2.8938	0.0855
	I-Transformer	0.5536	0.5529	0.5515	0.5523	425.2180	4.3682	0.1049
	TimeMixer	0.4847	0.4901	0.4874	0.4843	161.6847	2.8681	0.0819
	TimesNet	0.5288	0.5334	0.5311	0.5219	114.4559	2.6444	0.0698
	TimesMoE	0.5539	0.5532	0.5536	0.5529	81.3165	2.4583	0.0617

Appendix

Table 11: Comparative Experiment of Time Series Forecasting Models and Large Language Model.

Label	Statistics num
Number of templates	72
Query template count	42
Forecast template count	30
Number of QA pairs	31400
Single stock QA quantity	28000
Forecast QA quantity	20900

Appendix

Table 12: Dataset Metrics Statistics Table.

	QR				NF			FR
Model	P	R	F1	Acc	MSE	MAE	MRE	P	R	F1	Acc
Qwen3 30B	0.7105	0.7091	0.7085	0.6990	81.312530	2.4576	0.061611	0.5676	0.5662	0.5683	0.5586
GPT-oss-20B	0.7317	0.7300	0.7298	0.7286	81.226003	2.4528	0.061578	0.6255	0.6579	0.6414	0.6100
GPT-oss-120B	0.7407	0.7409	0.7402	0.7381	80.348970	2.4127	0.058952	0.6472	0.6619	0.6607	0.6234
GEMMA 4-31B	0.7412	0.7442	0.7409	0.7392	81.404654	2.4823	0.062075	0.6821	0.6916	0.6816	0.6357

Appendix

Table 13: Ground Truth SLU.

	QR				NF			FR
Model	P	R	F1	Acc	MSE	MAE	MRE	P	R	F1	Acc
Qwen3 30B	0.7168	0.7190	0.7177	0.7086	81.0282	2.4485	0.061560	0.5760	0.5659	0.5709	0.5597
GPT-oss-20B	0.7357	0.7309	0.7315	0.7296	80.9126	2.4452	0.061472	0.6430	0.6607	0.6557	0.6229
GPT-oss-120B	0.7409	0.7423	0.7416	0.7400	80.2875	2.4093	0.058217	0.6483	0.6632	0.6652	0.6359
GEMMA 4-31B	0.7612	0.7534	0.7533	0.7416	81.3612	2.4652	0.061848	0.6840	0.6932	0.6875	0.6374

Appendix

Table 14: Ground Truth Workflow.

Model	MSE	MAE	MRE	P	R	F1	Acc
Qwen3 30B	80.6790	2.4282	0.060553	0.6204	0.6224	0.6210	0.6029
GPT-oss-20B	80.8355	2.4346	0.060584	0.6741	0.6882	0.6712	0.6386
GPT-oss-120B	79.8980	2.4029	0.057562	0.6823	0.7043	0.6879	0.6542
GEMMA 4-31B	80.6177	2.4192	0.060550	0.6961	0.7080	0.7020	0.6629

Appendix

Table 15: Ground Truth History SQL.

Model	P	R	F1	Acc	MRE
Qwen3 30B	0.8593	0.8614	0.8603	0.8371	0.004563
GPT-oss-20B	0.8039	0.8515	0.8108	0.7971	0.004689
GPT-oss-120B	0.8186	0.8690	0.8335	0.8129	0.004153
GEMMA 4-31B	0.8783	0.8936	0.8859	0.8614	0.004931

Appendix

Table 16: Ground Truth Forecast Data.

Figure 3: The impact of different GT labels on final result accuracy and MRE (GPT-oss-20B).

Figure 4: The impact of different GT labels on final result accuracy and MRE (GPT-oss-120B).

Figure 5: The impact of different GT labels on final result accuracy and MRE (Qwen3-30B).

Figure 6: LLM Rewrite Effect: Before vs After.

Figure 7: Artificial Rewriting Effect: Before vs After.

Figure 12: Prompt_Agent Dynamic Routing.

Table QA

Table question answering studies how AI systems retrieve, interpret, and reason over structured data to answer natural-language questions.

See all 66 papers

Reasoning through Verifiable Forecast Actions: Consistency-Grounded RL for Financial LLMs
May 21, 2026Jialin Chen, Aosong Feng, Harshit Verma +7Time Series Reasoning Stock Price Forecasting
Yale University · Arizona State University · University of Texas Rio Grande Valley
FinTradeBench: A Financial Reasoning Benchmark for LLMs
Mar 19, 2026Yogesh Agrawal, Aniruddha Dutta, Md Mahadi Hasan +2Financial Reasoning in Language Models Financial Reasoning Benchmarks
University of Central Florida
XForecast: The KDD 2026 Finance Day Challenge on News-Grounded Financial Forecasting
Oct 12, 2026Jaehoon Lee, Jun Seo, Seunghan Lee +10Financial Forecasting Stock Price Forecasting
LG AI Research, Seoul, South Korea · Ulsan National Institute of Science and Technology (UNIST), Ulsan, South Korea

May 21, 2026J/K move · Enter open · S save

Reasoning through Verifiable Forecast Actions: Consistency-Grounded RL for Financial LLMs

Jialin Chen, Aosong Feng, Harshit Verma, Siyi Gu+6

Yale University · Arizona State University · University of Texas Rio Grande Valley

Time Series Reasoning Stock Price Forecasting RL for Language Model Reasoning Financial QA Time Series Forecasting Probabilistic Time Series Forecasting

Open paper AskPDF arXiv

Financial markets are characterized by extreme non-stationarity, low signal-to-noise ratios, and strong dependence on external information such as news, company fundamentals, and macroeconomic signals. Yet, existing approaches either abstract time-series into text or decouple forecasting from language-based reasoning, leading to a fundamental mismatch between qualitative reasoning and quantitative outcomes. To address this, we introduce StockR1, a time-series-enhanced LLM that unifies stock forecasting and financial reasoning through a verifiable forecast action. Based on a tool-call design, the model first emits a forecast action, which is a structured and interpretable representation of its qualitative market outlook. It then invokes a time-series decoder conditioned on this action to generate distributional future trajectories, leading to more informed question answering and financial reasoning. We optimize the full pipeline with reinforcement learning, where rewards jointly reflect answer validity, forecast accuracy, and consistency between generated actions and observed time-series dynamics. In addition, rewards are reweighted by a sample-level uncertainty scalar, encouraging the model to accommodate varying uncertainty in market dynamics. We evaluate StockR1 on financial question answering and stock forecasting over a large-scale 10-year benchmark. Our method consistently outperforms time-series baselines and general-purpose LLMs, improving reasoning accuracy by 17.7% (4B) and 25.9% (8B). These findings demonstrate that structuring the forecast actions establishes a powerful synergy between language reasoning and temporal prediction, enabling LLMs to reason through verifiable, interpretable, and numerically grounded decisions.

Abstract

Table QA

Explore similar work

Reasoning through Verifiable Forecast Actions: Consistency-Grounded RL for Financial LLMs

FinTradeBench: A Financial Reasoning Benchmark for LLMs

XForecast: The KDD 2026 Finance Day Challenge on News-Grounded Financial Forecasting