CN-Buzz2Portfolio: A Chinese-Market Dataset and Benchmark for LLM-Based Macro and Sector Asset Allocation from Daily Trending Financial News

Liyuan Chen; Shilong Li; Jiangpeng Yan; Shuoling Liu; Qiang Yang; Xiu Li

cs.LGMar 18, 2026

CN-Buzz2Portfolio: A Chinese-Market Dataset and Benchmark for LLM-Based Macro and Sector Asset Allocation from Daily Trending Financial News

Authors: Liyuan Chen, Shilong Li, Jiangpeng Yan, Shuoling Liu, Qiang Yang, Xiu Li

Organizations: Tsinghua Shenzhen International Graduate School · E Fund Management Co., Ltd. · Hong Kong Polytechnic University

Abstract

Large Language Models (LLMs) are rapidly transitioning from static Natural Language Processing (NLP) tasks including sentiment analysis and event extraction to acting as dynamic decision-making agents in complex financial environments. However, the evolution of LLMs into autonomous financial agents faces a significant dilemma in evaluation paradigms. Direct live trading is irreproducible and prone to outcome bias by confounding luck with skill, whereas existing static benchmarks are often confined to entity-level stock picking and ignore broader market attention. To facilitate the rigorous analysis of these challenges, we introduce CN-Buzz2Portfolio, a reproducible benchmark grounded in the Chinese market that maps daily trending news to macro and sector asset allocation. Spanning a rolling horizon from 2024 to mid-2025, our dataset simulates a realistic public attention stream, requiring agents to distill investment logic from high-exposure narratives instead of pre-filtered entity news. We propose a Tri-Stage CPA Agent Workflow involving Compression, Perception, and Allocation to evaluate LLMs on broad asset classes such as Exchange Traded Funds (ETFs) rather than individual stocks, thereby reducing idiosyncratic volatility. Extensive experiments on nine LLMs reveal significant disparities in how models translate macro-level narratives into portfolio weights. This work provides new insights into the alignment between general reasoning and financial decision-making, and all data, codes, and experiments are released to promote sustainable financial agent research.

Figures & tables

Figure 1: Comparison of Financial Agent Research Paradigms. (a) Direct Live Trading: Agents interact with real-time markets. While offering maximum realism, this approach poses scientific challenges regarding reproducibility and attribution , making it difficult to isolate valid reasoning from market randomness. (b) Entity-Centric Benchmarks (e.g., StockBench): The standard paradigm mapping news to pre-defined target stocks. This overlooks the “Public Attention Filtering” process and often suffers from high idiosyncratic volatility at the individual stock level, complicating the verification of logical consistency. (c) CN-Buzz2Portfolio (Ours): A rolling-horizon benchmark simulating the pipeline from Daily Trending News (Public Attention) to Macro & Sector Allocation . By targeting diversified asset classes to reduce noise, this framework serves as a diagnostic tool to rigorously evaluate the alignment between semantic understanding and verifiable portfolio logic.

Model	2024 Task A: Macro Allocation				2024 Task B: Sector Rotation
Model	Return	Sharpe	MaxDD	Volatility	Return	Sharpe	MaxDD	Volatility
Reasoning Models
DeepSeek-R1	7.47%	0.51	-10.71%	0.177	28.37%	1.14	-12.93%	0.255
Qwen3-32B-Think	5.56%	0.50	-7.66%	0.128	21.67%	0.94	-16.21%	0.249
Qwen3-Max-Think	8.98%	0.62	-10.95%	0.167	40.11%	1.52	-15.84%	0.251
Standard Models

Table 1: Comparative performance across two distinct market regimes: the high-volatility momentum environment of 2024 and the low-yield range-bound environment of 2025 . Reasoning-oriented models demonstrate superior alpha generation in complex sector rotation tasks during volatile regimes, while general instruction models maintain robustness in stable macro allocation. Quantitative Baselines denote classic algorithmic strategies: Momentum and MVO. Market Baselines denote broader indices (CSI 300), and Naive EW Portfolio represents an equally-weighted allocation of all candidate assets. 2025 returns represent cumulative period returns (non-annualized).

Variance Statistic	2024	2024	2025	2025
(Scaled by $10^{-3}$ )	Macro	Sector	Macro	Sector
Total Variance	1.57	9.14	0.48	0.50
Between-Model Var. ( $\sigma^{2}_{model}$ )	1.26	6.43	0.25	0.33
Within-Model Var. ( $\sigma^{2}_{stochastic}$ )	0.32	2.71	0.23	0.17
Efficacy Ratio ( $\sigma^{2}_{model}/\sigma^{2}_{stochastic}$ )	3.98	2.37	1.09	1.98
Avg. Reasoning Stability ( $\bar{\sigma}$ )	1.27	1.58	0.83	1.14

Table 2: Variance Decomposition Analysis. The Efficacy Ratio quantifies the dominance of model’s narrative-processing consistency over stochastic noise . A ratio significantly greater than 1.0 suggests that performance is driven by algorithmic logic rather than random variance.

Panel A: 2024 Full Year (High Volatility / Policy Shift)
Model	Task A: Macro Allocation				Task B: Sector Rotation
Model	Top-0	Top-5	Top-10	Top-20	Top-0	Top-5	Top-10	Top-20
Reasoning Models
DeepSeek-R1	5.93	7.38	7.47	8.56	26.32	21.66	28.37	23.90
Qwen3-Max-Think	10.27	11.46	8.98	10.03	29.10	44.98	40.11	36.25
Qwen3-32B-Think	10.34	2.77	5.56	6.99	28.82	19.27	21.67	24.02

Table 3: Ablation Results on Top-N News (Cumulative Return %). Bold indicates the best model within the same Top-

K

setting (column best). Underline indicates the best Top-

K

setting for a specific model (row best).

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Channel	Intra-day Deduplicated		Global Deduplicated
Channel	Count	Avg. Length	Count	Avg. Length
Caixin	13,365	926.71	9,813	967.08
Sina Finance	35,595	1,890.82	32,187	1,878.74
Tencent Stock	29,284	1,720.21	25,104	1,699.11
Tiantian Fund	28,573	1,653.94	25,195	1,643.44

Appendix

Table 4: Dataset statistics across different financial news channels. "Count" refers to the number of Top-20 news entries. Length is measured in characters.

Category	Code	Asset Name	Economic Proxy Role & Semantic Scope
Equity	000300.SH	CSI 300	Blue Chips: Represents China’s core economy (Financials, Consumption). Proxy for “General Market Beta.”
	000905.SH	CSI 500	Mid-Cap Growth: Representative of manufacturing and secondary growth drivers.
	399006.SZ	ChiNext	Innovation: Focuses on high-tech startups in Shenzhen (Healthcare, New Energy). High volatility.
	000688.SH	STAR 50	Hard Tech: Proxy for “National Strategic Tech” (Semiconductors, Biotech) and R&D intensity.
Cyclical	000932.SH	Consumer	Domestic Demand: Tracks essential and optional consumption. Proxy for “Retail Recovery” narratives.
	000941.SH	New Energy	Green Transition: Covers PV, Wind, EV batteries. Sensitive to “Carbon Neutrality” policies.

Appendix

Table 5: Asset Universe for Task A (Macro/Thematic). These assets allow the agent to express views on economic growth, inflation, and strategic policy directions.

Category	Code	Asset Name	Economic Proxy Role & Semantic Scope
Finance	512880.SH	Securities	Market Beta: High elasticity to market sentiment. “Bull Market Flagbearer.”
	512800.SH	Banks	Value Defense: High dividend yield. Proxy for “Systemic Stability” and SOE reform.
	512070.SH	Insurance	Long-Term Rates: Beneficiary of rising yields and demographic trends.
Tech	159995.SZ	Semi-cond	Tech Sovereignty: Chips, ICs. Key to “Self-Reliance” narratives.
	159819.SZ	AI	Trend: Computing power, Algorithms. Proxy for the global “AI Boom.”
	515880.SH	Comm. Eq.	Infrastructure: 5G/6G, Data Centers. Proxy for “New Infrastructure” spending.

Appendix

Table 6: Asset Universe for Task B (Sector Rotation). This granular selection tests the agent’s ability to differentiate between similar sectors (e.g., Pharma vs. Healthcare) based on news nuances.

Panel A: Popular Asset (CSI 300 Index)
Model	Trend Acc.	Price Acc. ( $\pm 1\%/3\%/10\%$ )
GPT-5	0.46	0.24 / 0.57 / 0.76
DeepSeek-R1	0.74	0.20 / 0.44 / 0.73
Qwen3-Max	0.32	0.06 / 0.13 / 0.67
Qwen3-32B	0.60	0.05 / 0.13 / 0.37
Panel B: Obscure Asset (ETF 159852.SZ)

Appendix

Table 7: Memory Probe Results. ETF 159852 represents assets unlikely to be present in pre-training data.

Financial Decision-Making

Financial decision-making studies how people and AI systems allocate money, assess risk, and choose actions under uncertainty.

See all 38 papers

From Knowing to Doing: A Memory-Controlled Benchmark for LLM Trading Agents on Stock Markets
May 27, 2026Taojie Zhu, Wentao Zhao, Rui Sun +7Benchmark Contamination LLM Agent Evaluation
1Tsinghua University, 2Stepfun · 2Stepfun · 4Shanghai Jiao Tong University +3
PortBench: A Correlation-Aware, Full-Pipeline Benchmark for LLM-Driven Portfolio Management
May 27, 2026Yuxuan Zhao, Sijia Chen, Ningxin SuLLM Evaluation Quantitative Finance
Yantai Research Institute of Harbin Engineering University · The Hong Kong University of Science and Technology (Guangzhou)
Fin-Bias: Comprehensive Evaluation for LLM Decision-Making under human bias in Finance Domain
May 9, 2026Xiaoyu Hu, Jinman ZhaoLLM Evaluation Social Bias in Language Models
Rutgers University · Department of Computer Science, University of Toronto

May 27, 2026J/K move · Enter open · S save

From Knowing to Doing: A Memory-Controlled Benchmark for LLM Trading Agents on Stock Markets

Taojie Zhu, Wentao Zhao, Rui Sun, Beidi Luan+6

1Tsinghua University, 2Stepfun · 2Stepfun · 4Shanghai Jiao Tong University+3

Benchmark Contamination LLM Agent Evaluation AI Agent Benchmarks Finance Algorithmic Trading

Open paper AskPDF arXiv

Evaluating whether large language model (LLM) agents can profit in capital markets is increasingly framed as end-to-end trading: place an agent in a historical market, let it trade, and measure portfolio returns. This setup is vulnerable to two evaluation failures. First, long backtests often overlap with the knowledge cutoffs of frontier LLMs, allowing memorized tickers, dates, prices, and market narratives to substitute for investment reasoning. Second, raw returns are a noisy proxy for stock-selection ability, since positive performance may come from market beta, style exposure, or favorable regimes rather than genuine alpha. We introduce KTD-Fin (Knowing-To-Doing Financial Benchmark), an end-to-end stock-market trading benchmark that addresses both issues. KTD-Fin uses a data-side masking protocol to anonymize key identifiers and calendar information consistently across prompts and tools, separating historical market memory from investment decision-making. It also incorporates a Barra-style performance attribution framework that decomposes portfolio returns into market, style, and stock-selection alpha components. Across ten frontier LLM agents evaluated on the Chinese CSI300 over a 2024--2026 window, masking substantially changes agent rationales, pushing them towards anonymized factor-based reasoning. Attribution analysis further shows that LLM agents' cumulative returns under leakage-controlled evaluation are largely explained by passive market and style exposure, with limited evidence of persistent stock-selection alpha. These findings suggest that financial LLM benchmarks should evaluate not only whether an agent makes money, but also whether the source of returns reflects transferable investment skill. We release KTD-Fin as a reproducible template for leakage-controlled and attribution-aware evaluation of LLM trading agents.

Abstract

Financial Decision-Making

Explore similar work

From Knowing to Doing: A Memory-Controlled Benchmark for LLM Trading Agents on Stock Markets

PortBench: A Correlation-Aware, Full-Pipeline Benchmark for LLM-Driven Portfolio Management

Fin-Bias: Comprehensive Evaluation for LLM Decision-Making under human bias in Finance Domain