cs.CLOct 2, 2026

BAIBAICHUCHU at the NTCIR-19 FinArg-3 Task: When Is Maximum Possible Profit Predictable from Investor Text?

Authors: Zong-Han Bai, Po-Yen Chu

Organizations: National Tsing Hua University Hsinchu, Taiwan · Stanford University Stanford, CA, USA

Abstract

The BAIBAICHUCHU team participated in the Social Media Subtask of NTCIR-19 FinArg-3, ranking Chinese investor posts by Maximum Possible Profit (MPP). A three-track ensemble of lexical features, a FinArg-2-pre-finetuned MacBERT ranker, and an LLM judge reaches 0.734 in post-grouped development evaluation, but our best official run scores 0.517. All twelve submitted runs lie between 0.4598 and 0.5402, and our 26 unanimous three-track pairs score 0.500. A post-hoc audit finds that the submitted judge applied a long-only rule to bearish posts although MPP is stance-aware. Correcting it changes 28 of 87 official predictions without changing accuracy, yet lowers development accuracy from 0.680 to 0.622: a regime-specific semantic shortcut improved validation fit. We decompose ranking into directional text, volatility and horizon, and pairwise margin. A running-extremum model predicts σTσ\sqrt{T} scaling, observed ex post in 502 price-aligned posts from a separate July 2026 collection. Pre-posting volatility scaled by the post-specific observation horizon, σpre,20Niσ_{\mathrm{pre},20}\sqrt{N_i}, is associated with a later truncated-horizon MPP proxy (Spearman ρ=0.320ρ=0.320; ticker-cluster 95% CI [0.133,0.466]) and correctly orders 61.4% of unequal-outcome pairs. Transferred text is nearly uncorrelated with the July outcome (ρ=0.055ρ=0.055) and adds little conditional on this score and stance. Development reliability rises from about 0.60 to 0.93 as the labeled MPP gap widens. An earlier ERAI result of 0.6207 rules out universal unpredictability as a simple explanation. Evaluation should jointly consider text, market regime, historical volatility, horizon, and pair composition.

Figures & tables

Explore similar work

CardsList
  1. StakeBench: Evaluating Language Understanding Grounded in Market Commitment

    May 25, 2026Yunhua Pei, Jingyu Hu, Yiwei Shi +3LLM EvaluationForecasting Benchmarks

  2. Measurement Risk in Supervised Financial NLP: Rubric and Metric Sensitivity on JF-ICR

    Apr 30, 2026Sidi Chang, Peiying Zhu, Yuxiao Chen +1LLM EvaluationRubric-Based Evaluation