Financial Language Model Evaluation

Follow this topic Ask about this topic

Latest papers

23 papers3 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Evidence Sufficiency and Model Comparisons in MexRegQA: An Item-Level Audit of Mexican-Spanish Regulatory QA
Oct 19, 2026Fred Hu, Yeqi Wang, Xing ZhouBenchmark Auditing Legal QA
Huawei Technologies de México, Mexico · Northeastern University, USA
From SEC Filings to Financial Signals: Evaluating LLMs on Stock Market Reactions and Analyst Forecasts
Oct 16, 2026Shengkun Wang, Taoran Ji, Linhan Wang +3Financial Language Model Evaluation Financial Forecasting
Recognition Is Not Grounding: Benchmarking Multilingual Financial Evidence Extraction
Oct 14, 2026Lingfei Qian, Yan Wang, Xueqing Peng +5Financial Language Model Evaluation Document Information Extraction
The Fin AI, USA · University of Manchester, UK · Georgia Institute of Technology, USA +2
FinInteract: Benchmarking Clarification and Intent Integration in Ambiguous Financial Question Answering
Sep 21, 2026Xinyu Wang, Tung Sum Thomas Kwok, Zhenghan Tai +1Financial QA Financial Language Model Evaluation
McGill University · University of California, Los Angeles · University of Toronto
Financial Language Models as Applied Artificial Intelligence Systems for News-Based Trading under Market Frictions
Sep 20, 2026Kemal KirtacFinancial Language Model Evaluation LLM Evaluation
Department of Computer Science, University College London, 66–72 Gower Street, London WC1E 6EA, United Kingdom · Department of Computer Science, University of Warwick, 6 Lord Bhattacharyya Way, Coventry CV4 7EZ, United Kingdom
PALM: Point-in-Time Adaptation for Financial Language Models
Sep 2, 2026Seunghan Lee, Jun Seo, Jaehoon Lee +9Financial Language Model Evaluation Language Modeling
LG AI Research, Seoul, Republic of Korea
FinRiskAtlas: Decision-Aligned Evaluation of Large Language Models for Financial Risk Review
Aug 26, 2026Suyang Zhong, Jingzhe Zhu, Qi Xu +5Financial Language Model Evaluation LLM Evaluation
Ant International · Xiamen University · Shanghai University +1
Can Open-Weight Models Compete on Financial Text Comprehension?
Aug 9, 2026Jan SpörerLLM Evaluation Financial QA
University of St. Gallen, Switzerland
CACHE-UK: A Stability-Aware Memory Editor for Sequentially Updated Quantized LLMs in Finance
Jul 30, 2026Anubhav Lakra, Yue FengFinancial Language Model Evaluation Knowledge Editing
University of Birmingham, Birmingham, United Kingdom
Language-Routed RAG and Direct Option Scoring for Multilingual Financial QA: DS@GT at FinMMEval
Jul 24, 2026Justice Ayela, Kabir SahniFinance Multilingual Language Model Evaluation
Georgia Institute of Technology, North Ave NW, Atlanta, GA 30332
Overview of FinMMEval 2026 Task 2: Multilingual Financial Short-Answer Question Answering
Jul 22, 2026Zhuohan Xie, Xueqing Peng, Georgi Georgiev +18Multilingual Language Model Evaluation Financial QA
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates · The Fin AI, United States · FMI, Sofia University “St. Kliment Ohridski”, Sofia, Bulgaria +7
Overview of FinMMEval 2026 Task 1: Multilingual Financial Multiple-Choice Question Answering
Jul 22, 2026Zhuohan Xie, Yuyang Dai, Rania Elbadry +18Multilingual Language Model Evaluation Multiple-Choice Question Answering
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates · INSAIT, Sofia University “St. Kliment Ohridski”, Sofia, Bulgaria · University of Arizona, Tucson, United States +6
The Stanford EDGAR Filings Dataset: Reconstructing U.S. Corporate and Financial Disclosures into Layout-Faithful and Token-Efficient Pretraining Data
Jun 16, 2026Nick Bettencourt, Xiaowei Ding, Kay GieseckeDocument Understanding Financial Language Model Evaluation
Department of Applied Statistics and Data Science University of California, Los Angeles · School of Information Management Nanjing University · Advanced Financial Technologies Lab Stanford University
YouZhi: Towards High-Concurrency Financial LLMs via Adaptive GQA-to-MLA Transition
Jun 4, 2026PSBC LLM Team, Huawei LLM Team, Ruihan Long +56Financial Language Model Evaluation LLM Compression
FINESSE-Bench: A Hierarchical Benchmark Suite for Financial Domain Knowledge and Technical Analysis in Large Language Models
May 14, 2026Dmitry Stanishevskii, Nini Kamkia, Alexey Khoroshilov +4Financial Language Model Evaluation LLM Evaluation
Lime FinTech
Measurement Risk in Supervised Financial NLP: Rubric and Metric Sensitivity on JF-ICR
Apr 30, 2026Sidi Chang, Peiying Zhu, Yuxiao Chen +1LLM Evaluation Rubric-Based Evaluation
Blossom AI Labs, Tokyo, Japan · Blossom AI Labs, San Francisco, USA
IndiaFinBench: An Evaluation Benchmark for Large Language Model Performance on Indian Financial Regulatory Text
Apr 21, 2026Rajveer Singh PallLLM Evaluation Financial QA
Gyan Ganga Institute of Technology and Sciences, Jabalpur, India
SAHM: A Benchmark for Arabic Financial and Shari'ah-Compliant Reasoning
Apr 21, 2026Rania Elbadry, Sarfraz Ahmad, Ahmed Heakl +12Finance Financial Services
1MBZUAI · 2INSAIT, Sofia University ”St. Kliment Ohridski” · 3The University of Manchester +1
Towards a more efficient bias detection in financial language models
Mar 9, 2026Firas Hadj Kacem, Ahmed Khanfir, Mike PapadakisFinancial Language Model Evaluation Intersectional Bias in Language Models
SnT, University of Luxembourg, Luxembourg · RIADI, ENSI, University of Manouba, Tunisia
Unlocking Data Value in Finance: A Study on Distillation and Difficulty-Aware Training
Mar 7, 2026Chuxue Cao, Honglin Lin, Zhanping Zhong +5Financial Language Model Evaluation LLM Training
Shanghai Artificial Intelligence Laboratory, OpenDataLab, OpenDataArena · Hong Kong University of Science and Technology
Constructing Synthetic Instruction Datasets for Improving Reasoning in Domain-Specific LLMs: A Case Study in the Japanese Financial Domain
Mar 2, 2026Yuma Okochi, Fabio Milentiansen Sim, Tomoyasu OkadaFinancial Language Model Evaluation Financial Reasoning in Language Models
Nomura Research Institute, Ltd. · NRI Indonesia
FIRE: A Comprehensive Benchmark for Financial Intelligence and Reasoning Evaluation
Feb 25, 2026Xiyuan Zhang, Huihang Wu, Jiayu Guo +8Financial Language Model Evaluation LLM Evaluation
Du Xiaoman Technology · The PBC School of Finance, Tsinghua University · The School of Finance, Renmin University of China
QuantEval: A Benchmark for Financial Quantitative Tasks in Large Language Models
Jan 13, 2026Zhaolu Kang, Junhao Gong, Wenqing Hu +13Financial Language Model Evaluation LLM Evaluation
Peking University · Tsinghua University · Fudan University +6

Oct 19, 2026J/K move · Enter open · S save

Evidence Sufficiency and Model Comparisons in MexRegQA: An Item-Level Audit of Mexican-Spanish Regulatory QA

Fred Hu, Yeqi Wang, Xing Zhou

Huawei Technologies de México, Mexico · Northeastern University, USA

Benchmark Auditing Legal QA Benchmark Validity Financial Language Model Evaluation Evidence Retrieval

Open paper AskPDF arXiv

Similarity-gated coverage does not establish that retrieved evidence contains the facts needed to answer a question. We audit Mexican-Spanish financial-regulatory QA and find that 35 of 135 covered items lack some or all load-bearing facts. On the post-hoc full-evidence slice (n=98), eleven prompt-matched models score 95-98/98 and none differs significantly from the anchor; this does not establish equivalence. Removing evidence-defective items changes the nominal significance status of 14 of 55 pairwise comparisons. On 56 zero-evidence items, accuracy spans 33.9%-89.3%. Removing evidence from the audited items under an abstention-permitting prompt reduces accuracy by 63-81 percentage points in all ten tested legs, with no recovered items. Building on sufficient-context research, we contribute MexRegQA v1.0: 192 items with provenance and evidence-status labels, frozen retrieval evidence for 191 evaluated items, predictions, judge outputs and offline checks. The audit was outcome-aware, the later model/panel sweep reused the same questions, and all reported scores and labels rely on machine evaluation without independent human validation. We propose releasing evidence-status labels alongside grounded QA benchmarks so that model comparisons can be examined separately from evidence availability.