Language Model Robustness

Follow this topic Ask about this topic

Momentum

37 papers in the last four weeks, up 54% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 338

All topics

Fighting Numerical Hallucinations via Data-centric Compilation for Online Financial QA
May 29, 2026Hao Chen, Xing Tang, Qirui Liu +6Retrieval-Augmented Generation Hallucination in Language Models
Shenzhen Technology University FiT, Tencent Shenzhen, China · South China University of Technology Guangzhou, China · The Hong Kong University of Science and Technology Hong Kong SAR, China +2
Toxic HallucinAItions: Perturbing Prompts and Tracing LLM Circuits
May 29, 2026Soorya Ram Shimgekar, Agam Goyal, Amruta Parulekar +6Prompt Sensitivity LLM Interpretability
University of Illinois Urbana-Champaign · Nimblemind
PReMISE: Policy Rubrics as Measurement Specifications for LLM Judges
May 29, 2026Swastik Roy, Rajkumar Pujari, Tharindu Kumarage +5LLM-as-a-Judge LLM Auditing
Amazon AGI
Triaging Threats to Specialized Guardrails
May 29, 2026Wenjie Jacky Mo, Xiaofei Wen, Rui Cai +6LLM Safety Benchmarks LLM Guardrails
University of California, Davis
Same Patient, Different Words, Different Diagnosis? Evaluating Semantic Stability in Clinical LLMs
May 28, 2026Mahdi Alkaeed, Adnan Qayyum, Nabeel Abo Kashreef +2Prompt Sensitivity Clinical Language Model Evaluation
Department of Computer Science and Engineering, College of Engineering, Qatar University, Doha, Qatar · College of Science and Engineering, Hamad Bin Khalifa University (HBKU), Doha, Qatar · Primary Health Care Corporation (PHCC), Doha, Qatar +1
Evaluating using Mock Tool Calls to Quarantine Untrusted Prompt Inputs
May 28, 2026David Gros, Adam GleaveLanguage Model Safety Evaluation Adversarial Attacks on LLMs
Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models
May 28, 2026Zizhuo Lin, Quanling Liu, Jinsheng Quan +6LLM Grounding Reasoning Consistency in Language Models
Zhejiang University · University of Science and Technology of China
GRUFF: LLM Pronoun Fidelity, Reasoning, and Biases in German
May 28, 2026Fabian Mewes, Anne Lauscher, Vagrant GautamGender Bias in Language Models Linguistic Bias in Language Models
JobMatchMe GmbH, Germany · Trustworthy AI Lab, University of Hamburg, Germany · Heidelberg Institute for Theoretical Studies, Germany
BioRefusalAudit: Auditing Biosecurity Refusal Depth Using General and Domain-Fine-Tuned Sparse Autoencoders
May 28, 2026Caleb DeLeeuwLanguage Model Safety Evaluation Sparse Autoencoders
Independent researcher
Temporal Stability and Few-Shot Prompting in Math Task Assessment
May 28, 2026Danielle S. Fox, Brenda L. Robles, Elizabeth DiPietro Brovey +1Generative AI in Education Few-Shot Prompting
Learning Research and Development Center, University of Pittsburgh · Institute for Learning, University of Pittsburgh
Harnessing non-adversarial robustness in large language models
May 28, 2026Qinghua Zhou, Ellina Aleshina, Andrey Lovyagin +6LLM Fine-Tuning Neural Network Robustness
International Joint Laboratory of AI for Industry, QUST, Qingdao, China · King’s College London, London, UK · Applied AI Institute, Moscow, Russia +2
The Curse of Helpfulness: Inverse Scaling Law in Robustness to Distractor Instructions via DistractionIF
May 28, 2026Zeli Su, Zhankai Xu, Tianlei Chen +4Language Model Scaling Laws Language Model Robustness
Minzu University of China, Beijing, China · Ant Group, Hangzhou, China · Renmin University of China, Beijing, China +1
Aligned but Fragile: Enhancing LLM Safety Robustness via Zeroth-Order Optimization
May 28, 2026Zhihao Liu, Yifan Wu, Jian Lou +3LLM Alignment Zeroth-Order Optimization
The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security · Sun Yat-sen University +1
Mind Your Tone: Does Tone Alter LLM Performance?
May 27, 2026Om Dobariya, Akhil KumarLLM Evaluation LLM Prompting
Pennsylvania State University University Park, PA, USA
FormInv: A Measurement Protocol for Semantic Invariance in Mathematical Reasoning Benchmarks
May 27, 2026Nishal Thomas, Noel ThomasMathematical Reasoning Benchmarks Benchmark Auditing
Independent Researcher · Mohamed Bin Zayed University of Artificial Intelligence
Framing Matters: Addressing Framing Sensitivity in Decision-Making through Behaviorally-Grounded Value Alignment
May 27, 2026Seojin Hwang, Minju Kim, Junhyuk Choi +2LLM Alignment Framing Effects in Language Models
Chung-Ang University, Seoul, Korea
Pressure-Testing Deception Probes in LLMs: Scaling, Robustness, and the Geometry of Deceptive Representations
May 27, 2026Sachin KumarLLM Evaluation Language Model Safety Evaluation
LexisNexis, USA
The Fragility of Chain-of-Thought Monitoring Across Typologically Diverse Languages
May 27, 2026Eric Onyame, Runtao Zhou, Kowshik Thopalli +2CoT Faithfulness Multilingual Language Model Evaluation
University of Virginia · Lawrence Livermore National Laboratory
A Paired Testing Protocol for Batch-Conditioned Refusal Robustness in LLM Serving
May 26, 2026Sahil KadadekarLLM Inference Language Model Safety Evaluation
Independent Researcher.
Generating Robust Portfolios of Optimization Models using Large Language Models
May 26, 2026Eleni Straitouri, Cheol Woo Kim, Milind TambeLarge Language Model-Guided Optimization Language Model Robustness
Max Planck Institute for Software Systems, Kaiserslautern, Germany · Harvard University, Cambridge, United States
SeDT: Sentence-Transformer Decision-Transformer Conditioning for Multi-Turn Conversation Reliability
May 26, 2026Ramakrishna Vamsi Setti, Jagadeesh Rachapudi, Sachin Chaudhary +2LLM Reliability Language Model Robustness
Independent Researcher · 2Drone Lab, IIT Mandi · 3UPES, Dehradun
Evaluating Interactive Reasoning in Large Language Models: A Hierarchical Benchmark with Executable Games
May 26, 2026Mingyuan Fan, Weiguang Han, Daixin Wang +3LLM Evaluation AI Agent Benchmarks
East China Normal University · Ant Group
Conceptual Steganography
May 26, 2026Zhejian Zhou, Jonathan MayCoT Reasoning LLM Security
University of Southern California · Information Sciences Institute
Which Changes Matter? Towards Trustworthy Legal AI via Relevance-Sensitive Evaluation and Solver-Grounded Reasoning
May 26, 2026Chen Linze, Cai Yufan, Hou Zhe +1Adversarial Robustness of Language Models Language Model Robustness
National University of Singapore · Griffith University
When Do LLM Agents Treat Surface Noise Differently from Semantic Noise? A 68-Cell Measurement Study with a Held-Out Trace-Level Validation
May 25, 2026Liyun Zhang, Jiayi GuoCoT Reasoning LLM Agent Evaluation
School of Information and Software Engineering, UESTC · Jacobs School of Engineering, UC San Diego
A Tertiary Review of Large Language Model-Based Code Generating Tasks: Trends, Challenges, and Future Directions
May 25, 2026Muslim Chochlov, Michael English, Jim BuckleyCode Generation Evaluation Language Model Robustness
University of Limerick, Ireland
Trust but Verify: Prover-Verifier Deliberation for Selective LLM Prediction
May 24, 2026João Sedoc, Baotong Zhang, Dean FosterLLM Answer Verification Selective Prediction
New York University
Measuring Reasoning Quality in LLMs: A Multi-Dimensional Behavioral Framework
May 23, 2026Ali Şenol, Garima Agrawal, Huan LiuLLM Evaluation Language Model Robustness
Department of Computer Engineering, Tarsus University, Takba¸s M. Kartaltepe S., Tarsus 33400, Mersin, Türkiye · School of Computing and Augmented Intelligence (SCAI), Arizona State University (ASU), Tempe, AZ 85281, USA · HumaConn AI Consulting, Queen Creek, AZ 85140, USA
ChaosBench-Logic v2: Evaluating LLM Logical Reasoning over Dynamical Systems at Scale
May 23, 2026Noel ThomasLLM Evaluation Language Model Robustness
Mohamed bin Zayed University of Artificial Intelligence · Abu Dhabi, UAE
Positional Failures in Long-Context LLMs: A Blind Spot in Reasoning Benchmarks
May 22, 2026Chuyifei Zhang, Hongyu Cui, Xiaowen Huang +1LLM Evaluation Long-Context Language Model Evaluation
Beijing Jiaotong University · Central South University of Forestry and Technology

May 29, 2026, cs.IRJ/K move · Enter open · S save

Fighting Numerical Hallucinations via Data-centric Compilation for Online Financial QA

Hao Chen, Xing Tang, Qirui Liu, Weijie Shi+5

Shenzhen Technology University FiT, Tencent Shenzhen, China · South China University of Technology Guangzhou, China · The Hong Kong University of Science and Technology Hong Kong SAR, China+2

Retrieval-Augmented Generation Hallucination in Language Models Financial QA Mathematical Reasoning Language Model Robustness LLM Reasoning

Open paper AskPDF arXiv

Large Language Models (LLMs) have significantly advanced online data services, particularly in the domain of financial question answering (FinQA). However, such systems remain susceptible to numerical reasoning hallucinations, which critically undermine reliability in high-stakes financial applications. Although retrieval-augmented generation (RAG) has been widely adopted to ground responses in external knowledge, it introduces three persistent challenges: noise sensitivity, calculation fragility, and an auditability crisis. Existing model-centric approaches, which primarily focus on optimizing either the retriever or generator in isolation, still struggle to address these issues in an integrated manner. In this work, we pioneer a data-centric paradigm and propose a novel framework, the Data-centric Reasoning Compiler (DCRC). The framework operates through three cohesive phases: (1) adversarial data construction, which synthesizes training examples with controlled noise to teach robustness; (2) multi-stage training that cultivates a Data-centric Structuring Agent (DSA) capable of explicit evidence auditing and program synthesis; and (3) a compile-and-execute inference process, where the DSA transforms user queries and retrieved documents into verifiable, executable reasoning programs. This data-driven framework ensures faithful numerical reasoning by design. We conduct extensive experiments on established offline benchmarks and further validate our framework through deployment in a real-world online financial QA system.

Momentum

Related topics

Latest papers 338