Search research

Topics & fields

Large Language Model Evaluation245 papers Unified Evaluation222 papers Jailbreak Evaluations184 papers Agentic Evaluations149 papers Artificial Intelligence Evaluation134 papers Adversarial Evaluation114 papers Evaluation Benchmarks96 papers Safety Evaluation84 papers Evaluation Metrics76 papers Off-Policy Evaluation70 papers Multimodal Evaluation67 papers Unified Evaluation66 papers

391–420 of 24,951

Papers

Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
May 19, 2026Husnain Amjad, Raja Khurram Shahzad, Aamir Shahzad +1Mathematical Reasoning Large Language Model Reasoning
FAGER: Factually Grounded Evaluation and Refinement of Text-to-Image Models
May 18, 2026Youngsun Lim, Cusuh Ham, Pin-Yu Chen +1Multilingual Fact-Checking Text-To-Image
The Annotation Scarcity Paradox in Low-Resource NLP Evaluation: A Decade of Acceleration and Emerging Constraints
May 18, 2026Vukosi MarivateLegal Natural Language Processing Benchmarks Data Scarcity
SKG-Eval: Stateful Evaluation of Multi-Turn Dialogue via Incremental Semantic Knowledge Graphs
May 15, 2026Avijit Shil, Suman SamuiHuman Grounded Evaluation Knowledge Graphs
Holistic Evaluation and Failure Diagnosis of AI Agents
May 14, 2026Netta Madvil, Gilad Dym, Alon Mecilati +12Agentic Evaluations
Dimension-Level Intent Fidelity Evaluation for Large Language Models: Evidence from Structured Prompt Ablation
May 14, 2026GAng PengLarge Language Model Evaluation Ablation
Comparative Evaluation of Machine Learning Approaches for Minority-Class Financial Distress Prediction Under Class Imbalance Constraints
May 13, 2026Karan Sehgal, Khawar Naveed BhattiExtreme Class Imbalance Credit Risk Prediction
Measuring Five-Nines Reliability: Sample-Efficient LLM Evaluation in Saturated Benchmarks
May 11, 2026Eungyeup Kim, Chenchen Gu, Vashisth Tiwari +1Large Language Model Reliability Large Language Model Evaluation
From Controlled to the Wild: Evaluation of Pentesting Agents for the Real-World
May 11, 2026Pedro Conde, Henrique Branquinho, Valerio Mazzone +3Penetration Testing Vulnerability Detection & Repair
Urban-ImageNet: A Large-Scale Multi-Modal Dataset and Evaluation Framework for Urban Space Perception
May 11, 2026Yiwei Ou, Chung Ching Cheung, Jun Yang Ang +5Urban Environments Multimodal Dataset
Augmenting Human Evaluation with LLM Judges: How Many Human Reviews Do You Need?
May 8, 2026Jane Paik KimLarge Language Model Evaluation Large Language Model Judges
PolySQL: Scaling Text-to-SQL Evaluation Across SQL Dialects via Automated Backend Isomorphism
May 8, 2026Yotam Perlitz, Elad Venezian, Corentin Royer +2Text-To-Sql Sql
Query-efficient model evaluation using cached responses
May 8, 2026Hayden Helm, Ben Johnson, Carey PriebeModel Evaluation Responses
SCRuB: Social Concept Reasoning under Rubric-Based Evaluation
May 7, 2026Jamelle Watson-Daniels, Himaghna Bhattacharjee, Skyler Wang +11Social Reasoning Rubrics
Evaluation Awareness in Language Models Has Limited Effect on Behaviour
May 7, 2026Amelie Knecht, Lucas Florin, Thilo HagendorffLarge Language Model Evaluation Moral Reasoning
Adversarial Graph Neural Network Benchmarks: Towards Practical and Fair Evaluation
May 7, 2026Tran Gia Bao Ngo, Zulfikar Alom, Federico Errica +2Graph Neural Networks
Why Expert Alignment Is Hard: Evidence from Subjective Evaluation
May 6, 2026Tzu-Mi Lin, Wataru Hirota, Tatsuya Ishigaki +2Human Judgment
Evaluation Cards for XAI Metrics
May 6, 2026Rokas Gipiškis, Olga KurasovaXai Explainable Artificial Intelligence
Synthetic Users, Real Differences: an Evaluation Framework for User Simulation in Multi-Turn Conversations
May 4, 2026Yu Lu Liu, Hyokun Yun, Tanya Roosta +1User Simulation Multi-Turn Conversations
STABLEVAL: Disagreement-Aware and Stable Evaluation of AI Systems
May 4, 2026Akash Bonagiri, Gerard Janno Anderias, Saee Patil +6Artificial Intelligence Evaluation Disagreement
Principles and Guidelines for Randomized Controlled Trials in AI Evaluation
May 3, 2026Christopher Kelly, Angelica Chowdhury, Alexandra Campili +5Artificial Intelligence Evaluation Controlled Study
PepSpecBench: A Unified Evaluation Benchmark for Peptide Tandem Mass Spectrometry Prediction
May 3, 2026Zhiwen Yang, Pan Liu, Yifan Li +2Tandem Mass Spectra Evaluation Benchmarks
From Prediction to Practice: A Task-Aware Evaluation Framework for Blood Glucose Forecasting
May 1, 2026Alireza Namazi, Heman ShakeriContinuous Glucose Monitoring Early Warning
Persona-Grounded Safety Evaluation of AI Companions in Multi-Turn Conversations
Apr 30, 2026Prerna Juneja, Lika LomidzeArtificial Intelligence Companions Safety Evaluation
HuM-Eval: A Coarse-to-Fine Framework for Human-Centric Video Evaluation
Apr 28, 2026Bingzi Zhang, Kaisi Guan, Ruihua SongHuman Motion Generation Text-To-Video Models
AgentEval: DAG-Structured Step-Level Evaluation for Agentic Workflows with Error Propagation Tracking
Apr 26, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuAgentic Evaluations Agentic Workflows
dWorldEval: Scalable Robotic Policy Evaluation via Discrete Diffusion World Model
Apr 24, 2026Yaxuan Li, Zhongyi Zhou, Yefei Chen +2Robot Policies Robotwin
Coverage, Not Averages: Semantic Stratification for Trustworthy Retrieval Evaluation
Apr 22, 2026Andrew Klearman, Radu Revutchi, Rohin Garg +3Information Retrieval Metrics Stratification
Beyond Semantic Similarity: A Component-Wise Evaluation Framework for Medical Question Answering Systems with Health Equity Implications
Apr 21, 2026Abu Noman Md Sakib, Md. Main Oddin Chisty, Zijie ZhangDisparities Semantic Similarity Metrics
Proposing Topic Models and Evaluation Frameworks for Analyzing Associations with External Outcomes: An Application to Leadership Analysis Using Large-Scale Corporate Review Data
Apr 20, 2026Yura Yoshida, Masato Kanai, Masataka Nakayama +5Topic Modeling Sentiment Analysis

May 19, 2026 · cs.CLJ/K move · Enter open · S save

Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges

Husnain Amjad, Raja Khurram Shahzad, Aamir Shahzad, Mehwish Fatima

School of Electrical Engineering and Computer Science, National University of Sciences and Technology, Islamabad, Pakistan · Department of Communication, Quality Management and Information Systems, Mid Sweden University, Östersund Campus, Sweden · School of Computing, Data, and Mathematical Sciences, Western Sydney University, Indonesia

Mathematical reasoning is essential for problem-solving in education, science, and industry, serving as a crucial benchmark for evaluating artificial intelligence systems. As Large Language Models (LLMs) improve their reasoning capabilities, understanding how well they perform mathematical reasoning has become increasingly important. This survey synthesizes recent advancements in mathematical reasoning with LLMs through a structured analysis of datasets, architectures, training strategies, and evaluation protocols. Our systematic review encompasses approximately 120 peer-reviewed studies and preprints, examining the evolution of this research area and providing a unified analytical framework to understand current progress and limitations. Our study particularly introduces a unified taxonomy of mathematical datasets, distinguishing between pretraining corpora, supervised fine-tuning resources, and evaluation benchmarks across varying levels of reasoning complexity. A systematic analysis of reasoning architectures and training strategies, including tool integration, verifier-guided reasoning, and parameter-efficient adaptation, is presented to assess their effects on reasoning robustness and generalization. Moreover, a comparative evaluation of existing metrics highlights the gap between final-answer accuracy and process-level reasoning verification. By synthesizing insights across these areas, our analysis identifies recurring failure modes, such as reasoning faithfulness issues, benchmark biases, and generalization limitations, and outlines key research directions toward improving symbolic grounding, evaluation reliability, and the development of more robust and trustworthy LLM-based reasoning systems.

Mathematical Reasoning Large Language Model Reasoning Benchmarking Large Language Models Large Reasoning Models Evaluation Benchmarks

Open paper Ask about it PDF