LLM Reliability

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

78 papers in the last four weeks, up 105% on the four weeks before. 0.8% of all new papers.

Jul 13Week of Sep 28

Latest papers 507

All topics

Which Models Perform Better in Inheritance Reasoning?
Jun 11, 2026Mohammed Amine Mouhoub, Chahinez BouchekifLLM Evaluation LLM Reliability
Paris Dauphine University · University of Abou Bekr Belkaïd
Authority, Truth, and Citation Bias: A Large-Scale Multi-Domain Benchmark for Studying Epistemic Susceptibility in Large Language Models
Jun 11, 2026Aryan Khurana, Aravind Ramana RN, Dhruv KumarLLM Evaluation Hallucination in Language Models
BITS Pilani, Pilani Campus, Rajasthan, India.
(Human) Attention Is (Still) All You Need: Human oversight makes AI-assisted social science reliable
Jun 11, 2026Chen Zhu, Xiaolu Wang, Weilong ZhangAI Agent Reliability AI-Assisted Scientific Research
China Agricultural University · University of Cambridge
Measuring Epistemic Resilience of LLMs Under Misleading Medical Context
Jun 10, 2026Hongjian Zhou, Xinyu Zou, Jinge Wu +19LLM Reliability Medical QA
University of Oxford · University of Washington · University College London +1
Reassessing High-Performing LLMs on Polish Medical Exams: True Competence or Bias-Driven Performance?
Jun 10, 2026Antoni Lasik, Jakub Pokrywka, Łukasz Grzybowski +7LLM Evaluation LLM Reliability
NASK National Research Institute · Adam Mickiewicz University · ARAAI Poland +4
Flaws in the LLM Automation Narrative
Jun 9, 2026George Perrett, Javae Elliott, Jennifer Hill +1LLM Evaluation Code Generation
New York University
PhantomBench: Benchmarking the Non-existential Threat of Language Models
Jun 9, 2026Haeji Jung, Hila GonenLLM Evaluation Hallucination in Language Models
University of British Columbia · Canada CIFAR AI Chair, Amii
Code Is More Than Text: Uncertainty Estimation for Code Generation
Jun 8, 2026Yuling Shi, Caiqi Zhang, Yuexian Li +4Confidence Estimation in Language Models LLM Reliability
Shanghai Jiao Tong University · University of Cambridge
Measurement Under Selection: Decoy-Calibrated Failure Audits for Language Models
Jun 8, 2026Vyzantinos Repantis, Ameya Gawde, Harshvardhan SinghLLM Evaluation Language Model Error Detection
The Educational Approaches to Virtual Reality Technologies Lab, University of Ioannina, Greece · Meta Platforms
Calibration of Structured Ignorance Certificates for Diagnosing Unknown Unknowns in Reasoning Models
Jun 7, 2026Subramanyam SahooHallucination in Language Models Language Model Calibration
Independent Researcher.
TrustMargin: Training-Free Arbitration between Parametric Memory and Retrieved Evidence in Large Language Models
Jun 7, 2026Jingyan Xu, Hong Shi, Yi Shan +4Retrieval-Augmented Generation LLM Grounding
DCST, Peking University · Beijing, China
Constrained Paraphrase Consistency for LLM Hallucination Detection
Jun 6, 2026Shanshan Lin, Dongsheng Hong, Sibo Ju +3LLM Reliability LLM Hallucination Detection
1Fuzhou University · 2Harbin Institute of Technology (Shenzhen) · 3Beijing University of Posts and Telecommunications
Stress-testing medical large language models reveals latent safety pathology beyond benchmark accuracy
Jun 6, 2026Yuan Shen, Xiaojun Wu, Linghua YuLLM Evaluation Language Model Safety Evaluation
College of Computer Science and Technology, Zhejiang University, PR China · The First Hospital of Jiaxing, Zhejiang Province, PR China
The ACUTE Protocol: Operationalizing Language Model Activations for Better Calibration, Utility, and Trust
Jun 5, 2026Nishant Subramani, Palash Goyal, Yiwen Song +4Confidence Estimation in Language Models Language Model Calibration
Carnegie Mellon University · Google · Scale AI
Where Instruction Hierarchy Breaks: Diagnosing and Repairing Failures in Reasoning Language Models
Jun 5, 2026Sanjay Kariyappa, G. Edward SuhLLM Auditing Instruction Following
NVIDIA
How reliable are LLMs when it comes to playing dice?
Jun 5, 2026Luca Avena, Gianmarco Bet, Bernardo BusoniLLM Evaluation LLM Prompting
Università degli Studi di Firenze Dipartimento di Matematica e Informatica ‘Ulisse Dini’
How Language Models Fail: Token-Level Signatures of Committed and Persistent Reasoning Failures
Jun 4, 2026Tanvi Thoria, Kiana Jafari, Marc R. Schlichting +1Language Model Error Detection LLM Reliability
Department of Computer Science, Stanford University · Department of Aeronautics and Astronautics, Stanford University
Decomposing Factual Sycophancy in Language Models: How Size and Instruction Tuning Shape Robustness
Jun 4, 2026Victor De Marez, Luna De Bruyne, Walter DaelemansLLM Evaluation LLM Sycophancy
Centre for Computational Linguistics, Psycholinguistics and Sociolinguistics University of Antwerp Antwerp, Belgium
When Tools Fail: Benchmarking Dynamic Replanning and Anomaly Recovery in LLM Agents
Jun 4, 2026Dongsheng Zhu, Xuchen Ma, Yucheng Shen +5AI Agent Reliability AI Agent Benchmarks
Shanghai AI Laboratory · East China Normal University · Soochow University +2
CaliDist: Calibrating Large Language Models via Behavioral Robustness to Distraction
Jun 4, 2026Mohammad Anas Jawad, Cornelia CarageaConfidence Estimation in Language Models Language Model Calibration
Computer Science, University of Illinois Chicago, USA.
Trust, but Don't Verify: Epistemic Blind Spots in LLM Source Evaluation
Jun 3, 2026Rohan N. Pradhan, Steve GoleyLLM Interpretability LLM Reliability
Amazon
Imbuing Large Language Models with Bidirectional Logic for Robust Chain Repair
Jun 3, 2026Zehua Cheng, Wei Dai, Jiahao Sun +1LLM Self-Correction Symbolic Reasoning
Department of Computer Science, University of Oxford, UK · Institute of Logic and Computation, TU Wien, Austria
Calibrating Overconfidence Without Sacrificing Confidence: Probe-Conditioned Head Intervention for LLMs
Jun 2, 2026Ke Li, Chongzhe Zhang, Zifan Zeng +3Confidence Estimation in Language Models Language Model Calibration
Huawei Heisenberg Research Center · EPFL · TU Berlin +1
Integrating Local and Global Entropy for Uncertainty Quantification in LLMs
Jun 2, 2026Johanne Medina, Tianyi Zhou, Keivin Isufaj +2Uncertainty Quantification Confidence Estimation in Language Models
Qatar Computing Research Institute, HBKU Doha, Qatar · KTH Royal Institute of Technology Stockholm, Sweden
Quantifying Faithful Confidence Expression in Large Reasoning Models
Jun 2, 2026Areeb Gani, Asal Meskin, Gabrielle Kaili-May Liu +1LLM Evaluation Confidence Estimation in Language Models
1Yale University
Clustered Self-Assessment: A Simple yet Effective Method for Uncertainty Quantification in Large Language Models
Jun 2, 2026Qi Cao, Takeshi Kojima, Andrew Gambardella +3LLM Evaluation Uncertainty Quantification
The University of Tokyo, Japan
Conformal Language Modeling via Posterior Sampling
Jun 2, 2026Nicolas Emmenegger, Theo X. Olausson, Armando Solar-Lezama +1Hallucination in Language Models Conformal Prediction
Massachusetts Institute of Technology
Testing LLM Arithmetic Reasoning Generalization with Automatic Numeric-Remapping Attacks
Jun 2, 2026Malia Barker, Bishal Lakha, Edoardo Serra +1Numerical Reasoning in Language Models Mathematical Reasoning Benchmarks
Department of Computer Science, Boise State University, Boise, ID, USA · University of L'Aquila, L'Aquila, Italy
Large Language Models Are Overconfident in Their Own Responses
Jun 2, 2026Mario Sanz-Guerrero, Manuel Mager, Katharina von der WenseConfidence Estimation in Language Models LLM Prompting
∇Johannes Gutenberg University Mainz, Germany · ♠University of Colorado Boulder, USA
Evaluating LLMs' Effectiveness on Real-World Consumer Device Repair Questions
Jun 2, 2026Atm Mizanur Rahman, Md Arid Hasan, Syed Ishtiaque Ahmed +1Multilingual Language Model Evaluation LLM Evaluation
University of Illinois Urbana-Champaign · University of Toronto

Jun 11, 2026, cs.CLJ/K move · Enter open · S save

Which Models Perform Better in Inheritance Reasoning?

Mohammed Amine Mouhoub, Chahinez Bouchekif

Paris Dauphine University · University of Abou Bekr Belkaïd

LLM Evaluation LLM Reliability Legal Reasoning LLM Reasoning

Open paper AskPDF arXiv

This paper presents the participation of team PSL in the QIAS 2026 Shared Task on Arabic Islamic inheritance reasoning. The task evaluates the ability of large language models to solve inheritance cases that require legal interpretation, multi-step reasoning, and precise numerical computation. We compare \textit{commercial} and \textit{open-source} models under a unified prompting strategy to assess their effectiveness in structured legal reasoning with minimal task-specific adaptation. \ Our results show a clear gap in reliability between the two model families. Commercial models demonstrate stronger performance in identifying eligible heirs, applying exclusion rules, and maintaining consistency across reasoning steps. In contrast, open-source models exhibit greater instability, particularly in cases involving dependent legal decisions and fractional share adjustments. The best performance is achieved by \textit{Gemini 2.5 Flash}, with an MRE of

0.989

Momentum

Related topics

Latest papers 507