LLM Reliability

LLM: Large Language Model

Overview

It asks not only whether a model gets an answer right, but whether it knows when it may be wrong, follows constraints, uses evidence faithfully, and behaves consistently across repeated or multi-turn use. Researchers test these properties with adversarial and perturbation benchmarks, repeated-run and cross-language evaluations, domain-specific tasks, and audits of LLM judges and tool-using agents. Common approaches include estimating uncertainty from logits, hidden representations, or response agreement; selective abstention and conformal calibration; retrieval and symbolic verification; and training methods that reward grounded answers, correction, or hesitation. The central tension is that benchmark accuracy and verbal confidence often fail to predict reliability on individual cases, long tasks, or shifted inputs, while stronger checks and abstention trade coverage, latency, and cost for safer decisions.

Latest papers

CardsList

866 papers92 papers in the last four weeks, down 21% on the four weeks before. 0.7% of all new papers.

  1. LLM-Generated Counterfactual Stress Scenarios for Portfolio Risk Simulation via Hybrid Prompt-RAG Pipeline

    Nov 26, 2025Masoud SoleimaniLLM ReliabilityQuantitative Finance

  2. Honesty over Accuracy: Trustworthy Language Models through Reinforced Hesitation

    Nov 14, 2025Mohamad Amin Mohamadi, Tianhao Wang, Zhiyuan LiLLM AbstentionLLM Hallucination Mitigation

  3. A Quantitative Study of Sustained Focus in Large Language Models via Repetitive Deterministic Prediction Tasks

    Nov 2, 2025Wanda Hou, Leon Zhou, Hong-Ye Hu +3LLM EvaluationLLM Reliability

  4. Reasoning Up the Instruction Ladder for Controllable Language Models

    Oct 30, 2025Zishuo Zheng, Vidhisha Balachandran, Chan Young Park +2LLM Safety AlignmentInstruction Hierarchy for Language Models

  5. LLMs Can Get "Brain Rot": A Pilot Study on Twitter/X

    Oct 15, 2025Shuo Xing, Junyuan Hong, Yifan Wang +5LLM TrainingLLM Reliability

  6. Failure-Driven Workflow Refinement

    Oct 11, 2025Jusheng Zhang, Kaitong Cai, Qinglin Zeng +4LLM Agent Workflow OptimizationLLM Reliability

  7. In-Place Feedback: Reliable Refinement for Multi-Turn Expert-LLM Collaboration

    Oct 1, 2025Youngbin Choi, Minjong Lee, Saemi Moon +4LLM ReliabilityMulti-Turn Language Model Reasoning

  8. Rethinking Reward Models for Multi-Domain Test-Time Scaling

    Oct 1, 2025Dong Bok Lee, Seanie Lee, Sangwoo Park +12Reward ModelingLLM Evaluation

  9. MedMMV: A Controllable Multimodal Multi-Agent Framework for Reliable and Verifiable Clinical Reasoning

    Sep 29, 2025Hongjun Liu, Yinghao Zhu, Yuhui Wang +3Clinical Reasoning in Language ModelsMLLM Reasoning

  10. Training-free Truthfulness Detection via Sparse MLP Value Vectors

    Sep 22, 2025Runheng Liu, Heyan Huang, Xingchen Xiao +2LLM ReliabilityLLM Hallucination Detection

  11. Clotho: Measuring Task-Specific Pre-Generation Test Adequacy for LLM Inputs

    Sep 22, 2025Juyeon Yoon, Somin Kim, Robert Feldt +1LLM EvaluationLLM Reliability

  12. Geometric Uncertainty for Detecting and Correcting Hallucinations in LLMs

    Sep 17, 2025Edward Phillips, Sean Wu, Soheila Molaei +3Uncertainty QuantificationLLM Hallucination Mitigation

  13. Cross-Layer Attention Probing for Fine-Grained Hallucination Detection

    Sep 4, 2025Malavika Suresh, Rahaf Aljundi, Ikechukwu Nkisi-Orji +1LLM Hallucination DetectionLLM Reliability

  14. Exploring the Limits of Large Language Models as a Reflection of Human Cognition: An Illustration in the Context of Moral Judgement

    Aug 9, 2025Sarah Schröder, Thekla Morgenroth, Ulrike Kuhl +2LLM ReliabilityLarge Language Model-Based Social Simulation

  15. Addressing Benchmarking Gaps in Large Language Models for Health and Medicine with Dynamic Red-Teaming

    Jul 30, 2025Jiazhen Pan, Bailiang Jian, Paul Hager +20LLM Safety BenchmarksLanguage Model Safety Evaluation

  16. Post-Training Large Language Models via Reinforcement Learning from Self-Feedback

    Jul 29, 2025Carel van Niekerk, Renato Vukovic, Benjamin Ruppik +3RL for Language ModelsRL for Language Model Reasoning

  17. A Mathematical Theory of Discursive Networks

    Jul 9, 2025Juan B. GutiérrezLLM ReliabilityMulti-Agent LLM Systems

  18. The Trilemma of Truth in Large Language Models

    Jun 30, 2025Germans Savcisens, Tina Eliassi-RadLLM InterpretabilityLLM Reliability

  19. FailureSensorIQ: A Multi-Choice QA Dataset for Understanding Sensor Relationships and Failure Modes

    Jun 3, 2025Christodoulos Constantinides, Dhaval Patel, Shuxin Lin +3Question AnsweringLLM Reliability

  20. Popular but Wrong: Understanding and Mitigating LLM Overconfidence through Knowledge Popularity

    May 23, 2025Shiyu Ni, Keping Bi, Jiafeng Guo +1LLM Uncertainty EstimationLLM Hallucination Mitigation

  21. Towards Reliable Long-Context Reasoning: Detecting Unanswerable Questions via FactGuard

    Apr 8, 2025Qian-Wen Zhang, Biao Liu, Fang Li +5LLM ReliabilityQuestion Answering

  22. HoT: Highlighted Chain of Thought for Referencing Supporting Facts from Inputs

    Mar 3, 2025Tin Nguyen, Logan Bolton, Mohammad Reza Taesiri +2LLM GroundingLLM Answer Verification

  23. Prompting the Unknown: Understanding Response Uncertainty in Large Language Models

    Jul 20, 2024Ze Yu Zhang, Arun Verma, Finale Doshi-Velez +1LLM PromptingLLM Reliability