LLM Reliability

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

78 papers in the last four weeks, up 105% on the four weeks before. 0.8% of all new papers.

Jul 13Week of Sep 28

Latest papers 507

All topics

How Much Prompt Is Enough? A Blackbox Minimization of Few-Shots in LLMs
Sep 28, 2026Ali Alfageeh, Rahul Gopinath, Amin AlipourLLM Prompting Few-Shot Prompting
University of Houston · The University of Sydney
A Polyphonic Conception of AI Understanding
Sep 28, 2026Matthieu Queloz, Pierre BeckmannLLM Interpretability LLM Reliability
University of Bern, Department of Philosophy · École Polytechnique Fédérale de Lausanne (EPFL) · Idiap Research Institute +1
Epistemic Policy Divergence in Multi-Turn LLM Contamination: A Protocol-Gradient Investigation
Sep 28, 2026Fahrell Giovanny, Geby Bayuningtyas, Sahrul Mukharom +1Knowledge Conflicts in Language Models Adversarial Attacks on LLMs
Independent Researcher, Tokyo, Japan · Independent Researcher, Jakarta, Indonesia · Department of Informatics, Institut Teknologi Sumatera, Lampung, Indonesia
Semantic Uncertainty Quantification Needs Factual Equivalence
Sep 28, 2026Joseph Hoche, Quentin Guimard, Gianni FranchiConfidence Estimation in Language Models Factual Consistency Evaluation
AMIAD, Pˆole Recherche Palaiseau, France
On the Limits of Metacognitive Monitoring in LLMs
Sep 28, 2026Dongqi Han, Yifan Yang, Dongsheng LiLanguage Model Error Detection Confidence Estimation in Language Models
Microsoft Research Asia
Applying Language Models in Clinical Medicine: Recent Trends and Perspectives
Sep 28, 2026Erik AertsLLM Reliability LLM Safety
When Words Fall Short: Iterative Synergy Between Verbalized Reasoning and Hidden Features for LLM Confidence Estimation
Sep 28, 2026Yekun Xu, Ante Wang, Jingyi Ren +3Confidence Estimation in Language Models LLM Reliability
College of AI, Tsinghua University, Beijing, China · Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China · Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China
Toward a Graded Measure of Belief Stability in Large Language Models
Sep 28, 2026Samantha Dies, Branden Fitelson, Tina Eliassi-RadLLM Reliability LLM Uncertainty Estimation
Northeastern University, 360 Huntington Ave, Boston, MA 02115 USA · Santa Fe Institute, 1399 Hyde Park Road, Santa Fe, NM 87501 USA
Lost with a Map: Conversational State and Behavioral Reliability in Language Models
Sep 27, 2026Atahan Dokme, Larry HeckSpoken Dialogue Systems LLM Reliability
Georgia Institute of Technology
Probability Contracts: Accuracy, Coherence, and Decisions Across LLM Interfaces
Sep 27, 2026Han Chen, Yingrui LiReasoning Consistency in Language Models LLM Reliability
Independent Researcher
A Cheap Verifier is Good Enough: LLM Post-training is Robust to Erroneous Rewards
Sep 27, 2026Andreas Plesner, Curtis Northcutt, Francisco Guzmán +1Reward Modeling LLM Reliability
ETH Zurich · Handshake AI
Feedback Makes Perfect: A Closed-Loop Framework for NL-to-STL Translation
Sep 27, 2026Bowen Ye, Xiang YinSignal Temporal Logic LLM Reliability
Shanghai Jiao Tong University Shanghai, China
Pinned and Still Unstable: Within-Judge Verdict Variance and the Noise Floor of LLM-as-Judge Leaderboards
Sep 27, 2026Krishna Chytanya AyyagariLLM Evaluation LLM-as-a-Judge
Google
Beyond Average Safety: Chance-Constrained LLM Fine-tuning
Sep 24, 2026Taha Entesari, Mahyar FazlyabStochastic Optimization Chance-Constrained Optimization
Department of Electrical and Computer Engineering Johns Hopkins University
Between the Commits: Process, Error, and Claim Reliability in a Wholly AI-Authored Codebase
Sep 24, 2026Douglas LeithAI Agent Reliability Data Provenance
Trinity College Dublin Dublin, Ireland
Exact Feedback Is Not Control: Evaluating Text-based Closed-Loop Revision in LLMs
Sep 23, 2026Haitong Jiang, Chunlin Liu, Yile Wang +1LLM Self-Correction Controllable Text Generation
College of Computer Science and Software Engineering, Shenzhen University
Greedy Decoding Is Not Precision-Invariant: Cross-Precision Output Divergence in LLM Inference
Sep 22, 2026Gaoyuan Du, Anam Nawaz Khan, Rex Zhou +4Language Model Decoding LLM Inference
University of Tennessee, Knoxville · University of Chicago · Amazon +1
Calibration as a First-Class Criterion in LLM Evaluation
Sep 22, 2026Mario Sanz-Guerrero, Katharina von der WenseLLM Evaluation Confidence Estimation in Language Models
Johannes Gutenberg University Mainz, Germany · University of Colorado Boulder, USA
On the security and privacy of LLMs in Mobility
Sep 22, 2026Mauro Conti, Lorenzo Perinello, Umberto SalviatiIntelligent Transportation Systems LLM Security
University of Padova Italy · Fondazione Bruno Kessler Italy
Clarification Is Not Correction: LLMs Fail to Let Go
Sep 21, 2026Jianzhe Lin, Xiaolin Li, Fei Wang +3LLM Reliability LLM Context Management
MetaAI
LLJ Cards: Best practices for the Use of LLMs as Judges
Sep 21, 2026Khaoula Chehbouni, Melina Medjdoub, Florian Carichon +2LLM Evaluation LLM-as-a-Judge
McGill University · Mila - Quebec AI Institute · Concordia University
ChartJudgeBench: Evaluating LMM Judges for Chart-to-Code Generation
Sep 21, 2026Lijian Wu, Henry Hengyuan Zhao, Zijian Zhang +3VLM Evaluation LLM Evaluation
CSU-JPG, Central South University · National University of Singapore
Constrained Decoding Eliminates Structural Failures in Small LLMs but Reveals a Scale-Dependent Semantic Gap
Sep 20, 2026Akash ChavanConstrained Decoding LLM Evaluation
EnterpriseVal: Quantifying the Efficacy, Reliability and Value of Generative AI in the Enterprise
Sep 18, 2026Abbas Raza Ali, Muhammad Ajmal Siddiqui, Moona ZahidHuman-in-the-Loop Evaluation LLM Reliability
Citigrounp, Inc, London, United Kingdom · Ernst & Young LLP, London, United Kingdom · NVIDIA Corporation, London, United Kingdom
An Analysis of Training-Free Self-Reported Confidence in Language Models
Sep 17, 2026Lukas Meyer, Sofia Rossi, Wei Chen +2Confidence Estimation in Language Models Language Model Self-Assessment
DreamAI
Intrinsic Sequence-Likelihood Confidence in Retrieval-Dominated Extractive QA: Two Pre-Specified Negatives, and What They Do and Do Not Attribute
Sep 17, 2026Gunwoo Lee, Changmin Sung, Sang-Hwan Gwak +3LLM Evaluation Selective Prediction
Large-scale AI Research Center, Korea Institute of Science and Technology Information (KISTI), 245 Daehak-ro, Yuseong-gu, Daejeon, 34141, Republic of Korea. · Department of Applied AI, University of Science and Technology (UST), 217 Gajeong-ro, Yuseong-gu, Daejeon, 34113, Republic of Korea.
Reproducibility is not construct validity: LLM measurement of institutionally situated communication
Sep 17, 2026Veronika Batzdorfer, Carlo Romano Marcello Alessandro SantagiustinaLLM Evaluation LLM-Assisted Annotation
KIT · Department of Sociology and Computational Sociology, Karlsruhe Institute of Technology (KIT), Douglasstr. 24, 76133 Karlsruhe, Germany · ALMAnaCH, médialab, Sciences Po +2
Chain-of-Thought Entropy as a Reliability Signal: A Preregistered Reproduction
Sep 17, 2026Theodore O. CochranNumerical Reasoning in Language Models LLM Reliability
AI for Altruism
Playing log(N)-Questions over Wikipedia Abstracts: How Per-Round Errors Compound Under Information Asymmetry
Sep 16, 2026Peter PotashLLM Evaluation Pragmatic Reasoning in Language Models
WaveTLM: Reliable Time-Series Language Modeling through Task Compilation
Sep 16, 2026Jiahui Chen, Bingke Zhu, Hongyu Pan +1LLM Reliability Time-Series Foundation Models
Institute of Automation, Chinese Academy of Sciences Beijing, China

Sep 28, 2026, cs.SEJ/K move · Enter open · S save

How Much Prompt Is Enough? A Blackbox Minimization of Few-Shots in LLMs

Ali Alfageeh, Rahul Gopinath, Amin Alipour

University of Houston · The University of Sydney

LLM Prompting Few-Shot Prompting LLM Reliability Prompt Optimization Prompt Compression

Open paper AskPDF arXiv

Prompts are the primary mechanism for directing the behavior of large language models (LLMs). Yet the internal structure and causal hierarchy of prompts remain poorly understood: which parts are causally necessary and which are redundant is an open question. This opacity can have severe consequences. Subtle prompt variations can silently shift model outputs in critical software systems, and engineers lack techniques to reason about prompt reliability. We present \framework, a blackbox prompt-minimization framework that reduces few-shot prompts to their necessary minimal subset. We use a case study to apply \framework to a few-shot learning system and demonstrate the insights that this framework can provide. Our experiments show that few-shot exemplars can be reduced by a mean of 65.3%~

\pm

~15.8% in character count while fully preserving propositional output fidelity. The models preferentially retain logical identifiers and constraint declarations while discarding natural language prose and cross-prompt relational annotations. Our analysis also shows that some models are universal encoders, able to produce highly legible yet minimized prompts, while others are universal decoders, able to interpret minimized prompts from most other models. By identifying which components are indispensable, \framework provides a principled basis for prompt compression and structural analysis of few-shot exemplars.

Momentum

Related topics

Latest papers 507