LLM Reliability

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

78 papers in the last four weeks, up 105% on the four weeks before. 0.8% of all new papers.

Jul 13Week of Sep 28

Latest papers 507

All topics

Lost in Compaction: Evaluating Side-Constraint Loss under Context Compaction
Jul 31, 2026Zhiqi Wang, Yichi Zhang, Dongwon Lee +1Persistent Memory for Language Models Instruction Following
The Pennsylvania State University
Reflection or Re-Generation? Why LLM Revision Fails Where Human Revision Succeeds
Jul 31, 2026Yefan Tao, Gerald Friedland, Madhusudhanan Chandrasekaran +1LLM Self-Correction LLM Self-Refinement
Amazon
To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing
Jul 30, 2026Amir M. Ebrahimi, Mohammed Mehedi Hasan, Aaditya Bhatia +2LLM Reliability Language Model Editing
School of Computing, Queen’s University, Kingston, Ontario, Canada
When Derived Measurements Mislead: Quantifying and Mitigating LLM Over-Trust with Privileged-Modality Reliability Evidence
Jul 30, 2026Zongheng Guo, Tao Chen, Tianli Li +6LLM Evaluation LLM Reliability
Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy · State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou, China · Department of Integrated Traditional Chinese and Western Medicine Cardiology, China-Japan Friendship Hospital, Beijing, China +2
LLMs struggle to simulate human belief updates in controlled environments
Jul 30, 2026Sebastian Pohl, Harsh Mehta, Pranav Mambayil +4Human Behavior Simulation LLM Reliability
IT:U, Interdisciplinary Transformation University Austria, Altenberger Straße 66c, 4040 Linz, Austria
(Towards) Scalable Reliable Automated Evaluation with Large Language Models
Jul 30, 2026Bertil Braun, Martin ForellLLM Evaluation Pairwise Comparison
KIT
AI and Authenticity in Islamic Research: A Critical Evaluation of Generative AI Reliability, Hallucination, and Source Fidelity in Quranic, Hadith, and Fiqh Knowledge
Jul 30, 2026Muhammad Sajjad AkbarQuestion Answering LLM Reliability
School of Computer Science, The University of Sydney, Australia
GGC: Selective Query Correction for Reliable Text-to-SPARQL Generation
Jul 30, 2026Ziyi Yang, Thanh-Son Nguyen, Tuan Anh Nguyen +1LLM Self-Correction LLM Reliability
1Nanyang Technological University, Centre for Info. Sciences and Systems · Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR), Singapore
Position: Evaluation Scores Are Perishable Knowledge Claims
Jul 28, 2026Sankalp Gilda, Shlok GildaLLM Evaluation LLM Reliability
DeepThought Solutions · Meta · University of Florida
LLM4OSC: Profile-Bound Natural Language Control with Deterministic Validation for Open Sound Control
Jul 28, 2026Yuan-Yi FanLanguage Model-Based Control LLM Reliability
Los Angeles
How Do LLMs Read Bug Reports? An Empirical Study of Attention in LLMs for Automated Program Repair
Jul 28, 2026Ramtin Ehsani, Irene Manotas, Saurabh Pujar +2Automated Program Repair LLM Interpretability
Drexel University Philadelphia, PA, USA · IBM Research Yorktown, NY, USA
AI's Capability in Assisting Scientific Research in Physics, Astrophysics, and Cosmology I: Literature Review
Jul 28, 2026Anamaria Hell, Kateryna Vovk, Veena Krishnaraj +18AI-Assisted Scientific Research LLM Reliability
IPMU
Beyond Epistemia: Epistemic Schizologia and Large Language Models as Techno-Semiotic Machines
Jul 28, 2026Federico Cabitza, Gianluca ColomboHuman-AI Interaction LLM Reliability
Department of Informatics, Systems and Communication, University of Milano-Bicocca, Milan, Italy · Digital Health and Wellbeing Center, Fondazione Bruno Kessler, Trento, Italy · Oneofftech-UG (haftungsbeschränkt), Berlin, Germany
Many-body Tipping Dynamics of ChatGPT-like AIs
Jul 28, 2026Frank Yingjie Huo, Neil F. JohnsonLLM Reliability LLM Safety
Department of Physics, The George Washington University, Washington, DC, USA
Reason-Mediated Behavioral Models for Auditing LLM Social Simulators
Jul 27, 2026Atharva Pandey, Gautam JajooLLM Auditing Human Behavior Simulation
Kairosity
Retrieval-Augmented Large Language Models as Components of Cognitive Computing architecture for Regulatory Knowledge Management
Jul 27, 2026Dariusz Nowak-NovaRetrieval-Augmented Generation Knowledge Augmentation for Language Models
1WSB University, Poland
CONSISTRE: A Unified Consistency-Aware Framework for Document-Level Relation Extraction with Large Language Models
Jul 27, 2026Mingxuan SunRelation Extraction Document Understanding
Department of Computer Science, Universit´e de Sherbrooke, Sherbrooke, Canada
Accuracy Hides How Language Models Fail: Measuring Failure States Under Matched Output Budgets
Jul 27, 2026Zongyou Yang, Yinghan HouLLM Evaluation LLM Answer Verification
Dyson School of Design Engineering Imperial College London London, United Kingdom · Department of Electrical and Electronic Engineering Imperial College London London, United Kingdom
Language Shapes Instruction Hierarchy Compliance in Multilingual LLMs
Jul 26, 2026Jiwon Moon, Yerin Hwang, Kyomin JungMultilingual Language Model Evaluation LLM Security
IPAI, Seoul National University · Dept. of ECE, Seoul National University
Confidently Wrong: Exception Chain Collapse in Frontier LLM Rule Evaluation
Jul 25, 2026Paul Simpson, John Kozak, Lisa DoakeLLM Reliability Neuro-Symbolic AI
Aethis (aethis.ai)
Toward Automated Detection of Documentation Inconsistencies in Electronic Health Records
Jul 24, 2026Jian Lu, Panyu Chen, Miriam Treggiari +5LLM Reliability Clinical Language Model Evaluation
Duke University, 2424 Erwin Road, Durham, 27705, North Carolina, USA. · Columbia University Medical Center, 622 West 168 Street, New York, 10032, NY, USA. · Vanderbilt University Medical Center, 2525 West End Avenue, Nashville, 37203, TN, USA.
Modeling Memory-Dependent Reliability of LLMs: A Hidden Markov Model
Jul 24, 2026Robab Aghazadeh Chakherlou, Siddartha Khastgir, Peter Popov +1LLM Evaluation Hierarchical Bayesian Modeling
WMG, University of Warwick, Coventry, United Kingdom · Centre for Software Reliability, City St George’s, University of London, London, United Kingdom
Opaque Epistemic Mediation: How LLM Deployment Configurations Shape the Validation of Pseudo-Science
Jul 24, 2026Davide Scarso, Hugo Noronha de Almeida, Joaquim PinaLLM Evaluation LLM Auditing
CIUHCT - Interuniversity Centre for the History of Science and Technology · MARE - Marine and Environmental Sciences Centre, ARNET - Aquatic Research Network Associate Laboratory
Semiotic logical hexagon theory for LLM logical reasoning
Jul 24, 2026Yunyao Zhang, Xinglang Zhang, Zeliang Chen +2Logical Reasoning LLM Reliability
aHuazhong University of Science and Technology, , Wuhan, 430074, Hubei, China
Trustworthiness Costs of Domain Adaptation in Small Language Models:A Cross-Architecture Empirical Study
Jul 23, 2026Ramesh B. ParamkushamLanguage Model Safety Evaluation Domain Adaptation
Independent Researcher; Adjunct Faculty, University of the Cumberlands, Williamsburg, KY, USA
Euclid-MCP: A Model Context Protocol Server for Deterministic Logical Reasoning via Prolog
Jul 23, 2026Bartolomeo BoglioloLogic Programming LLM Reliability
Capital Markets LLM Reliability Score (CM-LRS): From Plausible to Bankable
Jul 23, 2026Prerit AhujaFinancial Services LLM Evaluation
Independent Researcher · London-based
Reexamining zero-shot summarization: Empirical investigation of trustworthiness of LLM-summarizers
Jul 23, 2026Vasudha Bhatnagar, Purnima Bindal, Vikas Kumar +1LLM Evaluation Text Summarization
Department of Computer Science, University of Delhi, Delhi, 110007, Delhi, India · Department of Statistics, Ramjas College, University of Delhi, Delhi, 110007, Delhi, India
LegalCiteTrust: Benchmarking Citation Trustworthiness in Chinese Long-Form Legal Research Reports
Jul 23, 2026Yunhan Li, Mingjie Xie, Zeyang Shi +2Legal Citation Verification Citation Verification
Faculty of Data Science, City University of Macau, Macau · Southern University of Science and Technology, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, China
Reinforcement Learning for Large Language Model Selective Evidence Adoption from Contaminated Retrieval Results
Jul 22, 2026Yanyu Chen, Yue Li, Yongyi Cui +2RL for Language Models Retrieval-Augmented Generation
East China Normal University · Shandong University

Jul 31, 2026, cs.CLJ/K move · Enter open · S save

Lost in Compaction: Evaluating Side-Constraint Loss under Context Compaction

Zhiqi Wang, Yichi Zhang, Dongwon Lee, Yuchen Yang

The Pennsylvania State University

Persistent Memory for Language Models Instruction Following LLM Reliability LLM Context Management Context Compression

Open paper AskPDF arXiv

When the context window is under pressure, LLM systems compact prior context to continue ongoing tasks. We identify a class of user-issued instructions, Session Constraints (SCs), such as "do not delete any emails until I confirm," that are meant to constrain LLM's behavior for the remainder of a session but are silently dropped during compaction. To quantify this loss, we introduce COMPINT, an evaluation suite that evaluates compactors across three long-context scenarios: multi-turn chat, agentic trajectory, and long-horizon research. Current compactors retain only 17% of injected SCs on average, and most perform worse than running the same task without compaction. Retention varies sharply with compactor, prompt, context length, SC phrasing, and injection location, showing that the loss is systematic rather than tied to any single setting. We propose an SC-aware extractor that runs alongside the compactor as a plug-and-play module, achieving over 90% retention across all three scenarios without modifying the compactor or LLM. The COMPINT evaluation suite and accompanying implementation are available at https://github.com/ZhiqiEliWang/compaction-integrity.

Momentum

Related topics

Latest papers 507