Language Model Auditing

Ask about this topic

Momentum

22 papers in the last four weeks, up 100% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 110

All topics
CardsList

  1. Whose Voice Survives the Summary? A Voice-Retention Audit of LLM Employee Listening

    Sep 30, 2026Thilo Tamme, Anton Hantel, Bijan Khosrawi-RadText SummarizationSummarization Evaluation

    Technical University of Munich · Massachusetts Institute of Technology · Leuphana University Lüneburg

  2. Selecting The Most Informative Tokens in Natural Language Autoencoders

    Sep 29, 2026Federico Torrielli, Gianluca Barmina, Andrea Blasi Núñez +4LLM AuditingAutoencoders

    Department of Computer Science, University of Turin, Italy · Department of Mathematics and Computer Science, University of Southern Denmark, Denmark

  3. Louder, Longer, Livelier: Acoustic Shortcuts and Underspecified Rationales in Speech LLM Judges

    Sep 29, 2026Mingyue Huo, Shivam Mehta, Bhavin Jawade +2LLM-as-a-JudgeSpeech Quality Assessment

    University of Illinois Urbana-Champaign · Netflix

  4. Language Models Are "Insecure" Reporters

    Sep 28, 2026Jenny Y. Huang, Jiameng Fan, Ahmed Imtiaz Humayun +5Language Model Safety EvaluationDeception in Language Models

    Massachusetts Institute of Technology · Google Research · Harvard University

  5. RAGWarrant: Evidence-Preserving Governance for RAG Policy Promotion Under Quality, Cost, Latency, and Risk Constraints

    Sep 28, 2026Richard Krueger, Lucas Krause, Zach PocquetteRetrieval-Augmented GenerationRAG Evaluation

    Independent open-source research project

  6. Low-Cost Assays for Measuring Model Behavior Across Vendors and Releases

    Sep 24, 2026Tapan ParikhLLM SycophancyAI Agent Evaluation

    Cornell Tech

  7. Truth for Believable AI: Expressed Doubt, Provenance, and Belief Revision as an Engineerable Stance

    Sep 22, 2026Sebastian CochinescuLLM Uncertainty EstimationBelief Updating in LLMs

    University of Bucharest

  8. Auditing Proxy-Based Validation Across Text Spans

    Sep 22, 2026Daein Weon, Dong Ho KangBenchmark ValidityAutomated Evaluation

    Kookmin University · UStechlab

  9. Conduct Under Pressure: What Sixty Language Models Do When a User Pushes

    Sep 21, 2026Tapan ParikhLanguage Model Safety EvaluationPersuasion in Language Models

    Cornell Tech

  10. FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditing

    Sep 21, 2026Yusser Al Ghussin, Eva Gavaller, Cristina España-Bonet +2Multilingual Language Model EvaluationLLM Auditing

    Saarland University · German Research Center for Artificial Intelligence (DFKI) · Barcelona Supercomputing Center (BSC-CNS), Barcelona, Catalonia, Spain

  11. When Residualization Helps an Audit: Format Effects, Slice Gains, and Their Limits

    Sep 21, 2026Daein Weon, Dong Ho KangLanguage Model Bias EvaluationLanguage Model Auditing

    Kookmin University · UStechlab

  12. Competence-Preserving Resume Perturbations Expose Presentation Sensitivity in LLM Screening

    Sep 15, 2026Qiangju Chen, Yang XiaoLLM EvaluationLanguage Model Robustness

    1Macquarie University · 2The University of Melbourne

  13. Clean Scores, Buried Evidence, and Confident Wrong: A Receipt-Based Audit of Frontier Agentic QA

    Sep 14, 2026Luis M. SánchezData ProvenanceAI Agent Auditing

    Toryx Inc.

  14. More than half of recent astronomy papers are written with language-model assistance

    Sep 11, 2026Serat M. Saad, Yuan-Sen TingLanguage Model Auditing

    Department of Astronomy, The Ohio State University, 140 West 18th Avenue, Columbus, OH 43210, USA · Center for Cosmology and AstroParticle Physics (CCAPP), The Ohio State University, Columbus, OH 43210, USA · Max-Planck-Institut für Astronomie, Königstuhl 17, D-69117 Heidelberg, Germany

  15. Trust Me, I'm Your Developer: Self-Issued Authentication in Large Language Models

    Sep 10, 2026Syed Ghazanfar Abbas, Dongyan XuLLM SecurityLanguage Model Auditing

    Purdue University

  16. ContractEval: Query-Conditioned Execution Matching for Procedural Instruction Conformance

    Sep 10, 2026Praphul Singh, Shanu Kumar, Akshat Agarwal +1LLM Agent EvaluationInstruction Following

    Oracle Health AI · MBZUAI

  17. Watermarks Without Verification: AI Text Watermarking After the EU AI Act

    Sep 10, 2026Alexander Nemecek, Vipin Chaudhary, Erman AydayText WatermarkingAI Accountability

    Case Western Reserve University

  18. Decision Shifts, Lost Label Functionality, and an Inconclusive Grounding Audit in Correctness-Gated Multi-Teacher Distillation

    Sep 9, 2026Xiaofei FengLanguage Model Safety EvaluationMulti-Teacher Knowledge Distillation

    University of Illinois Urbana-Champaign

  19. The Audit Decides the Verdict: Instrument Effects Rival Demographic Bias in LLM Decision Audits

    Sep 8, 2026Siddharth Vohra, Manikandan RavikiranLanguage Model Bias EvaluationDemographic Bias in Language Models

    Carnegie Mellon University Amazon Web Services AI Native Pittsburgh, PA, USA · Indian Institute of Technology Mandi, India

  20. When Models Defer to Wrong Answers: A Robustness Audit of Source-Attributed Cues in Multiple-Choice QA

    Sep 8, 2026Manikandan Ravikiran, Siddharth VohraMultiple-Choice Question AnsweringLLM Reliability

    Indian Institute of Technology Mandi, India · Carnegie Mellon University Amazon Web Services AI Native Pittsburgh, PA, USA

  21. The Oversight Gap: What LLM Safety Monitors Miss, and Why It Is Not Capability

    Sep 7, 2026Xin XuLLM Safety BenchmarksLLM Safety Evaluation

    Carnegie Mellon University

  22. Harness-agnostic detection and immunization of reward hacking in self-evolving language models

    Sep 7, 2026Rongxin Yang, Yang Liu, Shang Luo +10Reward HackingLanguage Model Self-Improvement

    1Fullive-AI · 2Peking University · 4Nanyang Technological University +2

  23. Does the Selected Object Reach the Reader? Auditing Identity Handoffs in Grounded Language-Model Pipelines

    Sep 4, 2026Siddharth Vohra, Runmin Jiang, Xiaomo Li +1Question AnsweringRAG Evaluation

    Carnegie Mellon University, Pittsburgh, PA, USA · Amazon Web Services AI Native, Pittsburgh, PA, USA

  24. One note in three: a verified census of three deployed AI scribes, and the instrument that counted it

    Aug 31, 2026Sebastian Fox, Luke Markham, Ryan Lail +1Medical Report GenerationClinical Language Model Evaluation

    Composo

  25. Influence Is Not Authority: When Causal Guardrail Signals Make Legitimate Tool Use Look Like an Attack in Tool-Using LLM Agents

    Aug 30, 2026Tanzim Ahad, Ismail Hossain, Md Jahangir Alam +3LLM GuardrailsLLM Agent Security

    Department of Computer Science, University of Texas at El Paso, TX, USA 79902 · Department of Computer Science and Engineering, New Mexico Tech, Socorro, NM, USA 87801 · University of Illinois Urbana-Champaign, IL, USA

  26. Persona-Execution Separation: An Architecture Pattern for Evolving LLM Agents under Execution Audit

    Aug 27, 2026Yisen XiLLM Agent SecurityRuntime Enforcement for AI Agents

    Independent Researcher, Beijing, China.

  27. Excess Separability: Nuisance-Controlled Residual-Stream Probing for Benchmark Contamination Detection

    Aug 12, 2026Florian BraunBenchmark ContaminationData Contamination in Language Models

  28. When Explanations Betray Backdoors: Black-Box Auditing for Language Model Classifiers

    Aug 12, 2026Yang Liu, Ran ZouLLM Backdoor AttacksBackdoor Detection

    Department of Statistics and Operations Research, University of North Carolina at Chapel Hill · Department of Statistics, University of California, Irvine

  29. Located but Not Releasable: Silent Gate Inversion and Bounded Linear Release

    Aug 12, 2026Xining XunCausal Interventions in Language ModelsActivation Steering

    Tsingjiao Information Science (Beijing) Co., Ltd.

  30. AI Guardrail Survival under Single-Cycle Agentic Self-Summarization

    Aug 11, 2026Ted Kwartler, Alan Aqrawi, Arian AbbasiLLM GuardrailsLLM Safety Evaluation

    Harvard Extension School, Harvard University; Americas Advanced AI Practice Lead, Accenture. · Accenture.