Language Model Auditing

Ask about this topic

Momentum

22 papers in the last four weeks, up 100% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 110

All topics
CardsList

  1. Benchmarking LLM-Based Static Analysis for Secure Smart Contract Development: Reliability, Limitations, and Potential Hybrid Solutions

    May 11, 2026Stefan-Claudiu Susan, Andrei Arusoaie, Dorel LucanuLLM EvaluationSoftware Vulnerability Detection

    Department of Computer Science Alexandru Ioan Cuza University of Ias,i Ias,i, Romania

  2. Can You Keep a Secret? Involuntary Information Leakage in Language Model Writing

    May 11, 2026Ari Holtzman, Peter WestData LeakagePrivacy Leakage in Language Models

    University of Chicago · University of British Columbia

  3. The Translation Tax Is Not a Scalar: A Counterfactual Audit of English-Source Cue Inheritance in Chinese Multilingual Benchmarks

    May 8, 2026Zezheng Lin, Fengming Liu, Handi LiMultilingual Language Model EvaluationLanguage Model Auditing

  4. The Compliance Gap: Why AI Systems Promise to Follow Process Instructions but Don't

    May 3, 2026Kwan Soo ShinAI Agent BenchmarksLLM Honesty

    PolymathMinds AI Lab

  5. SCALE-LoRA: Auditing Post-Retrieval LoRA Composition with Residual Merging and View Reliability

    May 2, 2026Shuaipeng Zhou, Yu ZhangUncertainty QuantificationAdapter Tuning

    School of Computer Science and Technology, Shenyang University of Chemical Technology · 2Liaoning Key Laboratory of Intelligent Technology for Chemical Process Industry

  6. Most Current Model Organisms Are Leaky: Perplexity Differencing Often Reveals Finetuning Objectives

    May 1, 2026Mohammed Abu Baker, Luca Baroni, Dan WilhelmLanguage Model Safety EvaluationLanguage Model Auditing

    Meridian Visiting Researcher Programme.

  7. Semia: Auditing Agent Skills via Constraint-Guided Representation Synthesis

    May 1, 2026Hongbo Wen, Ying Li, Hanzhi Liu +4AI Agent AuditingStatic Code Analysis

    University of California, Santa Barbara Santa Barbara, CA, USA · University of California, Los Angeles Los Angeles, CA, USA · Fuzzland USA +1

  8. Latent Adversarial Detection: Adaptive Probing of LLM Activations for Multi-Turn Attack Detection

    Apr 30, 2026Prashant KulkarniPrompt Injection Attacks on LLMsAdversarial Example Detection

    Mountain View, CA

  9. Political Bias Audits of LLMs Capture Sycophancy to the Inferred Auditor

    Apr 30, 2026Petter Törnberg, Michelle SchimmelLLM SycophancyPolitical Bias in Language Models

    Institute of Logic, Language and Computation (ILLC), University of Amsterdam, Amsterdam, Netherlands

  10. Evaluating Epistemic Guardrails in AI Reading Assistants: A Behavioral Audit of a Minimal Prototype

    Apr 30, 2026Matthew Christian AgustinLanguage Model Safety EvaluationLLM Guardrails

    Responsible Innovation Lab Tempe, Arizona, USA

  11. Structured Security Auditing and Robustness Enhancement for Untrusted Agent Skills

    Apr 28, 2026Lijia Lv, Xuehai Tang, Jie Wen +2AI Agent AuditingAI Agent Security

    Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences

  12. Architecture Determines Observability of Transformers

    Apr 27, 2026Thomas CarmichaelTransformer InterpretabilityLanguage Model Error Detection

    Independent Researcher

  13. Escaping the Agreement Trap: Defensibility Signals for Evaluating Rule-Governed AI

    Apr 22, 2026Michael O'Herlihy, Rosa CatalàContent ModerationAutomated Evaluation

    Reddit, Inc.

  14. What Makes a Good AI Review? Concern-Level Diagnostics for AI Peer Review

    Apr 21, 2026Ming JinAutomated Peer ReviewGenerative AI Evaluation

    Bradley Department of Electrical & Computer Engineering Virginia Tech

  15. Semantic Needles in Document Haystacks: Sensitivity Testing of LLM-as-a-Judge Similarity Scoring

    Apr 20, 2026Sinan G. Aksoy, Alexandra A. Sabrio, Erik VonKaenel +1Semantic Textual SimilarityLLM-as-a-Judge

    Pacific Northwest National Laboratory · Washington University in St. Louis · Humana Inc.

  16. An Empirical Study of Multi-Generation Sampling for Jailbreak Detection in Large Language Models

    Apr 20, 2026Hanrui Luo, Shreyank N GowdaLanguage Model Safety EvaluationLLM Auditing

    School of Computer Science, University of Nottingham, Nottingham, United Kingdom.

  17. Catching The Correct Answer Trap: Characterising AI Tutor Blind Spots When Analysing Student Reasoning

    Apr 20, 2026Moiz Imran, Sahan BulathwelaIntelligent Tutoring SystemsEducational Assessment

    Department of Computer Science, University College London, UK · Centre for Artificial Intelligence, University College London, UK

  18. The Geometric Canary: Predicting Steerability and Detecting Drift via Representational Stability

    Apr 20, 2026Prashant C. RajuLanguage Model SteeringLLM Reliability

  19. Rater State Bias in RLHF Preference Data: An Audit Framework

    Apr 14, 2026Elena Kopteva, Vitaliy Hlynianyi-ZhukHuman Preference EvaluationHuman-in-the-Loop Annotation

    The Grainger College of Engineering, Department of Physics & Illinois Center for Advanced Studies of the Universe, University of Illinois Urbana-Champaign, Urbana, Illinois 61801, USA · Faculty of Applied Mathematics, Oles Honchar Dnipro National University, Dnipro, 49045, Ukraine · Department of Clinical Psychology, Kyiv Institute of Modern Psychology and Psychotherapy, Kyiv, 01133, Ukraine

  20. AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviors

    Feb 26, 2026Abhay Sheshadri, Aidan Ewart, Elias Kempf +8Benchmark AuditingAI Agent Evaluation

    Anthropic Fellows Program · Anthropic