Language Model Robustness

Follow this topic Ask about this topic

Momentum

37 papers in the last four weeks, up 54% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 338

All topics

From Correlation to Cause: A Five-Stage Methodology for Feature Analysis in Transformer Language Models
May 21, 2026Caleb MunigetyTransformer Interpretability Causal Reasoning in Language Models
Independent Researcher
Is a Document Educational or Just Wikipedia-Style? -- Pitfalls of Classifier-Based Quality Filtering
May 21, 2026Mateusz Klimaszewski, Piotr AndruszkiewiczLanguage Model Pretraining Training Data Curation
Warsaw University of Technology · 2IDEAS Research Institute
Do as I Say, Not as I Do: Instruction-Induction Conflict in LLMs
May 19, 2026Carolina Camassa, Derek ShillerLLM Evaluation Language Model Self-Assessment
Future Impact Group · Rethink Priorities
Generalization or Memorization? Brittleness Testing for Chess-Trained Language Models
May 17, 2026Ethan TangLLM Evaluation Language Modeling
School of Computing and Augmented Intelligence · Arizona State University · Tempe, AZ 85281
Towards Robust Argumentative Essay Understanding via TIDE: An Interactive Framework with Trial and Debate
May 17, 2026Zheqin Yin, Yupei Ren, Yadong Zhang +2Argument Mining Automated Essay Scoring
School of Computer Science and Technology, East China Normal University · 2Shanghai Institute of Artificial Intelligence for Education, East China Normal University · Lab of Artificial Intelligence for Education, East China Normal University
Artificial Intolerance: Stigmatizing Language in Clinical Documentation Skews Large Language Model Decision-Making
May 17, 2026Jen-tse Huang, Didi Zhou, Faith Kamau +5Social Bias in Language Models Clinical Language Model Evaluation
1Johns Hopkins University.
Stop Drawing Scientific Claims from LLM Social Simulations Without Robustness Audits
May 17, 2026Jinyi Ye, Lei Cao, Ding Chen +1Human Behavior Simulation Agent-Based Modeling
Thomas Lord Department of Computer Science, University of Southern California · Annenberg School for Communication and Journalism, University of Southern California · Marshall School of Business, University of Southern California
Multi-LLM Systems Exhibit Robust Semantic Collapse
May 16, 2026Weiyi Kong, Shiyang Lai, Jinghua Piao +1Multi-Agent LLM Systems Language Model Robustness
Department of Electrical Computer Engineering, University of Toronto; Toronto, M5S 3G4, Canada · Department of Sociology, University of Chicago; Chicago, 60637, USA · 3Knowledge Lab, University of Chicago; Chicago, 60637, USA +3
Evaluation Drift in LLM Personality Induction: Are We Moving the Goalpost?
May 16, 2026Prateek Rajput, Yewei Song, Iyiola E. Olatunji +2Personality Modeling in Language Models LLM Post-Training
University of Luxembourg, Esch-sur-Alzette, Luxembourg
Reasoners or Translators? Contamination-aware Evaluation and Neuro-Symbolic Robustness in Tax Law
May 15, 2026Parisa Kordjamshidi, Samer Aslan, Madhavan Seshadri +2Data Contamination in Language Models Neuro-Symbolic Reasoning
Bloomberg · Michigan State University
Judge Circuits Explain Format-Induced Inconsistency in LLM-as-a-Judge
May 15, 2026Nils Feldhus, Tanja Baeumel, Elena Golimblevskaia +10LLM-as-a-Judge LLM Interpretability
CLCG, University of Groningen · German Research Center for Artificial Intelligence (DFKI) · Centre for European Research in Trusted AI (CERTAIN) +5
PRISM: Prompt Reliability via Iterative Simulation and Monitoring for Enterprise Conversational AI
May 15, 2026Keshava Chaitanya, Jahnavi GundakaramLLM Prompting LLM Agent Evaluation
LPDS: Evaluating LLM Robustness Through Logic-Preserving Difficulty Scaling
May 14, 2026Philipp Mondorf, Samuel J. Bell, Jesse Dodge +1LLM Evaluation Neural Network Robustness
FAIR at Meta · LMU Munich · Munich Center for Machine Learning +1
Does RAG Know When Retrieval Is Wrong? Diagnosing Context Compliance under Knowledge Conflict
May 14, 2026Yihang Chen, Pin Qian, Su Wang +4Retrieval-Augmented Generation Knowledge Conflicts in Language Models
1Georgia Institute of Technology · 2Carnegie Mellon University · University of California San Diego
Fine-tuning with Hierarchical Prompting for Robust Propaganda Classification Across Annotation Schemas
May 13, 2026Lukas Stähelin, Veronika Solopova, Max Upravitelev +6LLM Prompting LLM Fine-Tuning
Technische Universität Berlin, QU Lab, XplaiNLP Group, Berlin, Germany · German Research Center for Artificial Intelligence (DFKI), Berlin, Germany · Centre for European Research in Trusted AI (CERTAIN)
LIFT: Last-Mile Fine-Tuning for Table Explicitation
May 13, 2026Divij Khaitan, Ashish TiwariSupervised Fine-Tuning Fine-Tuning
Microsoft Corporation Bangalore, India · Microsoft Corporation Redmond, WA 98052 USA
GateKD: Confidence-Gated Closed-Loop Distillation for Robust Reasoning
May 13, 2026Kasidit Sermsri, Teerapong PanboonyuenCoT Distillation Language Model Distillation
†Chulalongkorn University · ‡MARSAIL · §PBYAIL (Panboonyuen AI Lab)
Precision Tracked Transformer via Kalman Filtering, Kriging and Process Noise
May 12, 2026Bo Long, Deepak Agarwal, Jelena Markovic-Voronov +2Bayesian Filtering Transformer
LinkedIn Core AI
Early Data Exposure Improves Robustness to Subsequent Fine-Tuning
May 12, 2026Lawrence Feng, Gaurav R. Ghosal, Jacob Mitchell Springer +2Language Model Pretraining Continual Learning for LLMs
Carnegie Mellon University
LGMT: Logic-Grounded Metamorphic Testing for Evaluating the Reasoning Reliability of LLMs
May 12, 2026Zenghui Zhou, Man Li, Xiaoke Fang +3LLM Evaluation Automated Software Testing
School of Automation Science and Electrical Engineering, Beihang University, Xueyuan Road, Haidian District, Beijing, 100191, China
How Useful Is Cross-Domain Generalization for Training LLM Monitors?
May 12, 2026Sam Martin, Fabien RogerSupervised Fine-Tuning Cross-Domain Generalization
Anthropic Fellows Program · Anthropic
Metaphor Is Not All Attention Needs
May 12, 2026Olga Sorokoletova, Francesco Giarrusso, Giacomo De Luca +6Adversarial Attacks on LLMs Jailbreak Attacks
Sapienza University of Rome Department of Computer, Control and Management Engineering Sapienza University of Rome Via Ariosto 25, Rome 00185, Italy · DEXAI – Icaro Lab · University of Rome Tor Vergata +1
SAGE: Scalable Automated Robustness Augmentation for LLM Knowledge Evaluation
May 12, 2026Xiaoyuan Li, Yuzhe Wang, Moxin Li +6LLM Evaluation Benchmark Construction
University of Science and Technology of China · 2Alibaba Group · 3National University of Singapore
Towards Order Fairness: Mitigating LLMs Order Sensitivity through Dual Group Advantage Optimization
May 12, 2026Xu Chu, Guanyu Wang, Zhijie Tan +4Language Model Robustness
School of Software and Microelectronics, Peking University, Beijing, China
ROMER: Expert Replacement and Router Calibration for Robust MoE LLMs on Analog Compute-in-Memory Systems
May 12, 2026Wenyong Zhou, Yuannuo Feng, Yizhe Chen +6Compute-in-Memory Mixture-of-Experts Language Models
The Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong · The School of Integrated Circuit Science and Engineering, Beihang University, Beijing, China
PRISM: A Geometric Risk Bound for Decomposing Drift into Scale, Shape, and Head
May 12, 2026Chieh-Yen Lin, Shao-Hua SunLLM Quantization LLM Evaluation
Appier AI Research · National Taiwan University
Targeted Tests for LLM Reasoning: An Audit-Constrained Protocol
May 12, 2026Hongmin LiPrompt Sensitivity Language Model Robustness
School of Life Science and Technology, Institute of Science Tokyo · Department of Computational Biology and Medical Sciences, Graduate School of Frontier Sciences, The University of Tokyo
CPEMH: An Agentic Framework for Prompt-Driven Behavior Evaluation and Assurance in Foundation-Model Systems for Mental Health Screening
May 11, 2026Giuliano Lorenzoni, Ivens Portugal, Paulo Alencar +1LLM Auditing Prompt Optimization
University of Waterloo · University of Waterloo Waterloo, ON, Canada
Afrispeech Semantics: Evaluating Audio Semantic Reasoning in Spoken Language Models Across Domains and Accents
May 11, 2026Chibuzor Okocha, Christan GrantAudio-Language Model Evaluation Spoken Language Understanding
University of Florida
QUIVER: A Formal Framework for Quantifying Perturbation Propagation and Bifurcation in Compound AI Systems
May 11, 2026Prashanti Nilayam, Sankalp NayakLanguage Model Robustness
Servicenow · CA, USA

May 21, 2026, cs.CLJ/K move · Enter open · S save

From Correlation to Cause: A Five-Stage Methodology for Feature Analysis in Transformer Language Models

Caleb Munigety

Independent Researcher

Transformer Interpretability Causal Reasoning in Language Models Causal Interpretability Causal Interventions in Language Models Language Model Robustness

Open paper AskPDF arXiv

We propose a five-stage methodology for causal feature analysis in transformer language models (probe design, feature extraction, causal validation, robustness testing, and deployment integration) and demonstrate it end-to-end on GPT-2 small performing the Indirect Object Identification (IOI) task. Activation patching recovers the canonical IOI circuit (layer-9 head 9 alone gives recovery +1.02). A sparse autoencoder recovers per-name selective features with effect sizes of 30 to 50 activation units. Causal validation finds these features specifically but only partially causal: ablating fifteen of them leaves the model accurate on 98% of prompts. Two NLA-inspired evaluations strengthen this picture: the fifteen selective features explain only 31% of activation variance versus the SAE's 99.7%, and selectivity ratio anticorrelates with causal force (r = -0.56). Robustness testing under three distribution shifts finds that the circuit transfers cleanly but feature ablation effects degrade substantially, exposing a gap between detection robustness and causal robustness. A cost-based deployment evaluation (assumed

50/FN,

0.42/FP, 2% error rate) finds an optimal monitor configuration yielding

8.96 per 1000 queries against a

1000 baseline, a 99.1% saving. Optimal composition strategy varies with cost ratio and base rate. The conjunction of stages produces findings no single stage would.

Momentum

Related topics

Latest papers 338