Causal Interventions in Language Models

Follow this topic Ask about this topic

Momentum

55 papers in the last four weeks, up 162% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 317

All topics

Right Reset: Chunking by Prefix Removal
Aug 5, 2026Mike VegetoDocument Chunking Causal Interventions in Language Models
Independent Researcher
Intertemporal Preference Steering in Qwen3 via Contrastive Activation Addition
Aug 4, 2026Michal Mráz, Justin ShenkLanguage Model Steering Sequential Decision Making
1Independent
Sensitivity, Causality, and Repair Dissociate: A Layer-Wise Analysis of Perturbation Robustness and Its Scaling
Aug 4, 2026Nathan Labiosa, David Buff, Ena Nayak +1LLM Interpretability Adversarial Robustness of Language Models
University of Southern California
A Theory of Conditional Collapse under Low-Rank Weight-Space Ablations: I. The Single-Block Theory and Synthetic Validation
Aug 4, 2026Abdallah KhemaisTransformer Interpretability Weight-Space Analysis
ISITCOM, University of Sousse
When Refusal Looks Safe: The Refusal-Cue Shortcut in Safety Guard Models
Aug 4, 2026Yu Feng, Chunting Zang, Chen Shen +4Language Model Safety Evaluation LLM Guardrails
The University of Sydney · Alibaba Group · Jilin University +1
Activation-Guided Neuron Intervention to Induce Alzheimer's-Related Computational Language Phenotypes in a Large Language Model
Aug 4, 2026Rui He, Ercong Nie, Hong Jiang +3Alzheimer's Disease LLM Interpretability
Grammar and Cognition Lab, Department of Translation & Language Sciences, Universitat Pompeu Fabra, Barcelona, Spain. · School of Foreign Languages, Shanghai Jiao Tong University, Shanghai, China. · Zhuhai People’s Hospital (The Affiliated Hospital of Beijing Institute of Technology, Zhuhai Clinical Medical College of Jinan University), Zhuhai, China. +4
Inverted Detection and Control in Steering Vectors
Aug 3, 2026Max Torop, Aria Masoomi, Jennifer DyLanguage Model Steering Causal Interventions in Language Models
Northeastern University
Understanding Sparse Attention Selectivity in Long-Context Foundation Models via Counterfactual Evaluation
Aug 3, 2026Xingyu Ren, Youran Sun, Chugang Yi +1LLM Auditing Sparse Attention
The Chinese University of Hong Kong, Hong Kong, China · University of Maryland, College Park, MD, USA
Bridging the English-Arabic Medical Knowledge Gap: Targeted Low-Rank Adaptation via Causal Layer Selection
Jul 31, 2026Chaimae Abouzahir, Musa Khan, Hala Ali-Hassan +7LLM Fine-Tuning Low-Rank Adaptation
New York University Abu Dhabi · Cleveland Clinic Abu Dhabi
Evidence-Type Competition: When Can Interventional Data Teach Language Models Causal Direction?
Jul 31, 2026Xining XunCausal Discovery Causal Reasoning in Language Models
Tsingjiao Information Science (Beijing) Co., Ltd.
Token-Level Diagnosis of Sycophancy in LLMs with Attribution-Guided Steering
Jul 31, 2026Hieu Nguyen, Mahammed Kamruzzaman, Anshuman Chhabra +1LLM Sycophancy LLM Interpretability
Bellini College of AI, Cybersecurity and Computing University of South Florida
Stage-Replay Divergence Follows the KV Cache: Fixed-Prefix Precision Controls and Bidirectional Cache Transplantation
Jul 30, 2026Alexander Boesgaard LorupLLM Inference KV Caching
Openhagen
Fairness Pruning: Locating Demographic Bias in GLU-MLP Layers via Differential Activations
Jul 30, 2026Pere Martra, Eugenio Martínez Cámara, Alfonso Ureña LópezLLM Pruning LLM Interpretability
Universidad Internacional Men´endez Pelayo Barcelona, Spain · Departamento de Inform´atica Universidad de Ja´en Ja´en, Spain
Hidden APIs in Language Models: Discovering Reusable Causal Interfaces from Forked Futures
Jul 30, 2026SiYuan Ma, Yiqin Luo, Zhangji +8LLM Interpretability Causal Abstraction
1Nanyang Technological University · 2Southern University of Science and Technology · 3Tianjin University +7
Do Latent Channels Actually Communicate? A Causal Audit of Latent Multi-Agent LLM
Jul 29, 2026Huixiang Zhang, Mahzabeen EmuLLM Auditing Causal Interventions in Language Models
Georgia Institute of Technology · North Ave · Atlanta, GA 30332 USA +3
Steering Instruction Hierarchies at Inference Time
Jul 28, 2026Siqi Zeng, Sewoong Lee, Han Zhao +1Language Model Steering LLM Safety Alignment
Siebel School of Computing and Data Science University of Illinois, Urbana-Champaign Urbana, IL 61801, USA
Sparse Autoencoders Encode Both Concepts and Functions: The Downstream Geometry of Feature Effects
Jul 27, 2026Phu Gia Hoang, Anwoy Chatterjee, Tanmoy Chakraborty +2Sparse Autoencoders Mechanistic Interpretability
UKP Lab, Technical University of Darmstadt · Department of Electrical Engineering Indian Institute of Technology Delhi, India
Wrong Design Intent Is Worse Than Never Conditioning: A Derangement-Control Diagnosis of Header Conditioning in CAD Program Completion
Jul 25, 2026Yang XiaoCode Generation LLM-Based Program Synthesis
Sichuan University
Learning What Matters: Supervising Global Context Pruning with Causal Evidence Sets
Jul 23, 2026James E. AllchinLLM Pruning Long-Context Language Model Inference
Independent Researcher
Emergent Misalignment Recruits a Pre-existing Persona Subspace
Jul 23, 2026Mohammed Suhail B NadafLLM Fine-Tuning Emergent Misalignment in Language Models
Independent
Where Animacy Lives in Large Language Models: Tracing the Circuits of the Animacy Concept
Jul 23, 2026Samuele Punzo, Giovanni Cinà, Sandro PezzelleLLM Interpretability Circuits in Language Models
Graduate School of Informatics, University of Amsterdam · Department of Medical Informatics, Amsterdam University Medical Center · ILLC, University of Amsterdam
The Weight of Silence: A Causal Case for Weights Over the Scratchpad in Latent Chess Reasoning
Jul 23, 2026Ishan S. KshirsagarRL for Language Model Reasoning Continuous Latent Reasoning
Independent Researcher
Are Single-Token Sparse Autoencoder Features Causally Necessary? Layer-Depth and SAE-Family Effects
Jul 22, 2026Seonglae Cho, Zekun Wu, Kleyton Da Costa +3LLM Interpretability Sparse Autoencoders
Holistic AI · University College London
Understanding the Impact of Linguistic Realization Choices on LLM Stance with Causal Tracing
Jul 22, 2026Langchen Huang, Sebastian Padó, Franziska WeeberPrompt Sensitivity LLM Interpretability
Institute for Natural Language Processing, University of Stuttgart
Inference-Time Steering for Cross-Lingual Factual Consistency in LLMs
Jul 21, 2026Alexander ManevMultilingual Language Model Evaluation LLM Alignment
Technical University of Munich
The Knowing-Saying Gap: When Probes See Errors that Confidence Misses
Jul 21, 2026Jyotin Goel, Ipshita Bandyopadhyay, Justin ShenkLanguage Model Error Detection Confidence Estimation in Language Models
CASE: Causal Alignment and Structural Enforcement for Improving Chain-of-Thought Faithfulness
Jul 21, 2026Ziming Wang, Yinghua Yao, Changwu Huang +2CoT Faithfulness Faithfulness of Language Model Explanations
Department of Computer Science and Engineering of Southern University of Science and Technology, Shenzhen, China · Center for Frontier AI Research, Agency for Science, Technology and Research (A*STAR), Singapore · School of AI and Liberal Arts, Beijing Normal-Hong Kong Baptist University, Zhuhai, China +1
Can We Break LLMs Out of Self-Loops? Fine-Grained Reasoning Control with Activation Steering
Jul 20, 2026Sheldon Yu, Tong Yu, Xunyi Jiang +6Overthinking in Language Models Causal Interventions in Language Models
UC San Diego · Adobe Research · University of New South Wales
Resist and Update: Counterfactual Report Coordinates for Incentive-Compatible LLMs
Jul 14, 2026Sen Yang, Yuen-Hei YeungLLM Alignment Causal Interventions in Language Models
Stern School of Business · New York University · Courant Institute of Mathematical Sciences
Toward Localizing and Repairing Bias in Transformer Attention Heads
Jul 14, 2026Sigma JahanAttention Head Analysis Social Bias in Language Models
Dalhousie University, Canada

Aug 5, 2026, cs.CLJ/K move · Enter open · S save

Right Reset: Chunking by Prefix Removal

Mike Vegeto

Independent Researcher

Document Chunking Causal Interventions in Language Models

Open paper AskPDF arXiv

Removing the left context from a causal language model reveals a useful kind of boundary: an edge where the model processes the same right-hand tokens with little change. We turn this observation into prefix-removal probing and introduce Right Reset (RR), which measures preservation of the right-hand hidden-state trajectory. A dynamic program converts RR edge scores into variable-length chunks. On flattened text formed by concatenating topically similar records after deleting their separators and layout, RR recovers 47.7% of the original records as clean units, versus 25.9% for a BGE embedding boundary, the strongest tested conventional baseline without task-specific model training. The gain persists after rendering and OCR. Passive scores from the same Qwen3-4B layer and direct prompting of a same-scale instruction model perform substantially worse on flattened records. Across six language models, RR-selected cuts also undergo consistently less local output disruption than unselected candidate edges. An observed-token likelihood-ratio readout is competitive in some architectures, indicating that the central contribution is the intervention: context dependence itself can provide a boundary signal when surface structure is weak.

Momentum

Related topics

Latest papers 317