Hidden States

Follow this topic Ask about this topic

Momentum

21 papers in the last four weeks, up 200% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 125

All topics

Erased, Rerouted, or Rescaled? Post-Training and the Causal Quotient of a Language Model's Belief State
Oct 4, 2026Weihan Li, Tianshi Zheng, Junhao Wu +1Reinforcement Learning Post-Training Hidden States
The University of Tokyo · The Hong Kong University of Science and Technology · RWTH Aachen University +1
Looping Beyond Twice: A Scalable Recipe for Looped Mixture-of-Experts
Oct 1, 2026Di He, Pengxiang Li, Da Chang +3Mixture-Of-Experts Large Language Models Mixture-Of-Experts
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences · Peng Cheng Laboratory · University of Chinese Academy of Sciences +5
Reinforcement Learning with Complex (valued) Memories
Sep 29, 2026Sathya Kamesh Bhethanabhotla, Efstratios Gavves, André BiedenkappRecurrent Neural Networks Offline Reinforcement Learning
University of Freiburg
Predictive Geometry of Hidden Trajectories in Transformers
Sep 29, 2026Timur Mudarisov, Mikhail Burtsev, Tatiana Petrova +1Transformer Encoder Transformer Architectures
University of Luxembourg, Luxembourg
Scaling Video Generation for Reasoning: At What Cost?
Sep 29, 2026Weihang Guo, Xiaoyu Wu, Yifei Wang +2Generative Video Models Long Video Generation
Rice University · Carnegie Mellon University
Language Models Act on Hidden Valence
Sep 28, 2026Cameron Berg, Caspar KaiserHidden States Language Model Computes
Reciprocal Research · University of Warwick
TULIP: Targeted LLM Unlearning at Layers Identified Per-Input
Sep 28, 2026Yejin Kim, William F. Shen, Seokwon Jung +2Large Language Model Unlearning Hidden States
Korea Advanced Institute of Science & Technology (KAIST) · University of Cambridge · Sookmyung Women’s University
Admissible Diffusion for Multimodal Interventional Trajectories
Sep 28, 2026Xing Han, Shravan Chaudhari, Jiarui Shao +2Patient Trajectories Latent Dynamics
Johns Hopkins University · Massachusetts Institute of Technology · Bayesian Health
Dynamical Parameters: An Interpretability Framework for Time-Series Foundation Models
Sep 28, 2026Kang Yang, Gaofeng Dong, Liying Han +1Time Series Foundation Models Hidden States
Department of Electrical and Computer Engineering University of California, Los Angeles
Over-Personalization Is a Decision Failure: Generation-Induced Apply Bias in LLMs
Sep 28, 2026Haeun Jang, Yonghyun Jun, Hwanhee LeeLarge Language Model Personalization Large Language Model Bias
Department of Artificial Intelligence, Chung-Ang University
Coherence-Aware Distributional Evaluation of Open-Ended Text Generation
Sep 28, 2026Jinnuo Liu, Junhao Zhu, Weifeng Jiang +2Diversity Hidden States
New York University · Center for Data Science, NYU Shanghai · Georgia Institute of Technology
Finite Probes Suffice: Identifiability and Universality for Weight-Space Learning
Sep 27, 2026Soutrik Sarangi, Yonatan Sverdlov, Adir Dayan +2Neural Network Hidden States
Microsoft · Faculty of Mathematics, Technion – Israel Institute of Technology · Faculty of Electrical and Computer Engineering, Technion – Israel Institute of Technology +1
When Does the Concept of "Dog" Emerge in an Audio LLM?
Sep 27, 2026Zhe Wang, Shiqi Liu, Ruiyun Zhong +2Large Audio Language Models Audio Understanding
Huazhong University of Science and Technology, Wuhan, China
Controlling Backchannels in Streamable Full-duplex Models
Sep 24, 2026Maike Züfle, Peter Polák, Sefik Emre Eskimez +3Full-Duplex Speech Models Speech Language Models
Karlsruhe Institute of Technology, Germany · AppTek, Germany · Charles University, Czech Republic +2
LOCKR: A Hidden-State Trajectory-Guided Planner for Detecting and Repairing Stable-but-Wrong Lock-In in Diffusion Language Models
Sep 23, 2026Guoshenghui Zhao, Tan Yu, Weijie ZhaoDiffusion Language Models Lock-In
Rochester Institute of Technology · NVIDIA Corporation
Reasoning-Preserving Fine-Tuning of Post-RL LLMs with Null-Basis LoRA
Sep 22, 2026Wenzhi Fang, Nicholas Tzou, Lazar Valkov +1Model Fine-Tuning Post-Training
Samsung Research America, Mountain View, CA, USA · Purdue University, West Lafayette, IN, USA
Reading Emotions in the Token Space: Discriminative Adaptation of SpeechLLMs for Emotion Recognition
Sep 17, 2026Hasindri Watawana, Sergio Burdisso, Esaú Villatoro-Tello +4Emotion Recognition Speech Language Models
Idiap Research Institute, Switzerland · EPFL, Switzerland · Uniphore, USA & India +1
High-Performance Tensor Formulation of the Viterbi Algorithm for Hidden Semi-Markov Models
Sep 15, 2026Lorenzo Piarulli, Elia Belli, Daniele De SensiHidden Markov Models Hidden States
Department of Computer Science Sapienza University of Rome
Register Tokens for Bounded-State Reasoning in Diffusion Language Models
Sep 14, 2026Albert Ge, Chandan Singh, Yufan Zhuang +3Diffusion Language Models Masked Diffusion Language Models
University of Wisconsin–Madison · Microsoft Research · UC San Diego
Reference-Based Bias Detection in LLMs via Relative Representations of Hidden States
Sep 9, 2026Marek Jeliński, Jan Dubiński, Maciej Chrabaszcz +1Large Language Model Bias Model Auditing
NASK - National Research Institute, Poland · Warsaw University of Technology, Poland · Gda´nsk University of Technology, Poland
Contrastive Projection: Reading Transformer Internals by Differencing Logit Lenses
Sep 9, 2026Olli TuomiTransformer Architectures Hidden States
Evident Solutions Oy
Investigating Linear Probe Robustness to Linguistic Register, Medical Specialty, and Corpus Shifts in Medical QA
Sep 1, 2026Nishant Mishra, Ameen Abu-Hanna, Iacer CalixtoMultilingual Healthcare Q&A Text Corpora
Department of Medical Informatics, Amsterdam UMC, University of Amsterdam · Amsterdam Public Health, Methodology, Amsterdam, The Netherlands
StateSwap: Probing Support-Elimination Hidden States in Multiple-Choice Questions
Sep 1, 2026Chao Gao, Haijiang Liu, Qiyuan Li +3Multiple-Choice Questions Hidden States
Department of Computer Science, Vrije Universiteit Amsterdam, Amsterdam, The Netherlands · School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan, China · Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System, Wuhan University of Science and Technology, Wuhan, China
A Study of Hidden-State Optimization Order in Predictive Coding Networks
Sep 1, 2026Xueyuan Li, Danilo Vasconcellos VargasPredictive Coding Singular Learning Theory
Verbalizing Multi-Token Concepts in LLMs
Aug 31, 2026Xijie Gong, Zimeng Huang, Tonghan WangCompact Latent Sensor Tokens Hidden States
College of AI, Tsinghua University, Beijing, China
Can Video World Models Track Unobserved World States?
Aug 31, 2026Joonghyuk Shin, Yicong Hong, Jaesik Park +1Video World Models Hidden States
Seoul National University · Roblox
TRAPSBench: Vision-Language Models Encode but Fail to Express Epistemic Restraint
Aug 13, 2026Fnu Pramono, John Cai, Sourabh KulkarniVisual Evidence Abstention
Meta Superintelligence Labs · Reflection AI
Interpreting Language Model Hidden States at Scale
Aug 10, 2026Jordan Pettyjohn, Mansi Sakarvadia, Nathaniel Hudson +3Model Activations Hidden States
University of Chicago · 2Illinois Institute of Technology · 3Argonne National Laboratory +1
Archer: Adaptive Reuse of Cached Hidden States for Efficient Rollback in Diffusion Language Models
Aug 8, 2026Xuning He, Zinan Sheng, Yongding Tao +4Diffusion Language Models Diffusion Models
School of Computer Science, Shanghai Jiao Tong University · College of Artificial Intelligence, Nankai University · School of Computer Science, Peking University
Prompt Embedding Probes (PEP): Hallucination Detection in LLMs from Hidden States
Aug 8, 2026Zakhar Mrykhin, Valentin MalykhLarge Language Model Hallucination Citation Hallucination Detection
HSE University
Vision-Language Model Confidence Is Not a Property of the Answer
Aug 6, 2026Reza Khanmohammadi, Erfan Miahi, Ivan Brugere +4Confidence Calibration Vision-Language Foundation Models
Michigan State University · JPMorgan AI Research · Henry Ford Health
TruthLens: Object Hallucination Detection via Self-Evaluating Truthfulness Scores in LVLMs
Aug 6, 2026Yanqi Wu, Runhe Lai, Xinhua Lu +5Citation Hallucination Detection Object Hallucination
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China · Peng Cheng Laboratory, Shenzhen, China · Key Laboratory of Machine Intelligence and Advanced Computing, MOE, Guangzhou, China +2
Getting the Parameters Right: A Difficulty-Graded Benchmark and Probe-Guided Training for LLM Tool Calls
Aug 4, 2026Guoyao Yu, Xiaoqing Sun, Ziqi Huang +13Large Language Model Tool Use Large Language Model Agents
TransMem: Transforming Hidden States into Memory for Large Language Models
Jul 31, 2026Haodong Lei, Junming Liu, Yirong Chen +4Large Language Model Memory Large Language Model Agents
Southeast University, Nanjing, Jiangsu, China · Shanghai Artificial Intelligence Laboratory, Shanghai, China
Metaphor Tracer: A Theory-Informed Analysis of Hidden States
Jul 30, 2026Marc Heimann, Roxana Assadi Moghaddam, Olga Brovkina +2Hidden States Metaphors
Hermeneutic AI
Hidden APIs in Language Models: Discovering Reusable Causal Interfaces from Forked Futures
Jul 30, 2026SiYuan Ma, Yiqin Luo, Zhangji +8Hidden States Application Programming Interfaces
1Nanyang Technological University · 2Southern University of Science and Technology · 3Tianjin University +7
Same Evidence, Different Target: Decoding How Diagnostic Evidence Bears on Causal Questions from Language-Model States
Jul 29, 2026Weiyi Kong, Zhuoran LiDiagnosis Evidence Conflict
University of Toronto, Toronto, Canada · The University of Hong Kong, Hong Kong
Phase Structure in Rotary Attention: A Spectral Framework for Semantic Continuity and Execution-Boundary Governance
Jul 28, 2026Abraham ChachamovitsRotary Position Embedding Transformer Attention
ENTRUST AI
Emergent Latent-State Computation under Stochastic Volatility
Jul 28, 2026Xiaoyu Huang, Lulu WangLatent States Intermediate Latent States
Department of Mathematics Temple University Philadelphia, PA, USA · Department of Data Analytics Dickinson College Carlisle, PA, USA
Interpretable GOHR Agents via Sparse Autoencoders
Jul 27, 2026Shiwei Tan, Yusong Zhao, Weiyi Qin +6Autonomous Agents Hidden States
1Rutgers University · 3Paul B Kantor Consultant · University of Illinois Urbana-Champaign
D-Score: A Spectral Hidden-State Signal for Hallucination Detection in Large Language Models
Jul 27, 2026Bianca Raimondi, Davide Evangelista, Maurizio Gabbrielli +1Hallucination Detection Hallucination Rate
Department of Computer Science and Engineering, University of Bologna, Bologna, Italy
Try Once, Then Optimal: De-Redundified Procedure Memory for Cross-Episode Exploration Amortization
Jul 26, 2026Haizhou Ge, Haochen Ouyang, Zhixing Chen +6Episodic Memory Exploration
1Tsinghua University. · 3Northeast Agricultural University. · 2DISCOVER Robotics.
J-CoT: Chain-of-Thought in J-Space
Jul 24, 2026Junde Wu, Jiayuan Zhu, Fengling Liu +2Chain-of-Thought Reasoning Hidden States
University of Oxford · Stanford University
Reading and Steering Representations of Materials-Science Mechanisms in an Open-Weight Language Model
Jul 22, 2026Markus J. BuehlerMaterials Science Hidden States
Department of Civil and Environmental Engineering · Department of Mechanical Engineering · Schwarzman College of Computing +1
Planning as Emergent Behavior in Reinforcement Learning with Relational Hidden States
Jul 20, 2026Armin SommerModel-Free Reinforcement Learning Hidden States
ETH Zurich, Switzerland
The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Shared Category Geometry in Small Language Models
Jul 18, 2026Francesco Karim VicidominiTruth Hidden States
Independent researcher
ASK-NN: An Asymmetric Nearest-Neighbor Test that detects Distribution Drifts in Natural Language
Jul 17, 2026Sergey Zakharov, Rodion Oblovatny, Alexey ZaytsevLarge Language Model Hallucination Machine-Generated Text Detection
Saint-Peterburg university, Saint-Peterburg, Russia · Risk AI Research Lab, Moscow, Russia · Applied AI Institute, Moscow, Russia
Depth-Dependent Hidden-State Collapse in Dynamical System Autoencoders for LiDAR Point-Cloud Classification
Jul 16, 2026Patricia Medina, Hy P. G. LamLight Detection And Ranging Point Clouds Autoencoder Architectures
Department of Mathematics New York City College of Technology, CUNY Brooklyn, NY, USA · The Graduate Center, CUNY New York, NY, USA · Department of Mathematical Sciences Worcester Polytechnic Institute Worcester, MA, USA
The Refusal Residue: When Probes Catch Alignment Faking and When They Don't
Jul 15, 2026Aman MehtaFake News Refusals
Snowflake AI Research
SPARK: Susceptibility-Guided Profiling and Steering of Latent Reasoning States in Large Language Models
Jul 11, 2026Dongxu Zhang, Yiding Sun, Zihao Guo +5LLM Reasoning Strategies Hidden States
Xi’an Jiaotong University · Peking University · Tencent
TypeProbe: Recovering Type Representations from Hidden States of Pre-trained Code Models
Jul 9, 2026Giuliano Gorgone, Fausto CarcassiInterpretability Hidden States
ILLC, University of Amsterdam, Amsterdam, The Netherlands
TF-Engram: A Train-Free Engram with SSD-Backed Memory for Large Language Models
Jul 8, 2026Yutang Ma, Kecheng Huang, Xikun Jiang +1Large Language Model Memory LLM Inference Optimization
The Chinese University of Hong Kong · Beijing Institute of Technology, Zhuhai
Tensorized algorithms and scalable filtering methods for hidden Markov and factorial hidden Markov models
Jul 8, 2026Roxana Barrios, Ioannis SgouralisHidden Markov Models Tensor Decomposition
Department of Mathematics, University of Tennessee, Knoxville, TN
Empirical Minimal-Realisation Compression of Deep Neural Networks via Controllability-Observability Tests
Jul 5, 2026Anis Hamadouche, Amir HussainNeural Network Task-Aware Compression
The School of Engineering & Physical Sciences Heriot-Watt University Edinburgh EH14 4AS, UK · SDAIA-KFUPM Joint Research Centre for Artificial Intelligence King Fahd University of Petroleum and Minerals Dhahran, Saudi Arabia
Revealing Hidden Model Behaviors with Task-Specific Self-Reports
Jul 3, 2026Taras Kutsyk, Bartosz ZielińskiIntrospection Adapters Agent Behavior Modeling
1Jagiellonian University, Faculty of Mathematics and Computer Science · 2Jagiellonian University, Doctoral School of Exact and Natural Sciences
When Geometry Aligns: Dihedral Hidden-State Transformations in UNet, ViT, and DiT Architectures
Jul 3, 2026Mojtaba Faramarzi, Alex Lamb, Irina RishDiffusion Transformers U-Net
University of Montreal, Canada · Mila - Quebec AI Institute · Tsinghua University +1
Recovering Input Text from Hidden States: Study of Gradient-Based Inversion of Decoder-Only Language Models
Jul 1, 2026Mikołaj Słowikowski, Maciej Witold MajewskiHidden States Diffusion Inversion
AGH University of Krakow, Faculty of Physics and Applied Computer Science, al. Mickiewicza 30, 30-059 Krakow, Poland
DiscoLoop: Looping Discrete Embeddings and Continuous Hidden States for Multi-hop Reasoning
Jul 1, 2026Hengyu Fu, Tianyu Guo, Zixuan Wang +5Transformer Architectures Multi-Hop Reasoning
University of California, Berkeley
When Latent Agents Lie: KV-Cache Integrity in Multi-Agent LLM Collaboration
Jun 27, 2026Luís Brito, Carlos BaqueroLarge Language Model Agents Hidden States
Escola Superior de Tecnologia e Gestão (ESTG), Politécnico de Viana do Castelo (IPVC), Portugal · Faculdade de Engenharia (FEUP), Universidade do Porto (UP), Portugal
Event-Conditioned Diagnostics of Kinematic, Contact, and Object-Permanence Fields in Passive Object-State World Models
Jun 26, 2026Yang Liu, Yuming ChenIntermediate Latent States World Models
College of Intelligent Robitcs and Advanced Manufacturing, Fudan University · Shanghai, China

Oct 4, 2026, cs.LGJ/K move · Enter open · S save

Erased, Rerouted, or Rescaled? Post-Training and the Causal Quotient of a Language Model's Belief State

Weihan Li, Tianshi Zheng, Junhao Wu, Xinlei Chen

The University of Tokyo · The Hong Kong University of Science and Technology · RWTH Aachen University+1

Reinforcement Learning Post-Training Hidden States

Open paper AskPDF arXiv

What happens to information a pretrained model already encodes when post-training no longer rewards using it? The common language of representation compression conflates three fates: information may be erased, rerouted away from the decision while still represented, or rescaled to occupy less variance while still represented and used. We make these fates identifiable in models whose pretraining recovers Bayesian belief states. A reward that reads only a coarse function of the hidden state defines an exact reward-null kernel. The kernel lets us separately measure whether the information remains recoverable, whether decisions causally depend on it, and how much activation variance it occupies. Theory says what is protected: KL-anchored reinforcement learning preserves the reference policy's log-odds among equally rewarded outputs, supervised and unanchored objectives carry no such constraint, and spectral compression implies neither erasure nor loss of use. In controlled worlds, post-training mostly reroutes or rescales reward-null information and leaves it decodable. Without an anchor decisions can stop using it although the representation survives, and with one they keep using it. Erasure appears only under prolonged weight decay, for distinctions that neither reward nor next-token prediction can see. Open language models show the same dissociation: in-context belief geometry stays decodable under late-layer spectral compression, and within-class behavior depends on the anchor. Post-training thus selects a causal quotient of the pretrained belief state: the reward defines decision-equivalence, the anchor and the state update protect part of what it ignores, and optimization decides whether the rest is erased, rerouted, or rescaled.

Momentum

Related topics

Latest papers 125