Fast Weights

Latest papers

9 papers1 paper in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Select-TTT: Query-Conditioned Selective Fast Weight Updates for In-Place Test-Time Training
Oct 27, 2026Yue Wu, Yang Zhang, Weinan Gan +4Test-Time Training Inference-Time Adaptation of Language Models
University of Science and Technology of China China · National University of Singapore Singapore · Huawei Noah’s Ark Lab China
Looped Transformers as Optimizers
Sep 29, 2026Yulong Huang, Chen Jiang, Zhanpeng Zhou +5Fast Weights Recurrent Transformers
HKUST (GZ) · StepFun · SJTU +2
TERN: A Delta-rule Memory with a Seasonal Reference and Online Adaptation for Epidemic Forecasting
Sep 16, 2026Shunya Nagashima, Yuta FunayamaEpidemic Forecasting Time Series Forecasting
LTS, Inc., Japan
Modular TTT: Rethinking Test-Time Training as Composable Modules
Aug 7, 2026Bohao Tang, Zhen Qin, Yuqi Pan +3Test-Time Training Fast Weights
Shanghai Jiao Tong University · Shanghai Innovation Institute · ByteDance Seed
Stable Self-Modulating Quantum Fast-Weight Programmers with Bounded Memory Gates
Jul 2, 2026Kuo-Chung Peng, Jiun-Cheng Jiang, Chun-Hua Lin +8Time Series Forecasting Variational Quantum Circuits
National Taiwan University, Taiwan · Stevens Institute of Technology, NJ, USA · Seoul National University, Korea +3
Tensor Cache: Eviction-conditioned Associative Memory for Transformers
May 21, 2026Kabir Swain, Sijie Han, Daniel Karl I. Weidele +2Memory-Augmented Neural Networks Self-Attention
Massachusetts Institute of Technology, Cambridge, MA, USA · University of Toronto, Toronto, Canada · IBM Research, Cambridge, MA, USA
FAAST: Forward-Only Associative Learning via Closed-Form Fast Weights for Test-Time Supervised Adaptation
May 6, 2026Guangsheng Bao, Hongbo Zhang, Han Cui +4Efficient Neural Network Inference Test-Time Adaptation
Zhejiang University · Westlake University · Shanghai Polytechnic University +1
Where to Bind Matters: Hebbian Fast Weights in Vision Transformers for Few-Shot Character Recognition
Apr 22, 2026Gavin Money, Sindhuja Penchala, Jiacheng Li +1Hebbian Learning Vision Transformer
Department of Computer Science, The University of Alabama, Tuscaloosa, AL, USA
Reinforced Fast Weights via Next-Sequence Prediction
Feb 18, 2026Hee Seung Hwang, Xindi Wu, Sanghyuk Chun +2Fast Weights Long-Context Language Modeling
Princeton University

Oct 27, 2026J/K move · Enter open · S save

Select-TTT: Query-Conditioned Selective Fast Weight Updates for In-Place Test-Time Training

Yue Wu, Yang Zhang, Weinan Gan, Chongming Gao+3

University of Science and Technology of China China · National University of Singapore Singapore · Huawei Noah’s Ark Lab China

Test-Time Training Inference-Time Adaptation of Language Models Fast Weights Long-Context QA

Open paper AskPDF arXiv

Test-time training (TTT) enables a deployed language model to internalize incoming context on-the-fly into a small set of fast-weight parameters, rather than merely treating it as transient context for attention. In-Place TTT recently made this idea practical for large language models by repurposing the down-projection of standard MLP blocks as fast weights. However, we find that its benefit largely disappears on realistic web long-context question answering. We attribute this to In-Place TTT's uniform write schedule: every chunk is written with equal strength, so irrelevant content dilutes query-relevant information in fast-weight memory. Building on this insight, we propose Select-TTT, a model-agnostic, query-conditioned rule for In-Place TTT that scales each chunk's write strength by its relevance to the user query. This relevance signal requires no supervision and no new parameters: it is read directly from the model's frozen output head, as the probability mass that a chunk's hidden states assign to the query tokens. A temperature-scaled softmax turns these scores into learning-rate multipliers, leaving the chunk-wise, context-parallel computation of In-Place TTT intact. Select-TTT consistently improves long-context question answering on two benchmarks and two backbones. These findings highlight the value of query-conditioned fast weight writes for practical test-time training on long-context question answering, which is also potentially promising for long-horizon agent tasks involving multiple user queries over evolving contexts.