Overview

Recurrent Transformers study models that repeatedly apply shared Transformer blocks, pass hidden states or memory between tokens, or combine recurrence with attention. This can add test-time computation and support long-context and state-tracking tasks while trading parameters or cache memory for iterative work. Common approaches include depth-looped Transformers, recurrent and associative memory, hybrid attention–state-space layers, and adaptive halting, with stability controls and distillation used to make recurrence practical. Results span language modeling and retrieval, algorithmic tests such as Sudoku, ARC, and parity, and vision and audio tasks; a central tension is that extra loops can improve reasoning or length generalization but can also destabilize states, lose useful information, or add latency and memory costs.

Latest papers

CardsList

204 papers27 papers in the last four weeks, down 48% on the four weeks before. 0.2% of all new papers.

  1. LoopPrune: Recurrent Pruning for Looped Diffusion Transformers

    Nov 17, 2026Weizhong Huang, Huaijia Kou, Yan Zhang +3Diffusion Model PruningDiffusion Transformer

  2. InfuseLoop: Cross-Trajectory Latent Injection for Scalable Parallel Looped Transformers

    Nov 17, 2026Zhaofeng Sun, Lanxiang Hu, Yu-Yang Qian +6Parallel LLM InferenceParallel Decoding

  3. Latent Fact Composition under Next-Token Training

    Nov 16, 2026Tianyi Zhang, Feiying ChenMulti-Hop QACompositional Generalization

  4. Latent Feedback Is a Compute and Data Amplifier

    Nov 16, 2026Benhao Huang, Chaitanya Dwivedi, Xuezhe Ma +2Language Model PretrainingRecurrent Transformers

  5. When Does Reusing Layers Help Multimodal Embeddings?

    Nov 15, 2026Zhizhang Hu, Yue ZhangMultimodal EmbeddingRecurrent Transformers

  6. LoopFill: Pipelined Multi-Token Speculative Decoding for Looped Transformers

    Nov 14, 2026Siqi Zhu, Suozhi Huang, Yi Zhao +1Speculative DecodingLLM Inference Acceleration

  7. Architectures and KV Replay for Depth-Recurrent Language Models

    Nov 14, 2026Lucia Licakova, Niklas Schmidinger, Lukas Hauzenberger +2Recurrent TransformersKV-Cache Management

  8. Rectified Looped Transformer

    Nov 12, 2026Musheng Li, Jiawei Zhang, Ziyuan Liu +1Recurrent TransformersLanguage Modeling

  9. One Block Is Enough: LoopMR for Accelerated MRI Reconstruction

    Nov 11, 2026Yimeng He, Yibin Xie, Yi Zheng +7Accelerated MRI ReconstructionUnrolled Neural Networks

  10. Looping Deeper Without Growing the KV Cache: Recurrent Depth for Causal Encoder-Decoders

    Nov 9, 2026Jiaqi Leng, Minshen Zhang, Zhaofeng Sun +3LLM Inference AccelerationKV-Cache Management

  11. Extending Reasoning in Typed Decision Models via Looping

    Nov 5, 2026David Berghaus, Arber Zela, Behzad Shomali +4Efficient Language Model ReasoningAdaptive Reasoning in Language Models

  12. Loopback Transformer: Towards a Unified View of Residual Looped Transformers

    Nov 4, 2026Xuan Ouyang, Zefan Cai, Chenlong Zhang +3TransformerRecurrent Transformers

  13. LooM: Loop-Invariant Memory for Causal Encoder-Decoder Language Models

    Nov 4, 2026Lele Yang, Muxi Diao, Zhonghao Yan +7Adaptive ComputationRecurrent Transformers

  14. Loop Before You Leap: What Recurrent Depth Can and Cannot Buy in Data-Limited Pretraining

    Nov 4, 2026Daichi Hotta, Yohei Kobashi, Takeshi Kojima +2Language Model PretrainingRecurrent Transformers

  15. ARLoop: Fully-Open Pre-trained Looped Transformers with Attention Residuals

    Nov 3, 2026Yuekun Yao, Yupei Du, Dawei Zhu +1Recurrent TransformersAttention Residual Connections

  16. Best-of-Depth: Learning Exploratory Dynamics in Looped Transformers

    Nov 3, 2026Shota Takashiro, Takeshi Kojima, Masanori Koyama +2Deep SupervisionRecurrent Transformers

  17. UpLoop: Training Looped Transformers via Upcycling at No Extra Pretraining Compute

    Oct 30, 2026Cheng Liang, Tengfei Zhang, Ya Zhang +2Efficient Language Model TrainingRecurrent Transformers

  18. How to Loop Your Transformer: Loop Scaling, Specialization, and Parallel Inference

    Oct 29, 2026Chanakya Ekbote, Artidoro Pagnoni, Devendra Singh Sachan +5Recurrent TransformersEfficient Transformer Inference

  19. Depth through Recurrence: Towards Ultra-Efficient On-Device ASR

    Oct 27, 2026Chen Feng, Tianyi Xu, Ramchalam Kinattinkara Ramakrishnan +5Automatic Speech RecognitionModel Compression

  20. LoopFold: Parallelizing Recurrent Depth in Looped Transformers

    Oct 26, 2026Feichen Song, Boyi Zeng, Yiqin Hao +7Parallel Training of Recurrent NetworksRecurrent Transformers

  21. StateLoop: Persistent State for Looped Transformers

    Oct 26, 2026Feichen Song, Boyi Zeng, He Li +7Persistent Memory for Language ModelsLanguage Modeling

  22. Recursive CLIP: Vision-Language Distillation with Looped Transformer Blocks

    Oct 25, 2026Szymon Mazurek, Alberto Presta, Michał ByraVision-Language ModelsRecurrent Transformers

  23. Accordion: Elastic-Depth Inference via Fixed-Prefix Layer Looping

    Oct 23, 2026Yudong Zhang, Wei Ding, Kaili Zheng +2Efficient Language Model InferenceLLM Inference Acceleration

  24. Asymmetric Looped Transformers: Fixed Global Representations for an Iterative Decoder

    Oct 20, 2026Cheng Liang, Tengfei Zhang, Wenjie DU +6Recurrent Language ModelsRecurrent Transformers

  25. HistoLoop: Iterative Contextual Inference with a Looped Transformer for Histology-Based Spatial Gene Expression Prediction

    Oct 20, 2026Ziang Liu, Jie Liu, Shuai Li +2Spatial Gene Expression PredictionGene Expression Prediction from Histopathology Images