Neural Algorithmic Reasoning

Ask about this topic

Overview

Neural algorithmic reasoning asks how neural networks can learn the steps of algorithms and apply them to inputs or problem sizes not seen in training. Common approaches train on algorithm traces, align graph neural networks with message-passing algorithms, or use recurrent and looped models that perform more computation at test time; some methods encode symbolic rules or explicit memory directly. Researchers evaluate these systems on tasks such as CLRS graph algorithms, Sudoku, mazes, arithmetic, and graph optimization, often testing generalization to longer sequences, larger graphs, or deeper reasoning. A central challenge is ensuring that models truly execute and retain useful intermediate computations rather than relying on shortcuts, since accurate answers or decodable representations alone do not guarantee robust reasoning at greater depth.

Latest papers

CardsList

45 papers14 papers in the last four weeks, up 367% on the four weeks before. 0.1% of all new papers.

  1. Transformers with CoT Discover Heavy-Ball Acceleration Through Training

    Oct 22, 2026Yuxuan Cai, Yuan CaoCoT ReasoningNeural Algorithmic Reasoning

    School of Computing & Data Science, The University of Hong Kong

  2. Learning to Compute Is Not Learning to Retain or Reuse in Looped Transformers

    Oct 14, 2026Peilong Zhou, Hongtao Zhang, Yulong Huang +3Recurrent TransformersNeural Algorithmic Reasoning

    SKLP, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences +1

  3. Tiny Recursive Message Passing

    Oct 13, 2026Tommaso Salvatori, Matteo Forasassi, Alexis Toumi +1Neural Algorithmic ReasoningMessage Passing Neural Networks

    Relational Intelligence, Inc · Vienna University of Technology

  4. Looped Reasoning Finishes Earlier Than You Think

    Oct 13, 2026Sumin Park, T. Konstantin RuschNeural Algorithmic ReasoningContinuous Latent Reasoning

    ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems · Tübingen AI Center +1

  5. One-Layer Transformers Provably Learn In-Context K-Nearest Neighbor Prediction with Chain-of-Thought

    Oct 12, 2026Lyumin Wu, Yuan CaoTransformerIn-Context Learning

    School of Computing & Data Science, The University of Hong Kong

  6. Knowledge transfer between simple discrete worlds

    Oct 10, 2026Silvère Gangloff, Jan HulaLogical ReasoningCompositional Generalization

    University of Ostrava

  7. Fitting Every Hint Is Not Enough: Premature Floods in Recurrent Algorithmic Reasoners

    Oct 10, 2026Amin Najafgholizadeh, Mohsen AfsharchiNeural Algorithmic ReasoningGraph Neural Networks

    University of Zanjan

  8. Scaling to Tens of Thousands of Test-Time Iterations with Loop-Native Attention Residuals

    Oct 8, 2026Pengxiang Li, Dilxat Muhtar, Di He +3Recurrent TransformersAttention Residual Connections

    The Hong Kong Polytechnic University · Independent Researcher · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences +6

  9. Executing Causal Structure Learning with Linear-Attention Transformers

    Oct 7, 2026Amartya Roy, Sayar KarmakarNeural Algorithmic ReasoningCausal Structure Learning

    School of Interdisciplinary Research Indian Institute of Technology Delhi · University of Florida

  10. The Dichotomy Between Pattern Recognition and Step-by-Step Reasoning

    Oct 6, 2026Amrut Nadgir, Vijay Balasubramanian, Pratik ChaudhariLLM ReasoningStructured Reasoning in Language Models

    University of Pennsylvania

  11. Algorithmic Scratchpads and Curriculum Staging for Arithmetic Reasoning in Tiny Transformers

    Oct 6, 2026Sourabh KasliwalNumerical Reasoning in Language ModelsNeural Algorithmic Reasoning

  12. Neural Algorithmic Reasoning for Graph Saddle Point Problems

    Oct 5, 2026Samantha Chen, Jesse He, Coleman Clougherty +2Primal-Dual OptimizationGraph Neural Networks

    Oberlin College · Department of Computer Science, Oberlin College · University of California, San Diego +2

  13. What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling

    Sep 30, 2026Yuliana Shakhvalieva, Dmitrii Kharchev, Viacheslav Bezrukov +4Recursive Reasoning ModelsNeural Algorithmic Reasoning

    RND NLP, DAIMLD, Russian Federation

  14. Not All Thinking is Created Equal: Latent Reasoning Discovers a Recurrent Search Algorithm for Depth Generalization

    Sep 28, 2026Huzi Cheng, Zhewei ZhangLLM Reasoning with GraphsLLM Interpretability

    University of Minnesota · Independent Researcher

  15. ROTE: Benchmarking Neural Memorization on Complexity-Controlled Symbolic Sequences

    Sep 25, 2026Xinye Chen, Stefan Güttel, Mohammad MozaffariNeural Network MemorizationRecurrent Neural Networks

    Sorbonne Université, CNRS, LIP6 Paris, France · University of Toronto Toronto, Canada

  16. From Shortcut Learning to Discrete Neural Insertion Sort

    Sep 25, 2026Konstantinos Mylonas, Thrasyvoulos SpyropoulosAlgorithmic GeneralizationNeural Algorithmic Reasoning

  17. DART: Distributional Adversarial Recurrent Training for Algorithm Learning

    Sep 5, 2026Hieu Tran Bao, Phung Thanh Dang, Pham Quang Nhat Minh +1Neural Algorithmic ReasoningAdversarial Training

    FPT IS AI R&D Center Hanoi, Vietnam · VNU – University of Engineering and Technology Hanoi, Vietnam

  18. Think Shallow, Solve Deep: Controlling Recurrent Dynamics for Reliable Test-Time Depth

    Aug 18, 2026Ivan Viakhirev, Kirill Borodin, Amirah Almutairi +3Test-Time ScalingRecurrent Neural Networks

    FRC RAS, Saint-Petersburg, Russia · ITMO, Saint-Petersburg, Russia · lab260, Moscow, Russia +4

  19. Behaviour Is an Incomplete Measure of Reasoning Development: Cross-surface pre-arrival accessibility and the limits of developmental inference in a recurrent-depth reasoner

    Aug 17, 2026Simon Lam-MuirLatent ReasoningNeural Algorithmic Reasoning

    Prime Calibre, Australia

  20. Graph Machine: Exploring Edge Mechanisms as an Inductive Bias

    Aug 7, 2026Lintai HouGraph Structure LearningNeural Algorithmic Reasoning

    Iter Labs

  21. Measuring in-context algorithmic reasoning in language models against an exact Bayes-optimal standard

    Aug 3, 2026Hector Zenil, Luan OzelimLLM EvaluationBayesian Inference

    Oxford Immune Algorithmics, Oxford University Innovation & London Institute for Healthcare Engineering, U.K · Department of Biomedical Computing, School of Biomedical Engineering and Imaging Sciences & King’s Institute for AI, King’s College London, U.K

  22. A Symbolic Neural CPU for Quantization-Simulated Writeback and Interpretable Program Execution

    Jul 10, 2026Jose Luis Lima de Jesus SilvaNeural Algorithmic ReasoningNeural Network Interpretability

    Federal University of Bahia, Department of Geophysics, Salvador, BA 40170-115, Brazil · Grupo de Estudos e Aplicação de Inteligência Artificial em Geofísica (GAIA), Federal University of Bahia, Salvador, BA 40170-115, Brazil

  23. A Verifiable Search Is Not a Learnable Chain-of-Thought

    Jun 20, 2026Harsh PatelInference-Time SearchMemorization in Language Models

    Independent Researcher

  24. Efficiently Representing Algorithms With Chain-of-Thought Transformers

    Jun 18, 2026Yanhong Li, Anej Svete, Ashish Sabharwal +1TransformerCoT Reasoning

    Allen Institute for AI · ETH Zürich

  25. Contrastive Neural Algorithmic Reasoning for Graph Coloring

    Jun 2, 2026Thien Le, Tianyu Zhao, Melanie WeberGraph Neural NetworksGraph Representation Learning

    Harvard University SEAS · Harvard University T.H. Chan School of Public Health

  26. Richer Representations for Neural Algorithmic Reasoning via Auxiliary Reconstruction

    May 30, 2026Jiafu Huang, Chao Peng, Chenyang Xu +7Neural Algorithmic ReasoningState Representation Learning

    Shanghai Key Laboratory of Trustworthy Computing, East China Normal University, China · Overseas Education College, Xiamen University, China · Dept. of Electrical and Electronic Engineering, The University of Melbourne, Australia

  27. Neuro-symbolic Syntactic Parsing: Shaping a Neural Network with the CYK Algorithm

    May 29, 2026Fabio Massimo Zanzotto, Federico Ranaldi, Giorgio SattaNeural Algorithmic ReasoningRecurrent Neural Networks

    Human-centric ART, University of Rome Tor Vergata, Italy · DEI, University of Padua, Italy

  28. Represented Is Not Computed: A Causal Test of Candidate Algorithmic Intermediates in a Transformer

    May 21, 2026Ishita Darade, Sushrut ThoratNumerical Reasoning in Language ModelsTransformer Interpretability

    MKSSS’s Cummins College of Engineering for Women, Pune 411052, India · Institute of Cognitive Science, Osnabrück University, Osnabrück 49069, Germany

  29. Neural architectures for resolving references in program code

    Apr 15, 2026Gergő Szalay, Gergely Zsolt Kovács, Sándor Teleki +2Sequence-to-Sequence ModelsNeural Algorithmic Reasoning

    Faculty of Informatics, ELTE Eötvös Loránd University, Budapest, H-1117

  30. C-voting: Confidence-Based Test-Time Voting without Explicit Energy Functions

    Apr 15, 2026Kenji Kubo, Shunsuke Kamiya, Masanori Koyama +3Test-Time ScalingNeural Algorithmic Reasoning

    Graduate School of Engineering, The University of Tokyo