Algorithmic Reasoning

Follow this topic Ask about this topic

Algorithmic reasoning asks whether AI systems can apply step-by-step procedures to problems such as graph algorithms, symbolic puzzles, mathematics, and code—not merely recognize familiar patterns. Approaches include graph neural networks aligned with known algorithms, recurrent and recursive models that iterate over internal states, neuro-symbolic architectures, and language models that use code execution, search, or external tools. Researchers evaluate these systems on tasks such as CLRS graph problems, Sudoku and maze puzzles, ARC-style visual reasoning, and programming benchmarks, including tests of longer inputs and unfamiliar variants. A central challenge is achieving reliable multi-step execution and generalization beyond training examples: extra recurrence, search, or tools can help, but may increase cost and do not always prevent errors from accumulating.

Latest papers

108 papers11 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

Inside the Loop: A Mechanistic Study of Weight-Tied Transformers on Depth-Bound Algorithmic Tasks
Oct 16, 2026Magnus Sesodia, Juil Sock, Philip Torr +1Recurrent Transformers Algorithmic Reasoning
University of Oxford · BBC R&D
What Accuracy Hides In Learned Transformer Programs
Oct 16, 2026Jonathan Lee, Narmeen Oozeer, Philip QuirkeTransformer Interpretability Algorithmic Reasoning
Martian
Resample the Answer: Where to Spend Test-Time Compute in Recursive Reasoning
Oct 12, 2026Maxime IllouzRecursive Reasoning Models Inference-Time Search
Concordia University · Mila – Québec AI Institute · Montréal, Canada
A 3D Characterization Framework for Intelligent Sequential Decision Making
Oct 8, 2026Sadig Gojayev, Carolina FortunaSequential Decision Making AI Agent Evaluation
Department of Communication Systems, Jožef Stefan Institute, SI-1000 Ljubljana, Slovenia
Efficient Reasoning with Flow Language Models
Oct 7, 2026Hanru Bai, Faissal Izermine, Oscar Davis +1Efficient Language Model Reasoning Language Modeling
Max Planck Institute for Intelligent Systems · ETH Zurich · ELLIS Institute Tübingen +3
Recurrent Looped Transformer
Oct 6, 2026Yifan Zhang, Jichen Feng, Shihan Qin +2Length Generalization Recurrent Transformers
Princeton University · University of Pennsylvania
Neural Algorithmic Reasoning for Graph Saddle Point Problems
Oct 5, 2026Samantha Chen, Jesse He, Coleman Clougherty +2Primal-Dual Optimization Graph Neural Networks
Oberlin College · Department of Computer Science, Oberlin College · University of California, San Diego +2
Hypothesis-guided discovery of cognitive algorithms via program refinement
Oct 1, 2026Huiwen Alex Yang, Mark K. Ho, Bill D. ThompsonComputational Cognitive Modeling Automated Algorithm Discovery
Department of Psychology University of California, Berkeley · Department of Psychology New York University
What Limits Recursive Reasoning Models: Optimization, Architecture and Test-Time Scaling
Sep 30, 2026Yuliana Shakhvalieva, Dmitrii Kharchev, Viacheslav Bezrukov +4Recursive Reasoning Models Neural Algorithmic Reasoning
RND NLP, DAIMLD, Russian Federation
Reasoning with Neural Cellular Automata
Sep 28, 2026Mayalen Etcheverry, Pietro Miotti, Aidan Sirbu +7Neural Cellular Automata Neural Network Generalization
Google Paradigms of Intelligence Team · School of Computer Science, McGill University · Mila - Quebec AI Institute +4
TCSAlgBench: Benchmarking Automated Proving for Research-Level Theoretical Computer Science
Sep 28, 2026Chutong Yang, Xiyuan Zhang, Yu Huang +7Mathematical Reasoning Benchmarks Automated Theorem Proving
Department of Computer Science, The University of Texas at Austin · Amazon · Department of Statistics and Data Science, The Wharton School, University of Pennsylvania
Beyond the Training Horizon: Mechanisms and Limits of Length Generalization in Looped Transformers
Sep 27, 2026Jia Liang, Xi Jin, Liangming PanLength Generalization Recurrent Transformers
Institute for Computational and Mathematical Engineering, Stanford University · MOE Key Laboratory of Computational Linguistics, School of Computer Science, Peking University
RLVR landscapes for iterated multiplications can be benign: Insights from spin-glass theory
Sep 23, 2026Noa Rubin, Zohar RingelReinforcement Learning Maximum Entropy RL
Racah Institute of Physics The Hebrew University of Jerusalem
Scaling of Capability and Efficiency at Inference Time in Large Reasoning Models
Sep 22, 2026Moritz Laber, Zohair Shafi, Germans Savcisens +6LLM Inference Efficiency Language Model Scaling Laws
Northeastern University, Boston, Massachusetts, USA · Complexity Science Hub Vienna, Vienna, Austria · Santa Fe Institute, Santa Fe, New Mexico, USA
When Recursive Models Finish Computing
Sep 22, 2026Hare Krishna, Shubham Singh, Stephen Ebert +1Sudoku Solving Recurrent Neural Networks
Weinberg Institute, Department of Physics, University of Texas at Austin · Department of Urban and Environmental Policy Planning, Tufts University · Zyphra Technologies, San Francisco, CA, USA +2
ISA-Bench: A Benchmark for Computational Reasoning Across Instruction Set Architectures
Sep 19, 2026Aditya Pola, Arkaprava Majumdar, Vineeth N. BalasubramanianCode Generation Benchmarks LLM-Based Program Synthesis
IIT Hyderabad · Microsoft Research, India
Beyond Embedding Transfer: Component Roles in Grokking Transfer and Stability
Sep 16, 2026Zeyu JiaTransfer Learning Algorithmic Reasoning
Tianjin Medical University
AlgoRAG: Retrieval-Augmented Generation for Theoretical Computer Science Education -- A Comprehensive Evaluation Framework for Algorithm Analysis and Complexity Theory
Sep 13, 2026Sushan AdhikariComputer Science Education Retrieval-Augmented Generation
Dept. of Computer Science and Engineering, Kathmandu University, Dhulikhel, Nepal
GTA: Graph Theory Agent and Benchmark for Algorithmic Graph Reasoning with LLMs
Sep 10, 2026Zixiang Xu, Yanbo Wang, Chenxi Wang +6LLM Reasoning with Graphs LLM Agents
University of Southern California · University of California, Los Angeles · Mohamed bin Zayed University of Artificial Intelligence +2
Fractal basins trap latent reasoning
Sep 4, 2026Jeffrey Lai, Anthony Bao, John Quinn +1LLM Reasoning Large Reasoning Models
The Oden Institute, The University of Texas at Austin, Austin, Texas 78712, USA · Department of Electrical Engineering, The University of Texas at Austin, Austin, Texas 78712, USA · Department of Physics, The University of Texas at Austin, Austin, Texas 78712, USA
Interpretability for Turing Machines
Sep 4, 2026Billy Snikkers, Rumi Salazar, Daniel Murfet +1Algorithmic Reasoning
Resolution · University of Melbourne
Diffusion as a Training Curriculum for Timestep-Free Iterative Reasoning
Sep 1, 2026Mariia Drozdova, Aidan Sirbu, Pietro Miotti +4Constraint Satisfaction Algorithmic Reasoning
Google, Paradigms of Intelligence Team · University of Geneva · School of Computer Science, McGill University +4
Beyond Uncertainty: Multi-Solver Disagreement Rewards for Self-Evolving Reasoning Curricula
Aug 30, 2026Vinoth Selvendran, Zhanming ZhangReward Modeling RL for Language Model Reasoning
Independent Researcher · Palo Alto, CA, USA · New York, NY, USA
InteractBench: Benchmarking LLMs on Competitive Programming under Unrevealed Information
Aug 30, 2026Jiaze Li, Aocheng Shen, Bing Liu +4Code Generation Benchmarks Competitive Programming
School of Cyber Science and Engineering, Huazhong University of Science and Technology, Wuhan, China · Key Laboratory of Cyberspace Security, Ministry of Education, Zhengzhou, China · Hubei Key Laboratory of Distributed System Security, Wuhan, China.
MARS: Multi-Specialist LLM Relay System for Competitive Programming
Aug 24, 2026Andrei Mikhailov, Mikhail Burtsev, Alsu SagirovaLLM-Based Program Synthesis Multi-Agent Coding
MIRAI · London Institute for Mathematical Sciences · AXXX
Think Shallow, Solve Deep: Controlling Recurrent Dynamics for Reliable Test-Time Depth
Aug 18, 2026Ivan Viakhirev, Kirill Borodin, Amirah Almutairi +3Test-Time Scaling Recurrent Neural Networks
FRC RAS, Saint-Petersburg, Russia · ITMO, Saint-Petersburg, Russia · lab260, Moscow, Russia +4
Thought-Level Beam Search for Reasoning
Aug 8, 2026Lijie Yang, Hongyin Luo, Jiawei Zhao +2Inference-Time Search Efficient Language Model Reasoning
Princeton University · MIT CSAIL · Meta AI
CoBa: Cost-Effective Test-Time Scaling via Compute-Balanced Routing
Aug 7, 2026Yan Zhou, Yue Ouyang, Kaiyang Zheng +1Cost-Aware Inference Test-Time Scaling
School of Mathematics and Statistics, Changsha University of Science and Technology, Changsha, China · School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China.
Graph Machine: Exploring Edge Mechanisms as an Inductive Bias
Aug 7, 2026Lintai HouGraph Structure Learning Neural Algorithmic Reasoning
Iter Labs
Measuring in-context algorithmic reasoning in language models against an exact Bayes-optimal standard
Aug 3, 2026Hector Zenil, Luan OzelimLLM Evaluation Bayesian Inference
Oxford Immune Algorithmics, Oxford University Innovation & London Institute for Healthcare Engineering, U.K · Department of Biomedical Computing, School of Biomedical Engineering and Imaging Sciences & King’s Institute for AI, King’s College London, U.K

Oct 16, 2026J/K move · Enter open · S save

Inside the Loop: A Mechanistic Study of Weight-Tied Transformers on Depth-Bound Algorithmic Tasks

Magnus Sesodia, Juil Sock, Philip Torr, Christian Schroeder de Witt

University of Oxford · BBC R&D

Recurrent Transformers Algorithmic Reasoning

Open paper AskPDF arXiv

Solving algorithmic tasks with many sequential steps typically requires scaling up transformer depth or chain-of-thought length, incurring unfavorable parameter and inference-compute costs, respectively. Looped transformers, which reuse one block of layers across many iterations, are poised to address this, since they scale effective depth without adding parameters or tokens. Yet, it remains unclear what each iteration is actually computing, or whether the loop merely simulates a deeper untied stack. Here, we study directed permutation traversal, in which a model follows

H

hops from a starting node and reports the node it reaches, so every intermediate hop has a known ground truth. We find that a minimum effective depth is necessary to solve the task, beyond which the degree of weight tying becomes the determining factor. For instance, a looped transformer with a four-layer block iterated eight times solves the task in every seed, while an identically-trained untied transformer of the same effective depth never does, succeeding only with a lower learning rate or supervision of its intermediate layers. Probing and intervening on the residual stream, we find that the looped model advances through the intermediate nodes gradually, forming a staircase that adapts to its iteration budget, whereas the untied model, at that lower rate, exposes them only in its final layers. Taken together, our results recast looped transformers as iterative program learners, which allocate their iterations to successive steps of the algorithm and solve tasks that identically-trained untied models of the same effective depth do not. Code is available at https://github.com/magnus-sesodia/insidetheloop.