Compositional Generalization

Follow this topic Ask about this topic

Compositional generalization studies whether models can reuse learned concepts, skills, and rules in combinations they did not encounter during training. Approaches include modular and object-centric architectures, factorized or symbolic representations, and training methods such as curricula, structured reasoning traces, and reinforcement learning. Researchers test these abilities on controlled language, vision, arithmetic, planning, and robotics tasks, often holding out combinations, deeper structures, or longer sequences. Results remain uneven: models often recombine familiar parts more reliably than they discover new ones, and strong test accuracy can reflect shortcuts rather than reusable structure.

Latest papers

113 papers12 papers in the last four weeks, up 9% on the four weeks before. 0.1% of all new papers.

Learning to Extrapolate to New Tasks: A Relational Approach to Task Extrapolation
May 28, 2026Adam Ousherovitch, Yixin WangOOD Generalization Compositional Generalization
Department of Statistics, University of Michigan, Ann Arbor.
On Compositional Learning Behaviours in Formal Mathematics
May 27, 2026Kevin Yandoka DenamganaïMathematical Reasoning Benchmarks Formal Verification
University of York
Compositional Generalization in Autoregressive Models via Logit Composition
May 27, 2026Aakash Kumar, Maria Sofia Bucarelli, Emanuele NataleLanguage Model Merging Compositional Generalization
COATI, CNRS, Inria, I3S, Université Coté d’Azur, France
Learning Compositional Latent Structure with Vector Networks
May 27, 2026Niclas Pokel, Benjamin F. GreweRepresentation Learning Recurrent Neural Networks
Institute of Neuroinformatics, UZH / ETH Zurich ETH AI Center Zurich, Switzerland
What Does the Caption Really Say? Counterfactual Phrase Intervention for Compositional Data Selection in Vision-Language Pretraining
May 21, 2026Hyejin Go, Semi Lee, Hyesong ChoiTraining Data Selection Vision-Language Pretraining
Soongsil University Seoul, Republic of Korea
Factored Diffusion Policies:Compositionally Generalized Robot Control with a Single Score Network
May 21, 2026Sayan Mitra, Ege Yuceel, Noah Giles +1Robotic Control Autonomous Drone Racing
University of Illinois Urbana-Champaign
Decomposed Vision-Language Alignment for Fine-Grained Open-Vocabulary Segmentation
May 15, 2026Chenhao Wang, Yingrui Ji, Yu Meng +1Image Segmentation Open-Vocabulary Semantic Segmentation
Aerospace Information Research Institute, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Zhejiang University
GiLT: Augmenting Transformer Language Models with Dependency Graphs
May 15, 2026Tianyu Huang, Yida Zhao, Chuyan Zhou +1Transformer Language Modeling
School of Information Science and Technology, ShanghaiTech University · Shanghai Engineering Research Center of Intelligent Vision and Imaging
XDomainBench: Diagnosing Reasoning Collapse in High-Dimensional Scientific Knowledge Composition
May 14, 2026Gong Zhiren, Tiantong Wu, Jiaming Zhang +9AI for Science LLM Evaluation
College of Computing and Data Science, Nanyang Technological University, Singapore · Interdisciplinary Graduate Programme, Nanyang Technological University, Singapore · Department of Mathematics and Mathematical Statistics, Umeå University, Sweden +2
KamonBench: A Grammar-Based Dataset for Evaluating Compositional Factor Recovery in Vision-Language Models
May 13, 2026Richard Sproat, Stefano PeluchettiVLM Evaluation Compositional Generalization
Unlocking Compositional Generalization in Continual Few-Shot Learning
May 12, 2026Phu-Quy Nguyen-Lam, Phu-Hoa Pham, Dao Sy Duy Minh +3Object-Centric Representation Learning Few-Shot Learning
Faculty of Information Technology, University of Science, Vietnam National University, Ho Chi Minh City, Vietnam. · Department of Computer Science, University of Warwick, Coventry, United Kingdom.
Benchmarking Compositional Generalisation for Machine Learning Interatomic Potentials
May 9, 2026Amir Masoud Nourollah, Irtaza Khalid, Stefano Leoni +1Machine Learning Interatomic Potentials OOD Generalization
Cardiff University, Cardiff, CF10 3AT, UK
When Symbol Names Should Not Matter: A Logistic Theory of Fresh-Symbol Classification
May 8, 2026Wenjie Guan, Jelena BradicTransformer Classification
Department of Statistics and Data Science, Cornell University
Test-Time Compositional Generalization in Diffusion Models via Concept Discovery
May 8, 2026Zekun Wang, Anant Gupta, Tianyi Zhu +1Test-Time Adaptation Compositional Image Generation
Georgia Institute of Technology · University of Virginia
Reinforcement Learning for Compositional Generalization with Outcome-Level Optimization
May 6, 2026Xiyan Fu, Wei LiuReinforcement Learning RL for Generative Models
Nanyang Technological University · Independent Researcher
Critical Windows of Complexity Control: When Transformers Decide to Reason or Memorize
May 6, 2026Sarwan AliDeep Learning Optimization Weight Decay
Columbia University, Irving Medical Center, NY, USA
Towards Systematic Generalization for Power Grid Optimization Problems
May 3, 2026Zeeshan Memon, Yijiang Li, Hongwei Jin +2Optimal Power Flow Constrained Optimization
Emory University Atlanta, GA, USA · Argonne National Laboratory Lemont, IL, USA
Reconstructing conformal field theoretical compositions with Transformers
May 1, 2026Haotian Cao, Garrett Merz, Kyle Cranmer +1Compositional Generalization
a Department of Physics, University of Wisconsin-Madison · b Data Science Institute, University of Wisconsin-Madison
When Do Diffusion Models learn to Generate Multiple Objects?
Apr 30, 2026Yujin Jeong, Arnas Uselis, Iro Laina +2T2I Generation Compositional Image Generation
TU Darmstadt & hessian.AI · University of Tübingen · Visual Geometry Group, University of Oxford +1
Investigating More Explainable and Partition-Free Compositionality Estimation for LLMs: A Rule-Generation Perspective
Apr 30, 2026Ziyao Xu, Cong Wang, Houfeng WangCompositional Generalization
National Key Laboratory for Multimedia Information Processing, School of Computer Science, Peking University · OPPO AI Center
AGEL-Comp: A Neuro-Symbolic Framework for Compositional Generalization in Interactive Agents
Apr 29, 2026Mahnoor Shahid, Hannes RotheNeuro-Symbolic AI Inductive Logic Programming
Universität Duisburg-Essen, Essen, Germany
Grounding vs. Compositionality: On the Non-Complementarity of Reasoning in Neuro-Symbolic Systems
Apr 29, 2026Mahnoor Shahid, Hannes RotheTensor Networks Logical Reasoning
Universität Duisburg Essen, Germany
Structural Generalization on SLOG without Hand-Written Rules
Apr 28, 2026Zichao WeiNeural Cellular Automata Semantic Parsing
Saarland University
To See the Unseen: on the Generalization Ability of Transformers in Symbolic Reasoning
Apr 23, 2026Nevena Lazić, Liam Fowl, András György +1Logical Reasoning Decoder-Only Language Models
Diagnosing CFG Interpretation in LLMs
Apr 22, 2026Hanqi Li, Lu Chen, Kai YuLLM Evaluation In-Context Learning
1X-LANCE Lab, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · 3Shanghai Innovation Institution, Shanghai, China · 4Jiangsu Key Lab of Language Computing, Suzhou, China +2
Gradient-Based Program Synthesis with Neurally Interpreted Languages
Apr 20, 2026Matthew V. Macfarlane, Clément Bonnet, Herke van Hoof +1Program Synthesis Differentiable Programming
AMLab, University of Amsterdam · Work completed while a visiting student at the University of Alberta. · Ndea +2
Generalization in LLM Problem Solving: The Case of the Shortest Path
Apr 16, 2026Yao Tong, Jiayuan Ye, Anastasia Borovykh +1Inference-Time Scaling Language Model Generalization
National University of Singapore · Capital Fund Management · Google Research
THEIA: Learning Complete Kleene Three-Valued Logic in a Pure-Neural Modular Architecture
Apr 13, 2026Augustus Haoyang LiNeural Network Interpretability Compositional Generalization
Irvine Valley College Irvine, CA, USA
Loop, Think, & Generalize: Implicit Reasoning in Recurrent-Depth Transformers
Apr 9, 2026Harsh Kohli, Srinivasan Parthasarathy, Huan Sun +1Position Extrapolation Implicit Reasoning in Language Models
The Ohio State University
Transformer See, Transformer Do: Copying as an Intermediate Step in Learning Analogical Reasoning
Apr 7, 2026Philipp Hellwig, Willem Zuidema, Claire E. Stevenson +1Analogical Reasoning for Language Models Compositional Generalization
Psychological Methods, University of Amsterdam · Institute for Logic, Language and Computation, University of Amsterdam

May 28, 2026J/K move · Enter open · S save

Learning to Extrapolate to New Tasks: A Relational Approach to Task Extrapolation

Adam Ousherovitch, Yixin Wang

Department of Statistics, University of Michigan, Ann Arbor.

OOD Generalization Compositional Generalization

Open paper AskPDF arXiv

Modern learning systems excel at interpolation but struggle to generalize to unseen tasks outside the training distribution's support. This failure occurs even in simple settings, such as handling task parameters beyond the training range, and persists despite advances in foundation models. To this end, we develop the Relational Task Extrapolator (RTE), an algorithm designed to enable systematic extrapolation to novel tasks. The key observation is that extrapolation is inherently relational: extrapolating to unseen tasks requires learning how tasks transform into one another. If a model learns the transformation between tasks A and B during training, it can apply that same transformation to relate known tasks to unseen ones at test time. RTE operationalizes this idea by decomposing each target task into a known anchor task and a transformation linking the anchor and target. It then learns a relational operator, mapping an anchor-transformation pair to predictions for the target task. We instantiate RTE across multiple task extrapolation regimes in function prediction, e.g. where target tasks use out-of-range parameters (parameter extrapolation), have greater compositional depth (length extrapolation), and/or recombine function primitives in unseen ways (compositional extrapolation). We further extend RTE to sequence prediction, integrating it into fine-tuning algorithms for foundation models. Across empirical studies, we find that RTE substantially outperforms existing approaches on extrapolation to novel, unseen tasks.