Systematic Generalization

Systematic generalization, the ability of AI models to apply learned knowledge to novel situations by combining previously learned components, is a crucial area of research aiming to bridge the gap between current AI capabilities and human-like intelligence. Current efforts focus on developing architectures like neural module networks and incorporating inductive biases into models such as graph neural networks and transformers, often augmented with neuro-symbolic methods, to improve systematic reasoning and reduce issues like hallucinations. This research is significant because achieving robust systematic generalization is essential for building more reliable and adaptable AI systems across diverse applications, from autonomous driving to medical image analysis and natural language processing.

Papers

October 7, 2022

Achieving and Understanding Out-of-Distribution Generalization in Systematic Reasoning in Small-Scale Transformers
Andrew J. Nam, Mustafa Abdool, Trevor Maxfield, James L. McClelland
Transformer Based Distribution Generalization Step by Step Reasoning Transformer Based Large Language Model Systematic Generalization Easy to Hard Generalization Tiny Transformer

October 4, 2022

Neural-Symbolic Recursive Machine for Systematic Generalization
Qing Li, Yixin Zhu, Yitao Liang, Ying Nian Wu, Song-Chun Zhu, Siyuan Huang
Strong Generalization Systematic Generalization Human Like Generalization

October 2, 2022

Systematic Generalization and Emergent Structures in Transformers Trained on Structured Tasks
Yuxuan Li, James L. McClelland
Transformer Megatron Decepticons Latent Representation Attention Layer Transformer Network Layer Transformer Task Specification Causal Transformer Systematic Generalization Emergent Structure

August 24, 2022

On a Built-in Conflict between Deep Learning and Systematic Generalization
Yuanpeng Li
Deep Learning Systematic Generalization Conflict Classification Standard Deep Multiple Prediction

June 15, 2022

Rethinking Generalization in Few-Shot Classification
Markus Hiller, Rongkai Ma, Mehrtash Harandi, Tom Drummond
Classification Code Systematic Generalization Image Level Annotation Shot Classification Benchmark

May 19, 2022

LAGr: Label Aligned Graphs for Better Systematic Generalization in Semantic Parsing
Dora Jambor, Dzmitry Bahdanau
Semantic Parsing Fully Supervised Systematic Generalization Graph Alignment Semantic Parser Alignment Supervision

February 24, 2022

Using calibrator to improve robustness in Machine Reading Comprehension
Jing Jin, Houfeng Wang
Native Robustness Calibration Performance Machine Reading Comprehension Systematic Generalization Adversarial Datasets Unified Generalization

February 22, 2022

Improving Systematic Generalization Through Modularity and Augmentation
Laura Ruis, Brenden Lake
Neural Network Data Augmentation Soft Augmentation Efficient Learning Modularity Function Systematic Generalization Modular Neural Network

February 14, 2022

Provably Efficient Causal Model-Based Reinforcement Learning for Systematic Generalization
Mirco Mutti, Riccardo De Santi, Emanuele Rossi, Juan Felipe Calderon, Michael Bronstein, Marcello Restelli
Markov Decision Process Model Based Reinforcement Learning Sequential Decision Systematic Generalization Reward Free Novel Causal Model

January 27, 2022

Transformer Module Networks for Systematic Generalization in Visual Question Answering
Moyuru Yamada, Vanessa D'Amario, Kentaro Takemoto, Xavier Boix, Tomotake Sasaki
Visual Question Answering Systematic Generalization Neural Module Transformer Module

December 1, 2021

Systematic Generalization with Edge Transformers
Leon Bergen, Timothy J. O'Donnell, Dzmitry Bahdanau
Language Understanding Relational Reasoning Systematic Generalization Edge Transformer Compositional Generalization Benchmark

November 28, 2021

ORCHARD: A Benchmark For Measuring Systematic Generalization of Multi-Hierarchical Reasoning
Bill Tuck Weng Pung, Alvin Chan
New Benchmark LSTM Network Long Sequence Relational Reasoning Systematic Generalization Neural Sequence Model Hierarchical Reasoning

November 25, 2021

Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning
Chi Zhang, Sirui Xie, Baoxiong Jia, Ying Nian Wu, Song-Chun Zhu, Yixin Zhu
Spatio Temporal Cognitive Architecture Abstract Reasoning Systematic Generalization Connectionist Model Algebraic Learning

Systematic Generalization

Papers

Achieving and Understanding Out-of-Distribution Generalization in Systematic Reasoning in Small-Scale Transformers

Neural-Symbolic Recursive Machine for Systematic Generalization

Systematic Generalization and Emergent Structures in Transformers Trained on Structured Tasks

On a Built-in Conflict between Deep Learning and Systematic Generalization

Rethinking Generalization in Few-Shot Classification

LAGr: Label Aligned Graphs for Better Systematic Generalization in Semantic Parsing

Using calibrator to improve robustness in Machine Reading Comprehension

Improving Systematic Generalization Through Modularity and Augmentation

Provably Efficient Causal Model-Based Reinforcement Learning for Systematic Generalization

Transformer Module Networks for Systematic Generalization in Visual Question Answering

Systematic Generalization with Edge Transformers

ORCHARD: A Benchmark For Measuring Systematic Generalization of Multi-Hierarchical Reasoning

Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning