Inductive Reasoning in Language Models

Follow this topic Ask about this topic

Latest papers

16 papers4 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Typological Feature Induction as a Benchmark for Inductive Metalinguistic Reasoning
Nov 3, 2026Chen Zhang, Seth Aycock, Yusuke MiyaoInductive Reasoning in Language Models Multilingual Language Model Evaluation
NII LLMC · University of Amsterdam · The University of Tokyo
Multimodal In-Context Learning with Explicit Rule Induction
Oct 30, 2026Yuquan Li, Xinyang Jiang, Yuhao Xue +1Multimodal ICL Multimodal Large Language Models
Tongji University · Microsoft Research Asia
An Algorithmic Account of Few-Shot Concept Induction in Language Models
Oct 26, 2026Michael A. Lepori, Laila Johnston, Aalok Sathe +3Inductive Reasoning in Language Models In-Context Learning
Dept. of Computer Science, Brown University, Providence, Rhode Island · Dept. of Cognitive and Psychological Sciences, Brown University, Providence, Rhode Island
Cognitively Decoupled Meta-Induction for Out-of-Distribution Generalization
Oct 19, 2026Yuqian Wang, Yuanzheng Wang, Chaojun Xiao +7Inductive Reasoning in Language Models OOD Generalization
Beijing Institute of Technology · University of Chinese Academy of Sciences · Department of Computer Science and Technology, Tsinghua University +4
Latent Taxonomies: How Property Induction Develops in Language Model Pre-training
Oct 11, 2026Jan Sobotka, Ekaterina Goliakova, Marcin Detyniecki +2Inductive Reasoning in Language Models Inductive Inference
EPFL · AXA · Sorbonne University, CNRS, LIP6 +1
The IOL-AI Challenge: An Open Challenge towards Advancing Linguistic Reasoning
Aug 18, 2026Eduardo Sánchez, Rita Berrada, Dan-Mircea Mirea +8Inductive Reasoning in Language Models LLM Reasoning
University College London & Meta · CentraleSupélec & McGill University · Cohere Labs +6
Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks
Jul 13, 2026Tiberiu Musat, Tiago Pimentel, Nicolas Zucchet +1Transformer Training Dynamics In-Context Learning
ETH Zurich · Stanford University
Think Through a Bottleneck: Hourglass Reasoning for Rigorous Induction
Jul 13, 2026Huan ZhuStructured Reasoning LLM Self-Refinement
Peking University
Words as Difference Makers: How Large Language Models Determine Causal Structure in Text
Jun 21, 2026Wolfgang PietschCausal Reasoning in Language Models LLM Interpretability
tum
FALSIFYBENCH: Evaluating Inductive Reasoning in LLMs with Rule Discovery Games
Jun 3, 2026Leonardo Bertolazzi, Katya Tentori, Raffaella BernardiLLM Evaluation AI Agent Benchmarks
University of Trento · Free University of Bozen-Bolzano
Using Probabilistic Programs to Train Inductive Reasoning in Large Language Models
May 26, 2026Liyi Zhang, Akshay K. Jagadish, Brenden M. Lake +1Language Model Calibration LLM Uncertainty Estimation
Department of Computer Science Princeton University · Princeton AI Lab Princeton University · Department of Psychology and Computer Science Princeton University
Heuristic Classification of Thoughts Prompting (HCoT): Integrating Expert System Heuristics for Structured Reasoning into Large Language Models
Apr 14, 2026Lei Lin, Jizhao Zhu, Yong Liu +3Structured Reasoning in Language Models Constraint-Guided Reasoning in Language Models
Computer Network Information Center, Chinese Academy of Sciences, Beijing, China · Shenyang Aerospace University · ZGC LAB, Beijing,China +1
Watson & Holmes: A Naturalistic Benchmark for Comparing Human and LLM Reasoning
Feb 23, 2026Thatchawin Leelawat, Lewis D GriffinInductive Reasoning in Language Models Reasoning Evaluation
Computer Science, University College London, UK
OdysseyArena: Benchmarking Large Language Models For Long-Horizon, Active and Inductive Interactions
Feb 5, 2026Fangzhi Xu, Hang Yan, Qiushi Sun +14AI Agent Benchmarks Long-Horizon Agent Evaluation
Xi’an Jiaotong University · The University of Hong Kong · Tsinghua University +5
PBEBench: A Multi-Step Programming by Examples Reasoning Benchmark inspired by Historical Linguistics
May 29, 2025Atharva Naik, Prakam, Yash Mathur +6Program Synthesis Inductive Inference
Carnegie Mellon University · Independent Researcher · Ohio State University
An Incomplete Loop: Deductive, Inductive, and Abductive Reasoning in Language Models
Apr 3, 2024Emmy Liu, Graham Neubig, Jacob AndreasDeductive Reasoning in Language Models LLM Prompting
Language Technologies Institute Carnegie Mellon University · CSAIL Massachusetts Institute of Technology

Nov 3, 2026J/K move · Enter open · S save

Typological Feature Induction as a Benchmark for Inductive Metalinguistic Reasoning

Chen Zhang, Seth Aycock, Yusuke Miyao

NII LLMC · University of Amsterdam · The University of Tokyo

Inductive Reasoning in Language Models Multilingual Language Model Evaluation Linguistic Typology

Open paper AskPDF arXiv

Existing metalinguistic reasoning benchmarks require both inference of rules and their application, making failures hard to diagnose. We introduce typological feature induction (TFI) which targets the former step. Given sentences in an unnamed language paired with translations in a known pivot language, models must predict a typological feature of the unnamed language. From Grambank and multi-parallel corpora, we automatically build evaluation questions over 195 languages. We find that LLMs identify 59--80% of these languages from only 16 sentence pairs; and misaligning the translations barely affects their accuracy, suggesting reliance on prior knowledge of the language. Enciphering text to prevent recognition increases the corpus' utility, but mainly for features where target and pivot languages agree. On languages which models cannot identify, neither open model outperforms a majority-value baseline. Since induction from raw text aligns with field linguists' use-case, this work indicates the models tested are not yet ready for this role. TFI offers a scalable path to track progress in this direction.