In-Context Learning

Also known as ICL

Overview

In-context learning (ICL) studies how a model can infer a task, rule, or preference from information in its prompt and apply it to new inputs, making adaptation possible without task-specific weight updates. Common approaches include few-shot demonstrations, retrieval and ordering strategies, synthetic-task pretraining for prior-fitted models, and test-time optimization of prompts or memory; evaluations range from synthetic regression and Markov tasks to language, vision, time-series, graph, and robotics benchmarks. Theory and mechanistic studies examine what algorithms attention and recurrent models implement, and how context length, training-task diversity, and architecture affect generalization. A central tension is that models may encode contextual information yet fail to use it reliably: performance can be sensitive to example quality, order, noise, and conflict with pretrained priors, while longer contexts increase inference costs.

Latest papers

CardsList

429 papers46 papers in the last four weeks, down 26% on the four weeks before. 0.3% of all new papers.

  1. Leaner Transformers Can Easily Learn to Cluster

    Oct 7, 2026Charlotte Park, Kenneth L. Clarkson, Lior Horesh +2TransformerClustering

  2. When Should an In-Context Learner Expand Its Hypothesis Space?

    Oct 7, 2026Weihan Li, Xinlei Chen, Junhao Wu +1In-Context LearningMetacognition in Language Models

  3. Adaptive Mean Estimation by In-Context Learning: A Gradient-Flow Analysis

    Oct 6, 2026Martin Eppert, Krishna Balasubramanian, Subhro Ghosh +2In-Context LearningPrior-Data Fitted Networks

  4. Learning Decision-Stump Thresholds in Context: Dynamics of Softmax Attention

    Oct 5, 2026Hong Ha Le, Jackie Lok, Atsushi Nitanda +1Language Model PretrainingSoftmax Attention

  5. Learning to Learn a Language

    Oct 5, 2026Lennart Carstens-Behrens, Holger FröhlichLanguage Model PretrainingMeta-Learning

  6. RepICL: Reusable In-Context Prediction Across Heterogeneous Representation Spaces

    Oct 5, 2026Yu-Hsiang Liu, Kuan-Yu Chen, Chih-Sheng Chen +3Meta-LearningFew-Shot Learning

  7. Retrieval-Based In-Context Learning: A Domain Adaptation Framework

    Oct 5, 2026Yilun Zhu, Naihao Deng, Yingcong Li +2Distribution ShiftDomain Adaptation

  8. Verb-ICL: Rethinking In-Context Learning for Structured Prediction

    Oct 3, 2026Fan Bai, Hengshuo Miao, Sanjit S Batra +3In-Context LearningStructured Prediction

  9. Generalization of Transformer-Based Neural Quantum States via In-Context Learning

    Oct 2, 2026Zhen Qin, Qing Qu, Alfred O. HeroNeural Quantum StatesNeural Network Generalization

  10. When History Fails to Become Experience: Action Calibration in Language Agents

    Oct 2, 2026Jingyu Liu, Zhiwen Wang, Yuxin Jing +2Memory-Augmented Language Model AgentsIn-Context Learning

  11. In-context Learning of Single-index Targets: Comparing Kernel and Feature Learners

    Oct 1, 2026Haotian Gu, Yizhou Xu, Lenka ZdeborováRepresentation LearningSingle-Index Models

  12. Skeleton-and-Strategy Prompting: Training-Free Negation Understanding for Vision-Language Models

    Oct 1, 2026Yuliang Cai, Mohammad Rostami, Jesse ThomasonVisual Question AnsweringIn-Context Learning

  13. Capturing In-Context Learning Dynamics with Task Operators

    Oct 1, 2026Guangzhi Xiong, Zhenghao He, Bohan Liu +3Transformer InterpretabilityIn-Context Learning

  14. Efficient Task Adaptation in Large Language Models: A Survey of Weight-Based, Prompt-Based, and Embedding-Based Adaptations

    Oct 1, 2026Jungwon Park, Changin Choi, Jimyeong Kim +2LLM Fine-TuningIn-Context Learning

  15. Message Passing Does More with Less for In-Context Learning on Graphs

    Sep 29, 2026Dooho Lee, Jinmo Lee, Minho Jeong +2In-Context LearningGraph Neural Networks

  16. Theory on Attention Dynamics for Out-of-Distribution In-Context Learning

    Sep 29, 2026Junze Deng, Daouda Sow, Sen Lin +1In-Context LearningOOD Generalization

  17. Support-Set Target Leakage in Relational Foundation Models during In-Context Learning: Model Dependence and Evaluation Reliability

    Sep 29, 2026Roshan Reddy Upendra, Alexandre Dorais, Joe Meyer +6Relational Foundation ModelsData Leakage

  18. In-Context Learning for Robots: Methods and Applications

    Sep 28, 2026Haojian Huang, Zexi Li, Junhao Guo +36ICL for RoboticsGeneralization in Robotic Manipulation

  19. ALICE: In-context, Zero-shot, Mutual Information Estimation

    Sep 28, 2026Giulio Franzese, Simone Rossi, Pietro MichiardiZero-Shot LearningIn-Context Learning

  20. Unifying Video Tasks via Spatiotemporal Analogy

    Sep 27, 2026Chia-Hsiang Kao, Belinda Zeng, Bharath Hariharan +1Analogical ReasoningIn-Context Learning

  21. In-Context Adaptation of Encoder-Decoder Models in Speech Recognition

    Sep 27, 2026Yen Meng, Sharon Goldwater, Hao TangContextual ASRDomain Adaptation for ASR

  22. Downstream-Aware Context Selection for Online In-Context Reinforcement Learning

    Sep 27, 2026Ruihan A. Li, Shangtong Zhang, Rohan ChandraIn-Context RLIn-Context Learning