In-Context Learning

Also known as ICL

Ask about this topic

Overview

In-context learning (ICL) studies how a model can infer a task, rule, or preference from information in its prompt and apply it to new inputs, making adaptation possible without task-specific weight updates. Common approaches include few-shot demonstrations, retrieval and ordering strategies, synthetic-task pretraining for prior-fitted models, and test-time optimization of prompts or memory; evaluations range from synthetic regression and Markov tasks to language, vision, time-series, graph, and robotics benchmarks. Theory and mechanistic studies examine what algorithms attention and recurrent models implement, and how context length, training-task diversity, and architecture affect generalization. A central tension is that models may encode contextual information yet fail to use it reliably: performance can be sensitive to example quality, order, noise, and conflict with pretrained priors, while longer contexts increase inference costs.

Latest papers

CardsList

404 papers64 papers in the last four weeks, up 52% on the four weeks before. 0.4% of all new papers.

  1. Lifelong In-Context Learning with Transformers Requires Parametric Forms of Attention

    Jun 24, 2026Luke McDermott, Robert W. Heath, Rahul ParhiLifelong Learning AgentsLong-Context Modeling

    UC San Diego

  2. Pigeonholing: how bad prompts hurt models, causing collapse and mistakes

    Jun 23, 2026Hyunji Nam, Keertana Chidambaram, Dorottya Demszky +1LLM ReliabilityIn-Context Learning

    Stanford University · University of Washington

  3. Scaling Performance and Low-Resource Annotation with Many-Shot In-Context Learning for Named Entity Recognition

    Jun 20, 2026Qi Zhang, Fangping Lan, Cornelia Caragea +2Named Entity RecognitionLLM-Assisted Annotation

    Temple University · University of Illinois Chicago

  4. Multi-Task Bayesian In-Context Learning

    Jun 18, 2026Qingyang Zhu, Eric Karl Oermann, Kyunghyun ChoAmortized InferenceBayesian Inference

    New York University · NYU Langone Health

  5. What Do Safety-Aligned LLMs Learn From Mixed Compliance Demonstrations?

    Jun 18, 2026Sihui Dai, Mann PatelAdversarial Attacks on LLMsIn-Context Learning

    CapitalOne

  6. Can In-Context Learning Support Intrinsic Curiosity?

    Jun 17, 2026Eric Elmoznino, Sangnie Bhardwaj, Johannes von Oswald +5Intrinsic Reward MethodsIntrinsic Motivation

    1Google – Paradigms of Intelligence Team, 2Google DeepMind

  7. Bounded Context Management for Tabular Foundation Models on Stream Learning

    Jun 17, 2026Jinmo Lee, Doyun Choi, Moongi Choi +1In-Context Example SelectionTabular Foundation Models

    Department of Computer Science and Engineering, Seoul National University, Seoul, Republic of Korea · The Kim Jaechul Graduate School of AI, KAIST, Daejeon, Republic of Korea.

  8. BCL: Bayesian In-Context Learning Framework for Information Extraction

    Jun 17, 2026Haoliang Liu, Chengkun Cai, Xu Zhao +7In-Context LearningLLM Information Extraction

    HiThink Research · University College London · University of Edinburgh +6

  9. When English Isn't the Best Teacher: Source Language Effects in Cross-Lingual In-Context Learning

    Jun 16, 2026Fred Philippy, Siwen Guo, Jacques Klein +1Multilingual Language Model EvaluationIn-Context Learning

    Snt, University of Luxembourg, Luxembourg · Luxembourg Institute of Science and Technology, Luxembourg

  10. Brick-DICL: Dynamic In-Context Learning for Automated Brick Schema Classification

    Jun 16, 2026Yiyue Qian, Shinan Zhang, Huan Song +3In-Context Learning

    Amazon AWS Generative AI Innovation Center

  11. Revisiting the Systematicity in Negation in the Era of In-Context Learning

    Jun 15, 2026Hitomi Yanaka, Taisei YamamotoTask VectorsIn-Context Learning

    The University of Tokyo · Riken · Tohoku University

  12. When Does Depth Matter For In-Context Learning? Adaptive Inference in Deep Transformers

    Jun 15, 2026Ravin Raj, Gautam ReddyTransformer InterpretabilityAdaptive Inference

    Joseph Henry Laboratories of Physics, Princeton University, Princeton, New Jersey 08544, USA

  13. Distilling Examples into Task Instructions: Enhanced In-Context Learning for Real-World B2B Conversations

    Jun 14, 2026Guy Rotman, Adi Kopilov, Danit Berger Zalmanson +1Text ClassificationFew-Shot Learning

  14. From Frames to Temporal Graphs: In-Context Egocentric Action Recognition with Vision-Language Models

    Jun 13, 2026Bessie Dominguez-Dager, Francisco Gomez-Donoso, Miguel Cazorla +3Egocentric Video UnderstandingIn-Context Learning

    University of Alicante · ETH Zürich · Microsoft

  15. Encode Errors: Representational Retrieval of In-Context Demonstrations for Multilingual Grammatical Error Correction

    Jun 13, 2026Guangyue Peng, Wei Li, Wen Luo +1In-Context Example SelectionGrammatical Error Correction

    State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University

  16. Knowledge Graph Enhanced Memory-Augmented Retrieval for Long Context Modeling

    Jun 12, 2026Ghadir Alselwi, Basem Suleiman, Hao Xue +4Memory-Augmented Language ModelsKnowledge Augmentation for Language Models

    University of New South Wales, Sydney, NSW, Australia · 2Hong Kong University of Science and Technology (Guangzhou) · University of Southampton, Southampton, UK +2

  17. GRIP: Feedback-Guided Prompt Retrieval for Large Multimodal Models

    Jun 10, 2026Garvita Allabadi, Matteo Sodano, Roberto Estevão +4Multimodal ICLMultimodal Large Language Models

    University of Illinois Urbana Champaign · University of Bonn · Microsoft

  18. Slots, Transitions, Loops: Learning Composable World Models for ARC

    Jun 10, 2026Gege Gao, Bernhard Schölkopf, Andreas GeigerWorld Model LearningVisual Reasoning

    University of Tübingen · ETH Zürich

  19. VICX: Generalizable Robot Manipulation via Video Generation and In-Context Operator Network

    Jun 10, 2026Song Chen, Linyan Xiang, Ying Zhou +1ICL for RoboticsLanguage-Conditioned Robot Manipulation

    National University of Singapore

  20. Categorical Prior Lock-in: Why In-Context Learning Fails for Structured Data

    Jun 10, 2026Antonio Pelusi, Stefano Braghin, Alberto TrombettaLLM Fine-TuningIn-Context Learning

    University of Insubria, Italy · IBM Research, Ireland

  21. Task-Aware Structured Memory for Dynamic Multi-modal In-Context Learning

    Jun 10, 2026Zhirui Chen, Ziwei Chen, Ling ShaoMemory-Augmented Language ModelsMultimodal ICL

    UCAS-Terminus AI Lab, University of Chinese Academy of Sciences, China · ByteDance Inc.

  22. CRUMB: Efficient Prior Fitted Network Inference via Distributionally Matched Context Batching

    Jun 9, 2026Jamie Heredge, Mattia J. Villani, Pranav Deshpande +2In-Context LearningPrior-Data Fitted Networks

    Global Technology Applied Research, JPMorganChase, New York, NY 10001, USA

  23. Quo Vadis, Visual In-Context Learning? A Unified Benchmark Across Domains and Tasks

    Jun 9, 2026Pradnya Halady, Jiale Wei, Zdravko Marinov +2In-Context Learning

    Karlsruhe Institute of Technology

  24. Beyond Model Size: Probing the Gaps in Visual in-Context Learning by Training a Tiny Model

    Jun 9, 2026Sunil Khatri, Steven Landgraf, Markus Ulrich +1Few-Shot LearningIn-Context Learning

    Karlsruhe Institute of Technology

  25. Pose-ICL: 3D-Aware In-Context Learning for Pose-Controllable Subject Customization

    Jun 9, 2026Xuan Han, Yihao Zhao, Mingyu YouSubject-Driven Image GenerationIn-Context Learning

    College of Electronic and Information Engineering, Tongji University, Shanghai, China · State Key Laboratory of Autonomous Intelligent Unmanned Systems, Frontiers Science Center for Intelligent Autonomous Systems, Ministry of Education, Shanghai, China.

  26. In-Context Learning for the Imputation of Public Opinion Data with Large Language Models

    Jun 8, 2026Tobias Holtdirk, Georg Ahnert, Joseph W Sakshaug +1Incomplete Data ImputationLearning with Missing Data

    LMU Munich · Munich Center for Machine Learning · University of Mannheim +2

  27. Safety is Contextual, LLM-Judges Are Not: Navigating the Rigid Priors of Evaluators

    Jun 5, 2026Anissa Alloula, Federico Licini, Ava Batchkala +1LLM-as-a-JudgeIn-Context Learning

    University of Oxford · Cohere

  28. Supervision versus Demonstration-Based In-Context Learning for Multiword Expression Classification

    Jun 5, 2026Sercan Karakaş, Yusuf ŞimşekLLM EvaluationZero-Shot Text Classification

    University of Chicago · Fırat University

  29. The Effect of Training Task Diversity on In-Context Learning through the Lens of Low-Dimensional Subspaces

    Jun 5, 2026Soo Min Kwon, Alec S. Xu, Can Yaras +3OOD GeneralizationTransformer Attention

    Department of Electrical and Computer Engineering, University of Michigan · Department of Statistics, University of California, Davis

  30. In-Context Multiple Instance Learning

    Jun 4, 2026Alexander Möllers, Marvin Sextro, Julius Hense +2Synthetic Data PretrainingFew-Shot Learning

    Berlin Institute for the Foundations of Learning and Data, Berlin, Germany · Machine Learning Group, Technische Universität Berlin, Berlin, Germany · Aignostics, Berlin, Germany +3