In-Context Learning

Also known as ICL

Ask about this topic

Overview

In-context learning (ICL) studies how a model can infer a task, rule, or preference from information in its prompt and apply it to new inputs, making adaptation possible without task-specific weight updates. Common approaches include few-shot demonstrations, retrieval and ordering strategies, synthetic-task pretraining for prior-fitted models, and test-time optimization of prompts or memory; evaluations range from synthetic regression and Markov tasks to language, vision, time-series, graph, and robotics benchmarks. Theory and mechanistic studies examine what algorithms attention and recurrent models implement, and how context length, training-task diversity, and architecture affect generalization. A central tension is that models may encode contextual information yet fail to use it reliably: performance can be sensitive to example quality, order, noise, and conflict with pretrained priors, while longer contexts increase inference costs.

Latest papers

CardsList

404 papers64 papers in the last four weeks, up 52% on the four weeks before. 0.4% of all new papers.

  1. CAARL: In-Context Learning for Interpretable Co-Evolving Time Series Forecasting

    Apr 20, 2026Etienne Tajeuna, Patrick Asante Owusu, Armelle Brun +1Explainable Time Series ForecastingTime Series Forecasting

  2. Beyond Fine-Tuning: In-Context Learning and Chain-of-Thought for Reasoned Distractor Generation

    Apr 19, 2026Elaf Alhazmi, Quan Z. Sheng, Wei Emma ZhangLLM PromptingIn-Context Learning

    School of Computing, Macquarie University, Australia · College of Engineering and Computing in Al-Lith, Umm Al-Qura University, Saudi Arabia · School of Computer and Mathematical Sciences, Adelaide University, Australia

  3. ICLAD: In-Context Learning with Comparison-Guidance for Audio Deepfake Detection

    Apr 17, 2026Benjamin Chou, Yi Zhu, Surya KoppisettiAudio Deepfake DetectionIn-Context Learning

    Purdue University, USA · Reality Defender Inc., USA

  4. Why Multimodal In-Context Learning Lags Behind? Unveiling the Inner Mechanisms and Bottlenecks

    Apr 15, 2026Yu Wang, Sharon LiMultimodal ICLIn-Context Learning

    Department of Computer Sciences, University of Wisconsin-Madison

  5. KumoRFM-2: Scaling Foundation Models for Relational Learning

    Apr 14, 2026Valter Hudovernik, Federico López, Vid Kocijan +4Relational ICLRelational Foundation Models

  6. A Bayesian Perspective on the Role of Epistemic Uncertainty for Delayed Generalization in In-Context Learning

    Apr 14, 2026Abdessamed Qchohi, Simone RossiIn-Context LearningGrokking

    EURECOM, France

  7. Distinct mechanisms underlying in-context learning in transformers

    Apr 14, 2026Cole Gibson, Wenping Cui, Gautam ReddyIn-Context LearningTransformer Training Dynamics

    Lewis-Sigler Institute for Integrative Genomics, Princeton University, Princeton, NJ 08544, USA · Department of Physics, Princeton University, Princeton, NJ 08544, USA

  8. UCS: Estimating Unseen Coverage for Improved In-Context Learning

    Apr 13, 2026Jiayi Xin, Xiang Li, Evan Qiang +4In-Context Example SelectionIn-Context Learning

    University of Pennsylvania

  9. Legal2LogicICL: Improving Generalization in Transforming Legal Cases to Logical Formulas via Diverse Few-Shot Learning

    Apr 13, 2026Jieying Xue, Phuong Minh Nguyen, Ha Thanh Nguyen +2Legal ReasoningLegal NLP

    Center for Juris-Informatics, ROIS-DS, Tokyo, Japan · Japan Advanced Institute of Science and Technology, Ishikawa, Japan

  10. Symmetry Reveals Layerwise Dynamics: How Transformers Perform In-Context Classification

    Apr 13, 2026Patrick Lutz, Themistoklis Haris, Arjun Chandra +2In-Context LearningTransformer Interpretability

    Boston University, Departments of Computer Science and Electrical & Computer Engineering.

  11. Toward Robust In-Context Learning: Leveraging Out-of-distribution Proxies for Target Inaccessible Demonstration Retrieval

    Apr 13, 2026Hao Xu, Rite Bo, Fausto Giunchiglia +2In-Context LearningLanguage Model Robustness

    College of Computer Science and Technology, Jilin University, China · Department of Information Engineering and Computer Science, University of Trento, Italy

  12. Learning to Adapt: In-Context Learning Beyond Stationarity

    Apr 13, 2026Zhen Qin, Jiachen Jiang, Zhihui ZhuIn-Context LearningGated Attention

    Michigan Institute for Computational Discovery and Engineering, Department of Electrical Engineering and Computer Science, Department of Statistics, University of Michigan · Department of Computer Science and Engineering, The Ohio State University

  13. Transformers Learn Latent Mixture Models In-Context via Mirror Descent

    Apr 12, 2026Francesco D'Angelo, Nicolas FlammarionIn-Context LearningTransformer Attention

    TML Lab, EPFL, Switzerland

  14. EviCare: Enhancing Diagnosis Prediction with Deep Model-Guided Evidence for In-Context Reasoning

    Apr 12, 2026Hengyu Zhang, Xuyun Zhang, Pengxiang Zhan +5Clinical PredictionMedical LLMs

    Macquarie University Sydney, Australia · Fuzhou University Fuzhou, China · Monash University Melbourne, Australia +2

  15. Single-Position Intervention Fails: Distributed Output Templates Drive In-Context Learning

    Apr 10, 2026Bryan Cheng, Jasper ZhangIn-Context LearningLLM Interpretability

    William A. Shine Great Neck South High School

  16. Noise-Aware In-Context Learning for Hallucination Mitigation in ALLMs

    Apr 10, 2026Qixuan Huang, Khalid Zaman, Masashi UnokiAudio-Language ModelsAudio-Language Model Hallucination Detection

    Graduate School of Advanced Science and Technology, Japan Advanced Institute of Science and Technology 1-1 Asahidai, Nomi, Ishikawa 923–1292 Japan

  17. Evaluating In-Context Translation with Synchronous Context-Free Grammar Transduction

    Apr 8, 2026Jackson Petty, Jaulie Goe, Tal LinzenMachine TranslationIn-Context Learning

    Department of Linguisticsλ New York University · Center for Data Scienceδ New York University

  18. From Static to Interactive: Adapting Visual in-Context Learners for User-Driven Tasks

    Apr 8, 2026Carlos Schmidt, Simon ReißIn-Context LearningPromptable Image Segmentation

    Fraunhofer IOSB Karlsruhe, Germany · Karlsruhe Institute of Technology Karlsruhe, Germany

  19. GraphWalker: Patient Analogy Meets Information Gain for Clinical Reasoning with Large Language Models

    Apr 8, 2026Yue Fang, Weibin Liao, Yuxin Guo +8Case-Based ReasoningClinical Reasoning

    School of Computer Science, Peking University, Beijing, China · National Engineering Research Center for Software Engineering, Peking University, Beijing, China

  20. In-Context Learning in Speech Language Models: Analyzing the Role of Acoustic Features, Linguistic Structure, and Induction Heads

    Apr 7, 2026Charlotte Pouw, Hosein Mohebbi, Afra Alishahi +1In-Context LearningSpeech Language Models

    ILLC, University of Amsterdam · CSAI, Tilburg University

  21. Reason Analogically via Cross-domain Prior Knowledge: An Empirical Study of Cross-domain Knowledge Transfer for In-Context Learning

    Apr 7, 2026Le Liu, Zhiming Li, Jianzhi Yan +7In-Context LearningCross-Domain Transfer Learning

    Harbin Institute of Technology, Shenzhen, China · Pengcheng Laboratory, Shenzhen, China

  22. Decocted Experience Improves Test-Time Inference in LLM Agents

    Apr 6, 2026Maohao Shen, Kaiwen Zha, Zexue He +6LLM Context ManagementMemory-Augmented Language Model Agents

    Massachusetts Institute of Technology · Stanford University · MIT-IBM Watson AI Lab +2

  23. Emergent Inference-Time Semantic Contamination via In-Context Priming

    Apr 5, 2026Marcin AbramLLM SecurityIn-Context Learning

    Los Angeles, California 90089, USA

  24. Love Me, Love My Label: Rethinking the Role of Labels in Prompt Retrieval for Visual In-Context Learning

    Apr 4, 2026Tianci Luo, Haohao Pan, Jinpeng Wang +5In-Context LearningRetrieval-Augmented ICL

    Tsinghua Shenzhen International Graduate School, Tsinghua University · School of Computer Science and Engineering, Northeastern University · Harbin Institute of Technology, Shenzhen

  25. Reflective Context Learning: Studying the Optimization Primitives of Context Space

    Apr 3, 2026Nikita Vassilyev, William Berrios, Ruowang Zhang +3LLM Agent Self-ImprovementIn-Context Learning

    Contextual AI

  26. Deformation-based In-Context Learning for Point Cloud Understanding

    Apr 3, 2026Chengxing Lin, Jinhong Deng, Yinjie Lei +1Point Cloud DenoisingPoint Cloud Learning

    Shenzhen Institute for Advanced Study, UESTC · School of Computer Science and Engineering, UESTC · Sichuan University

  27. An Empirical Study of Many-Shot In-Context Learning for Machine Translation of Low-Resource Languages

    Apr 3, 2026Yinhan Lu, Gaganpreet Jhajj, Chen Zhang +2In-Context LearningLow-Resource MT

    Mila – Quebec AI Institute · McGill University · Athabasca University +2

  28. Contrast Matters: Understanding Robustness of In-Context Fine-Tuning to Target-Context Relatedness

    Apr 2, 2026Deeptanshu Malu, Deevyanshu Malu, Aditya Nemiwal +1In-Context Example SelectionLLM Fine-Tuning

    IIT Bombay

  29. Temporal Dependencies in In-Context Learning: The Role of Induction Heads

    Apr 1, 2026Anooshka Bajaj, Deven Mahesh Mistry, Sahaj Singh Maini +3In-Context LearningTemporal Reasoning in Language Models

    Department of Computer Science, Indiana University Bloomington

  30. Survival In-Context: Amortized Bayesian Survival Analysis via Prior-Fitted Networks

    Mar 31, 2026Dmitrii Seletkov, Paul Hager, Georgios Kaissis +3Survival AnalysisPrior-Data Fitted Networks

    Institute of Diagnostic and Interventional Radiology, Technical University of Munich, Germany · Chair for AI in Healthcare and Medicine, Technical University of Munich, Germany · Hasso Plattner Institute for Digital Engineering, University of Potsdam, Germany +3