In-Context Learning

Also known as ICL

Ask about this topic

Overview

In-context learning (ICL) studies how a model can infer a task, rule, or preference from information in its prompt and apply it to new inputs, making adaptation possible without task-specific weight updates. Common approaches include few-shot demonstrations, retrieval and ordering strategies, synthetic-task pretraining for prior-fitted models, and test-time optimization of prompts or memory; evaluations range from synthetic regression and Markov tasks to language, vision, time-series, graph, and robotics benchmarks. Theory and mechanistic studies examine what algorithms attention and recurrent models implement, and how context length, training-task diversity, and architecture affect generalization. A central tension is that models may encode contextual information yet fail to use it reliably: performance can be sensitive to example quality, order, noise, and conflict with pretrained priors, while longer contexts increase inference costs.

Latest papers

CardsList

402 papers64 papers in the last four weeks, up 52% on the four weeks before. 0.4% of all new papers.

  1. Active In-Context Learning for Tabular Foundation Models

    Mar 28, 2026Wilailuck Treerath, Fabrizio PittorinoActive LearningTabular ICL

    Dipartimento di Elettronica, Informazione e Bioingegneria, Politecnico di Milano, Milano, Italy

  2. Can AI Scientist Agents Learn from Lab-in-the-Loop Feedback? Evidence from Iterative Perturbation Discovery

    Mar 27, 2026Gilles Wainrib, Barbara Bodinier, Haitem Dakhli +5AI Agents for Scientific DiscoveryLarge Language Model-Guided Scientific Discovery

    Owkin Inc.

  3. In-Context Molecular Property Prediction with LLMs: A Blinding Study on Memorization and Knowledge Conflicts

    Mar 26, 2026Matthias Busch, Marius Tacke, Sviatlana V. Lamaka +4LLM EvaluationKnowledge Conflicts in Language Models

    Institute for Continuum and Material Mechanics, Technical University of Hamburg, Eißendorfer Straße, 21073 Hamburg, Germany · Institute of Material Systems Modeling, Helmholtz-Zentrum Hereon, Max-Planck-Straße, 21502 Geesthacht, Germany · Institute of Surface Science, Helmholtz-Zentrum Hereon, Max-Planck-Straße, 21502 Geesthacht, Germany +2

  4. Toward domain-specific machine translation and quality estimation systems

    Mar 26, 2026Javad Pourmostafa Roshan SharamiMachine TranslationDomain Adaptation

  5. Context-Mediated Domain Adaptation in Multi-Agent Sensemaking Systems

    Mar 25, 2026Anton Wolter, Leon Haag, Vaishali Dhanoa +1Human-AI InteractionDomain Adaptation for LLMs

    Aarhus University, Denmark · Maastricht University, Netherlands · TU Wien, Austria

  6. LiFT: How to Enable In-Context Learning for Longitudinal Modelling

    Mar 25, 2026Iqra Ali, Talia Tseriotou, Mahmud Elahi Akhter +2In-Context LearningLLM Fine-Tuning

    Queen Mary University of London (UK) · The Alan Turing Institute (UK)

  7. UniICL: Systematizing Unified Multimodal In-context Learning through a Capability-Oriented Taxonomy

    Mar 25, 2026Yicheng Xu, Jiangning Zhang, Zhucun Xue +5Multimodal ICLUnified Multimodal Models

    1Zhejiang University, China. · 2Shanghai Jiaotong University, China. · 3National University of Singapore, Singapore. +1

  8. ConceptKT: A Benchmark for Concept-Level Deficiency Prediction in Knowledge Tracing

    Mar 25, 2026Yu-Chen Kang, Yu-Chien Tang, An-Zi YenKnowledge TracingIntelligent Tutoring Systems

    Department of Computer Science, National Yang Ming Chiao Tung University, Taiwan

  9. StateLinFormer: Stateful Training Enhancing Long-term Memory in Navigation

    Mar 24, 2026Zhiyuan Chen, Yuxuan Zhong, Fan Wang +4Linear AttentionIn-Context Learning

    Shenzhen Institute of Artificial Intelligence and Robotics for Society, Shenzhen, China

  10. Multitask-Informed Prior for In-Context Learning on Tabular Data: Application to Steel Property Prediction

    Mar 24, 2026Dimitrios Sinodinos, Bahareh Nikpour, Jack Yi Wei +5Multi-Task LearningTabular Foundation Models

    Dep. of Electrical and Computer Engineering, McGill University · Mila – Québec AI Institute · Dep. of Mining and Materials Engineering, McGill University +1

  11. A Foundation Model for Instruction-Conditioned In-Context Time Series Tasks

    Mar 23, 2026Anish Saha, Konstantin ShmakovZero-Shot Time Series ForecastingMeta-Learning

    Walmart Sunnyvale, USA

  12. Rashid: A Cipher-Based Framework for Exploring In-Context Language Learning

    Mar 23, 2026Niyati Bafna, Ryan Soh-Eun Shim, Barbara Plank +2In-Context LearningLow-Resource Language Processing

    Johns Hopkins University, Center for Language and Speech Processing · MaiNLP, Center for Information and Language Processing, LMU Munich

  13. PLR: Plackett-Luce for Reordering In-Context Learning Examples

    Mar 22, 2026Pawel Batorski, Paul SwobodaIn-Context LearningFew-Shot Learning

    Heinrich Heine Universität Düsseldorf

  14. Understanding Contextual Recall in Transformers: How Finetuning Enables In-Context Reasoning over Pretraining Knowledge

    Mar 21, 2026Bhavya Vasudeva, Puneesh Deora, Alberto Bietti +2In-Context LearningImplicit Reasoning in Language Models

    University of Southern California · University of British Columbia · Flatiron Institute

  15. When Demonstrations Fail: Diagnosing the Limits of In-Context Learning in Large Audio-Language Models with Progressive Cue Removal

    Mar 20, 2026Yen-Ting Piao, Jay Chiehen Liao, Wei-Tang Chien +5Audio-Language Model EvaluationLLM Evaluation

    National Taiwan University, Taiwan

  16. Demonstrations, CoT, and Prompting: A Theoretical Analysis of ICL

    Mar 20, 2026Xuhan Tong, Yuchen Zeng, Jiawei ZhangIn-Context LearningCoT Reasoning

    Department of Computer Sciences, University of Wisconsin-Madison · Microsoft Research

  17. Lossless Prompt Compression via Dictionary-Encoding and In-Context Learning: Enabling Cost-Effective LLM Analysis of Repetitive Data

    Mar 19, 2026Andresa Rodrigues de Campos, David Lee, Imry Kissos +1Lossless Data CompressionLLM Compression

    Amazon.com.

  18. PromptHub: Enhancing Multi-Prompt Visual In-Context Learning with Locality-Aware Fusion, Concentration and Alignment

    Mar 19, 2026Tianci Luo, Jinpeng Wang, Shiyu Qin +5In-Context LearningFeature Fusion

    Tsinghua Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology, Shenzhen · Meituan, Beijing

  19. Transformers Learn Robust In-Context Regression under Distributional Uncertainty

    Mar 19, 2026Hoang T. H. Cao, Hai D. V. Trinh, Tho Quan +1In-Context LearningTransformer

    Faculty of Computer Science and Engineering, Ho Chi Minh University of Technology (HCMUT), Vietnam National University Ho Chi Minh City (VNU-HCM), Vietnam

  20. Arabic Morphosyntactic Tagging and Dependency Parsing with Large Language Models

    Mar 17, 2026Mohamed Adel, Bashar Alhafni, Nizar HabashLLM EvaluationArabic NLP

    Computational Approaches to Modeling Language Lab 1New York University Abu Dhabi · 2Mohamed bin Zayed University of Artificial Intelligence

  21. Can Linguistically Related Languages Guide LLM Translation in Low-Resource Settings?

    Mar 17, 2026Aishwarya Ramasethu, Niyathi Allu, Rohin Garg +2Machine TranslationLow-Resource MT

    Prediction Guard · Independent · Scale AI +1

  22. Parallel In-context Learning for Large Vision Language Models

    Mar 17, 2026Shin'ya Yamaguchi, Daiki Chijiwa, Tamao Sakao +1Product of ExpertsIn-Context Learning

    NTT

  23. A Comparative Study of Demonstration Selection for Practical Large Language Models-based Next POI Prediction

    Mar 16, 2026Ryo Nishida, Masayuki Kawarada, Tatsuya Ishigaki +2In-Context LearningLarge Language Model-Based Recommendation

    National Institute of Advanced Industrial Science and Technology, Japan

  24. AD-Copilot: A Vision-Language Assistant for Industrial Anomaly Detection via Visual In-context Comparison

    Mar 14, 2026Xi Jiang, Yue Guo, Jian Li +7Industrial Anomaly DetectionIndustrial Inspection

    Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China · Macau University of Science and Technology, Macau, China · Tencent YouTu Lab, Shenzhen and Shanghai, China +3

  25. Reconciling In-Context and In-Weight Learning via Dual Representation Space Encoding

    Mar 13, 2026Guanyu Chen, Ruichen Wang, Tianren Zhang +1In-Context LearningRepresentation Learning

    Department of Automation, Tsinghua University

  26. Modality-free Graph In-context Alignment

    Mar 13, 2026Wei Zhuo, Siqiang LuoGraph Foundation ModelsIn-Context Learning

    Nanyang Technological University

  27. Design-Specification Tiling for ICL-based CAD Code Generation

    Mar 13, 2026Yali Du, San-Zhuo Xi, Hui Sun +1In-Context LearningText-to-CAD Generation

    National Key Laboratory for Novel Software Technology, School of Artificial Intelligence, Nanjing University, China

  28. Implicit Statistical Inference in Transformers: Approximating Likelihood-Ratio Tests In-Context

    Mar 11, 2026Faris Chaudhry, Siddhant GadkariLikelihood Ratio TestingIn-Context Learning

    Department of Computer Science Imperial College London

  29. Regime-aware financial volatility forecasting via in-context learning

    Mar 11, 2026Saba Asaad, Shayan Mohajer Hamidi, Ali BereyhiFinancial Time SeriesLLM-Based Time Series Forecasting

    Department of Electrical and Computer Engineering, University of Toronto, Toronto, Canada · Department of Electrical Engineering, Stanford University, Stanford, CA, USA

  30. Using Vision Language Foundation Models to Generate Plant Simulation Configurations via In-Context Learning

    Mar 9, 2026Heesup Yun, Isaac Kazuo Uyehara, Earl Ranario +4VLM EvaluationAgricultural Image Analysis

    University of California, Davis