In-Context Learning

Also known as ICL

Overview

In-context learning (ICL) studies how a model can infer a task, rule, or preference from information in its prompt and apply it to new inputs, making adaptation possible without task-specific weight updates. Common approaches include few-shot demonstrations, retrieval and ordering strategies, synthetic-task pretraining for prior-fitted models, and test-time optimization of prompts or memory; evaluations range from synthetic regression and Markov tasks to language, vision, time-series, graph, and robotics benchmarks. Theory and mechanistic studies examine what algorithms attention and recurrent models implement, and how context length, training-task diversity, and architecture affect generalization. A central tension is that models may encode contextual information yet fail to use it reliably: performance can be sensitive to example quality, order, noise, and conflict with pretrained priors, while longer contexts increase inference costs.

Latest papers

CardsList

429 papers31 papers in the last four weeks, down 58% on the four weeks before. 0.2% of all new papers.

  1. Downstream-Aware Context Selection for Online In-Context Reinforcement Learning

    Sep 27, 2026Ruihan A. Li, Shangtong Zhang, Rohan ChandraIn-Context RLIn-Context Learning

  2. PULSE: Identifying Demonstration-Utility Features with Sparse Autoencoders

    Sep 26, 2026Chenduo Hao, Chuanbao Gao, Pinjun Zeng +4In-Context Example SelectionIn-Context Learning

  3. A Comparative Analysis of Attention versus State-Space Models for In-Context Learning

    Sep 26, 2026Enes Arda, Semih Cayci, Atilla Eryilmazin-context regressionState Space Models

  4. Decodable In-Context State and Model Output Across Training

    Sep 25, 2026Manas Venkata Sai Ravulapalli, Samrath Singh ChadhaLLM InterpretabilityActivation Steering

  5. In-Context Binding Capacity in Language Models

    Sep 24, 2026Manas Venkata Sai Ravulapalli, Samrath Singh ChadhaWorking Memory in Language ModelsLanguage Model Scaling Laws

  6. MILO: Efficient Many-shot In-Context Learning with Block-wise Low-rank Compression

    Sep 24, 2026Youpeng Zhao, Tian Tan, Liqian Peng +2Memory-Efficient InferenceLLM Inference Acceleration

  7. Transformers as Cross-Task Learners: Shared Structure Drives Sample Efficiency in In-Context Learning

    Sep 24, 2026Zhongjie Shi, Rongjie Lai, Alexander Cloninger +1TransformerMulti-Task Learning

  8. Gender Bias in Vision-Language In-Context Learning

    Sep 23, 2026Tong Xiang, Yuta Nakashima, Noa GarciaVLM EvaluationIn-Context Learning

  9. When Context Misleads: In-context Learning with Jurisdiction in Large Language Models

    Sep 23, 2026Pei-lin Li, Qingle Liu, Junyang Feng +4LLM EvaluationIn-Context Learning

  10. URA-NER: A Unified Retrieval-Augmented Framework with Retrieval Alignment and Uncertainty Reduction for Low-Resource NER

    Sep 21, 2026Jingyu Wang, Shijie Wu, Fusheng JinNamed Entity RecognitionIn-Context Learning

  11. Few-Shot Demonstrations Elicit the Use of In-Context World Representations in LLMs

    Sep 21, 2026Kohsei Matsutani, Gouki Minegishi, Core Francisco Park +3LLM World ModelsIn-Context Learning

  12. Representation-guided in-context learning for medical image interpretation with multimodal large language models

    Sep 21, 2026Minda Zhao, Fangyu Hu, Yan Luo +8Medical Image AnalysisMultimodal Large Language Models

  13. Evaluating In-Context Learning and Retrieval Strategies for Devanagari Post-OCR Correction

    Sep 18, 2026Abhishek Bhandari, Gaurav HaritIn-Context LearningLLM Evaluation

  14. In-Context Robot Learning with VLM Agents

    Sep 16, 2026Dongzhou Cheng, Taoran Yi, Ye Fang +12Robot Policy LearningICL for Robotics

  15. Long-Context Demonstration Selection Using State Space Models

    Sep 15, 2026Ziniu Zhang, Zhenshuo Zhang, Ruoxuan Xiong +2In-Context Example SelectionIn-Context Learning

  16. Large Language Models Develop Belief State Geometry In-Context

    Sep 15, 2026Daniel Balcells, Andrew Jun Lee, Chirag Rastogi +3In-Context LearningBelief Updating in LLMs

  17. Attention Mean Fields Predict Average Representation Dynamics and Reveal Context-Specific Computation

    Sep 14, 2026Micah Adler, John W. Byers, Mark CrovellaAttention Head AnalysisIn-Context Learning

  18. Convergent Emergence of In-Context Learning Across Modalities

    Sep 12, 2026Nathan Breslow, Seungwook Han, Daniel Hyunsoo Lee +3In-Context LearningMultimodal ICL

  19. Making Alternative Data Work: Context-Augmented LLMs for Financial Forecasting

    Sep 10, 2026Jihoon Kwon, Lawrence Liu, Daekyung Park +15Financial ForecastingTime Series Forecasting

  20. Legible Failures: Detecting and Repairing In-Context Binding Errors

    Sep 10, 2026Manas Venkata Sai Ravulapalli, Samrath Singh Chadha, Abhinav M. HariLanguage Model Error DetectionLLM Reliability

  21. MUCnoHARM@GermEval Shared Task 2026: Retrieval-based In-Context Learning for Defamatory Offences, and Where It Falls Short

    Sep 9, 2026Kristin Gnadt, Maximilian Meidinger, Matthias AßenmacherLegal NLPIn-Context Retrieval

  22. Transformers as In-Context Samplers: From Closed-Form Diffusion to Estimation-Free Sampling

    Sep 8, 2026Arman Adibi, Alireza Jafari, Mohammad Ghavamzadeh +1TransformerDiffusion Sampling