In-Context Learning

Also known as ICL

Overview

In-context learning (ICL) studies how a model can infer a task, rule, or preference from information in its prompt and apply it to new inputs, making adaptation possible without task-specific weight updates. Common approaches include few-shot demonstrations, retrieval and ordering strategies, synthetic-task pretraining for prior-fitted models, and test-time optimization of prompts or memory; evaluations range from synthetic regression and Markov tasks to language, vision, time-series, graph, and robotics benchmarks. Theory and mechanistic studies examine what algorithms attention and recurrent models implement, and how context length, training-task diversity, and architecture affect generalization. A central tension is that models may encode contextual information yet fail to use it reliably: performance can be sensitive to example quality, order, noise, and conflict with pretrained priors, while longer contexts increase inference costs.

Latest papers

CardsList

429 papers46 papers in the last four weeks, down 26% on the four weeks before. 0.3% of all new papers.

  1. PhysICL: In-Context Learning of Physical Fields on Unstructured Meshes from Procedural Priors

    Nov 11, 2026Fabien CasenavePDE Surrogate ModelingNeural Surrogate Modeling

  2. The Neighbour's Label: Opposite Subgroup Effects of Same-Dialogue Exposure in Next-Act Evaluation

    Nov 11, 2026Ali Amini, Masoumeh Chapariniya, Rytis Maskeliunas +1Data LeakageMulti-Turn Dialogue Evaluation

  3. Query-Only Prompt-Agnostic Attacks on Visual In-Context Learning

    Nov 8, 2026Xiao Hu, Muhammad Ibrahim, Hao Wen +3Black-Box Adversarial AttacksIn-Context Learning

  4. Beyond Discrete Ancestry Groups: Distribution-Aware In-Context Learning for Cancer Outcome Prediction Across the Genetic Ancestry Continuum

    Nov 7, 2026Weizhi Li, Yan CuiLearning with Distribution ShiftClinical Risk Prediction

  5. An Effective Spoken Language Understanding Approach Leveraging Query-Adaptive Intent Detection

    Nov 6, 2026Bing-Long Kuo, Yu-Hsuan Fang, Tien-Hong Lo +2Spoken Language UnderstandingIntent Classification

  6. Definitions, Label Names, or Demonstrations? Auditing What Governs LLM Annotation

    Nov 6, 2026Viraj Mishra, Shubham Gupta, Dhruv Gupta +2LLM-Assisted AnnotationLLM Auditing

  7. NutriVisRAG: Multimodal Exemplar Grounding for Nutrient Estimation with Vision-Language Models

    Nov 5, 2026Lubnaa Abdur Rahman, Ioannis Papathanail, Maria Panagiotou +5In-Context Learning

  8. When, Where, How: Active In-Context Learning for Tabular Foundation Models

    Nov 5, 2026Christian M. M. FreyActive LearningIn-Context Learning

  9. A Circuit for In-Context Learning

    Nov 3, 2026Andrew Gao, William GaoIn-Context LearningCircuits in Language Models

  10. Molecular Prior Networks

    Nov 3, 2026Pawel Gomoluch, Anthony BradleyProtein-Ligand Binding Affinity PredictionMolecular Foundation Models

  11. A Systematic Comparison of In-Context Learning Strategies for Peptide Property Prediction in Low-Data Regimes

    Nov 3, 2026Jiantao Zhao, Andrei Voinea, Mathijs F. J. MabesooneMolecular Property PredictionIn-Context Learning

  12. Variable names provide priors: semantic grounding enables data-efficient numerical prediction by large language models

    Nov 2, 2026Xin-Rui Ma, Shi-Zhi Chen, Hai-Yan Jiang +2LLM GroundingTabular Prediction

  13. Solvable dynamics model of in-context learning

    Oct 31, 2026Ruoran Bai, Haiping HuangIn-Context LearningTransformer Training Dynamics

  14. Can Jev Improve Demonstration Retrieval for In-Context Learning?

    Oct 31, 2026Rui Song, Yingji Li, Hao XuIn-Context Example SelectionIn-Context Learning

  15. Do LLMs Know When They Are Wrong About Molecules? In-Context Property Prediction with Self-Reported Uncertainty

    Oct 30, 2026Matthias Busch, Marius Tacke, Tamadur Albaraghtheh +5Molecular Property PredictionLLM Evaluation

  16. In-Context Learning for Multi-Fidelity Multiscale Model Closure and Cross-Operator Out-Of-Distribution Generalization

    Oct 28, 2026Xinyuan Li, Matthew Zahr, Panos Stinis +1Multifidelity ModelingPDE Operator Learning

  17. An Algorithmic Account of Few-Shot Concept Induction in Language Models

    Oct 26, 2026Michael A. Lepori, Laila Johnston, Aalok Sathe +3Inductive Reasoning in Language ModelsIn-Context Learning

  18. POS-Signature Demonstration Retrieval for In-Context Dependency Parsing

    Oct 22, 2026Muhammad Azam Afridi, Rajab Ali, Marco PassarottiIn-Context Example SelectionDependency Parsing

  19. How Depth Growth and Looping Shape In-Context Learning in Language Models

    Oct 22, 2026Ferdinand Kapl, Kaitlin Maile, Stefan Bauer +1In-Context LearningParameter Sharing

  20. Transformers with CoT Discover Heavy-Ball Acceleration Through Training

    Oct 22, 2026Yuxuan Cai, Yuan Caoin-context regressionCoT Reasoning

  21. Watch and Learn: Teaching Visual In-Context Learning by Removing Textual Shortcuts

    Oct 22, 2026Nimrod Shabtay, Asaf Yehudai, Avihu Dekel +2Multimodal ICLIn-Context Learning

  22. Collaborative In-Context Learning for Indoor Radio Map Estimation

    Oct 21, 2026Yuanzhe Peng, Jie XuIn-Context Learning

  23. Cognitively Decoupled Meta-Induction for Out-of-Distribution Generalization

    Oct 19, 2026Yuqian Wang, Yuanzheng Wang, Chaojun Xiao +7Inductive Reasoning in Language ModelsOOD Generalization