In-Context Learning

Also known as ICL

Overview

In-context learning (ICL) studies how a model can infer a task, rule, or preference from information in its prompt and apply it to new inputs, making adaptation possible without task-specific weight updates. Common approaches include few-shot demonstrations, retrieval and ordering strategies, synthetic-task pretraining for prior-fitted models, and test-time optimization of prompts or memory; evaluations range from synthetic regression and Markov tasks to language, vision, time-series, graph, and robotics benchmarks. Theory and mechanistic studies examine what algorithms attention and recurrent models implement, and how context length, training-task diversity, and architecture affect generalization. A central tension is that models may encode contextual information yet fail to use it reliably: performance can be sensitive to example quality, order, noise, and conflict with pretrained priors, while longer contexts increase inference costs.

Latest papers

CardsList

429 papers31 papers in the last four weeks, down 58% on the four weeks before. 0.2% of all new papers.

  1. Data Efficient Sample Selection for In-Context Learning

    Sep 6, 2026V Venktesh, Cem Levi, Avishek AnandIn-Context Example SelectionIn-Context Learning

  2. LoGIC: Budgeted Context Construction for Node-Level Graph In-Context Learning with Tabular Foundation Models

    Sep 5, 2026Mingqi Yang, Zidong Guo, Jihui Yang +1In-Context LearningRetrieval-Augmented ICL

  3. Constructing and Evaluating Clinical Reasoning Trajectories for Medical Agent

    Sep 4, 2026Yunqi Zhu, Wen Tang, Jinghao Niu +2Trajectory-Based EvaluationReasoning Evaluation

  4. LLM4CKD: Large Language Models for Early Stage Chronic Kidney Disease Screening

    Sep 3, 2026Muhammad Ashad Kabir, Sirajam MuniraMedical DiagnosisIn-Context Learning

  5. Typological Feature Prediction with Large Language Models: An In-Context Learning Approach

    Sep 3, 2026Qianwen Wang, York Hay Ng, Aditya Khan +1In-Context LearningLinguistic Typology

  6. LLMs Learn Better In-Context from Rules than from Examples

    Sep 2, 2026Xiang Fu, Seungmin Cho, Yukyung Lee +1Few-Shot LearningIn-Context Learning

  7. SALA: Semantic-Aware Logical Alignment for Complex Reasoning in In-Context Learning

    Sep 2, 2026Zhao Ji, Wenqing Chen, Zhixuan Chu +4demonstration selectionIn-Context Learning

  8. One-Layer Transformer Provably Learns Multiclass One-Nearest Neighbor in Context

    Sep 1, 2026Skanda Athreya, Yutong WangTransformerIn-Context Learning

  9. Compile, Don't Memorize: A Context Compilation Architecture (CCA) for In-Context Learning

    Sep 1, 2026Jinhu Qi, Minda Hu, Wentao Zhang +4Instruction FollowingIn-Context Learning

  10. Are These Modules Worth Their Cost? A Paradigm-Level Accuracy-Cost Analysis of In-context Learning Text-to-SQL

    Aug 28, 2026Jiayan Lin, Yujia Liu, Zijin Hong +6Execution-Guided Text-to-SQLText-to-SQL

  11. Landau theory of quenched criticality in linear in-context learning

    Aug 28, 2026Daesik Kim, Sumin Choi, Hyojae Jeon +1In-Context LearningDouble Descent

  12. Reasoning about In-Context Samples for Machine-Translation

    Aug 27, 2026Maxime Bouthors, Josep Crego, François YvonMachine TranslationIn-Context Learning

  13. Instruction Distillation: Text Instructions as Visual Examples

    Aug 27, 2026Hardik Jindal, Soumyabrata Pal, Sayak Ray ChowdhuryIn-Context LearningEfficient VLM Inference

  14. TransPhy: Visual In-Context Learning for Physically Grounded Image Editing

    Aug 25, 2026Siyi Xie, Xuanke Shi, Jinsheng Quan +4Image EditingIn-Context Learning

  15. ChorusTIC: Training-Free Multivariate Time Series Classification via Chorus In-Context Learning

    Aug 25, 2026Juntao Fang, Shifeng Xie, Ruichu Cai +6Multivariate Time Series ClassificationTime Series Classification

  16. In-Context Inpainting for Time Series Forecasting

    Aug 24, 2026Thang Nguyen, Dung Nguyen, Romero Morais +1Time Series ForecastingZero-Shot Time Series Forecasting

  17. On Predicting Vulnerability Severity Using In-Context Learning: An Industrial Case Study

    Aug 22, 2026Daniel Rodriguez-Cardenas, David Nader Palacio, Anna Schmedding +8Software SecurityLLMs for Cybersecurity

  18. RecPFN: Prior-Fitted Networks for In-Context-Based Recommendations

    Aug 20, 2026En Zhi Tan, Jia Xiang Lim, Bryan Lijie Chew +2Recommender SystemsSequential Recommendation

  19. Pretraining Reusable Inference Across Views with Synthetic Task Priors

    Aug 19, 2026Jielong Lu, Zhihao Wu, Jiajun Yu +2Multi-View LearningMultimodal ICL

  20. Institution-Specific LLM Prompting Recovers PHI That De-identification Systems and Their Gold Standards Both Miss

    Aug 17, 2026Daniel Palacios, Matthew Brady Neeley, Angel Adetomike Otto +6PII DetectionElectronic Health Records

  21. From Answers to Policies: Efficient In-Context Learning System through Emulating Expert Investigation

    Aug 17, 2026Minh-Ha Nguyen, Ngoc-Ngo Quang Tran, Thuy Dung Nguyen +1In-Context LearningIn-Context RL

  22. When Do Explanations Help In-Context Learning? A Comparative Study of Natural Language Explanation Types and Faithfulness

    Aug 17, 2026Mahdi Dhaini, Adam Dejl, Juraj Vladika +3In-Context LearningFaithfulness of Language Model Explanations

  23. Handover of In-Context Learning State Across Session Boundaries

    Aug 14, 2026Masahiro Kato, Taka KatoLLM Context ManagementPersistent Memory for Language Models

  24. Identifiability and Order-Dimension Limits of In-Context Learning on Partial Orders

    Aug 14, 2026Faizanuddin Ansari, Debanjan Dutta, Swagatam DasQuery IdentifiabilityIn-Context Learning

  25. Predicting Space Groups of Double Perovskites by LLM with Dynamic Few-Shot Learning

    Aug 11, 2026Jongwon Park, Inhyo Lee, Junhyeong Lee +1Class-Imbalanced LearningCrystal Structure Prediction