In-Context Learning

Also known as ICL

In-context learning (ICL) studies how a model can infer a task, rule, or preference from information in its prompt and apply it to new inputs, making adaptation possible without task-specific weight updates. Common approaches include few-shot demonstrations, retrieval and ordering strategies, synthetic-task pretraining for prior-fitted models, and test-time optimization of prompts or memory; evaluations range from synthetic regression and Markov tasks to language, vision, time-series, graph, and robotics benchmarks. Theory and mechanistic studies examine what algorithms attention and recurrent models implement, and how context length, training-task diversity, and architecture affect generalization. A central tension is that models may encode contextual information yet fail to use it reliably: performance can be sensitive to example quality, order, noise, and conflict with pretrained priors, while longer contexts increase inference costs.

Latest papers

404 papers64 papers in the last four weeks, up 52% on the four weeks before. 0.4% of all new papers.

Supervised Fine-Tuning vs. In-Context Learning: An Equilibrium Analysis of LLM Personalization under Congestion
Jul 15, 2026Fengzhuo Zhang, Zhuoran Yang, Dirk BergemannSupervised Fine-Tuning LLM Serving
Yale University, 30 Hillhouse Ave., New Haven, CT 06520, USA
Invariant Learning Dynamics of Transformers in Inductive Reasoning Tasks
Jul 13, 2026Tiberiu Musat, Tiago Pimentel, Nicolas Zucchet +1Transformer Training Dynamics In-Context Learning
ETH Zurich · Stanford University
Self-Evolving In-Context Learning for Direct Pilot-to-Beamformer Design in MU-MISO Systems
Jul 13, 2026Yubo Zhang, Xiaodong WangWireless Communications In-Context Learning
Department of Electrical Engineering, Columbia University, New York, NY 10027
Agentic Context Learning with Self-Discovered Specification
Jul 9, 2026Jike Zhong, Ming Li, Yuxiang Lai +8In-Context Learning
University of Southern California · University of Florida · 3Emory University +3
TimEE: End-to-end Time Series Classification via In-Context Learning
Jul 8, 2026Jaris Küken, Shi Bin Hoo, Martin Mráz +2Synthetic Data Pretraining Time Series Classification
University of Freiburg · Zuse School ELIZA Darmstadt · Prior Labs +1
Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity
Jul 8, 2026Loïc Cabannes, Pierre-Emmanuel Mazaré, Gergely Szilvasy +6Long-Context Retrieval Memory-Augmented Language Models
Meta FAIR · Inria Paris & ENS-PSL University · University of Tübingen
Tree-of-Thoughts Reasoning for Text-to-Image In-Context Learning
Jul 8, 2026Stepanida Alekseeva, Jenifer Kalafatovich, Seong-Whan LeeTree of Thoughts Multimodal ICL
Department of Artificial Intelligence, Korea University, Seoul, Korea.
AbICL: In-Context Learning for Antigen-Specific Antibody Affinity Ranking
Jul 7, 2026Zhiyuan Chen, Jing Hu, Junzhe Wang +4Learning to Rank Meta-Learning
Henlius Shanghai, China
Onnes: A Physics-Grounded Multi-Agent LLM Simulator for Cryogenic Fault Diagnosis in Quantum Computing Infrastructure
Jul 7, 2026Praneeth Narisetty, Uday Kumar Reddy Kattamanchi, Shiva Nagendra Babu KoreMulti-Agent LLM Systems LLM Agent Evaluation
Onnes Research | San Francisco, CA
Pretraining Curricula Enable Selective Fine-tuning
Jul 6, 2026Sebastian A. Bruijns, Jirko Rubruck, Mia H. Whitefield +3Disentangled Representation Learning Fine-Tuning
University of Oxford · University of Oxford & Martian · Martian
A Unified Framework for In-Context Learning with Causal and Masked Language Models
Jul 5, 2026Chenrui Liu, Chuanlong Xie, Falong Tan +2Masked Language Modeling Autoregressive Language Modeling
Beijing Normal University at Zhuhai Zhuhai, China · Hunan University Changsha, China · Sun Yat-sen University Shenzhen, China
Sequential Correlations Change In-Context Learning: Effective Context Length and Architectural Mismatch
Jul 4, 2026Mary Letey, Yue M. Lu, Cengiz Pehlevan +1In-Context Learning Attention Mechanisms
John A. Paulson School of Engineering and Applied Sciences, Harvard University · Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University · Society of Fellows and Center for Brain Science, Harvard University
Best-of-Better- $N$ : Generating Pre-Aligned Responses with In-Context Learning
Jul 3, 2026Eric Lei, Hsiang Hsu, Chun-Fu ChenLLM Alignment Best-of-N Sampling
JPMorganChase Global Technology Applied Research
In-context learning from self-generated trajectories for adaptive model reduction
Jul 3, 2026Amirpasha Hedayat, Laura Balzano, Karthik DuraisamyIn-Context Learning Reduced-Order Modeling
Department of Aerospace Engineering · Michigan Institute for Computational Discovery and Engineering, University of Michigan, Ann Arbor, MI, USA · Department of Electrical Engineering and Computer Science
Induction Heads Interpolate N-Grams
Jul 2, 2026Francesco D'Angelo, Oguz Kaan Yuksel, Swathi Shree Narashiman +1Transformer Interpretability In-Context Learning
1TML Lab, EPFL, Switzerland. · 2IIT Madras, India. · †Work done during an internship at the TML Lab, EPFL.
Show Me Examples: Inferring Visual Concepts from Image Sets
Jul 2, 2026Nick Stracke, Kolja Bauer, Stefan Andreas Baumann +3Vision-Language Models Conditional Image Generation
Munich Center for Machine Learning · Apple
Dendritic In-Context Learning in a Single-Layer Spiking Neural Network
Jul 2, 2026Juwei Shen, Yujie Wu, Changwen ChenSpiking Neural Networks In-Context Learning
Department of Computing, FCMS The Hong Kong Polytechnic University Hong Kong, China
Do LLMs Truly Generalize in the Molecular Domain? A Perturbation-Based Analysis
Jul 2, 2026Jiatong Li, Weida Wang, Changmeng Zheng +4In-Context Learning Molecular Property Prediction
The Hong Kong Polytechnic University · National University of Singapore · Shanghai AI Lab +1
ICDepth: Taming Video Diffusion Models for Video Depth Estimation via In-Context Conditioning
Jul 2, 2026Xuanhua He, Jiaxin Xie, Mingzhe Zheng +1Video Diffusion Models Depth Estimation
The Hong Kong University of Science and Technology, Hong Kong SAR, China
From Approximation to Emergence: A Theory of Deep Learning
Jul 1, 2026Zhilin ZhaoNeural Network Generalization Deep Learning Optimization
Sun Yat-sen University
Linguistic Relative Policy Optimization for Video Anomaly Reasoning
Jul 1, 2026Jiaxu Leng, Jiankang Zheng, Mengjingcheng Mo +4VLM Adaptation RL for Language Model Reasoning
School of Computer Science and Technology, Chongqing University of Posts and Telecommunications, Chongqing, China · Chongqing College of Artificial Intelligence, Chongqing, China
Retrieved Images as Visual Thought: Training-Free Multimodal In-Context Learning for the Open-vs-Closed Gap
Jul 1, 2026Bingchen Huang, Zhiling Wang, Yifu Chen +1Multimodal RAG Multimodal ICL
Meituan
Ghost in the Kernel: In-Context Learning with Efficient Transformers via Domain Generalization
Jul 1, 2026Peilin Liu, Ding-Xuan ZhouLinear Attention In-Context Learning
School of Mathematics and Statistics, University of Sydney, NSW Australia
Personalized Object Identification and Localization via In-Context Inference with Vision-Language Models
Jul 1, 2026Kensuke Nakamura, Byung-Woo HongFew-Shot Object Detection Vision-Language Grounding
Artificial Intelligence Department, Chung-Ang University, Seoul, 06974, Korea
Criterion-Conditional In-Context Learning: Evaluating Criterion-Shift Adaptation in Vision-Language Models
Jun 30, 2026Kaiyun Yang, Ruilin Yang, Zhimin Yao +2VLM Evaluation Multimodal ICL
University of Science and Technology of China, Hefei, China · Megvii Technology Inc., Beijing, China.
Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
Jun 30, 2026Jiachun Li, David Simchi-LeviCausal Effect Estimation Transformer
Laboratory for Information and Decision Systems, MIT
LC-ICL: Label-Guided Contrastive In-Context Learning for Robust Information Extraction
Jun 28, 2026Xiao You, Tianwei Yan, Shan ZhaoRelation Extraction Named Entity Recognition
1Hefei University of Technology, Hefei, China · 2Chongqing Jiaotong University, Chongqing, China
A Theoretical Interpretation of In-Context Learning via Probabilistic Modeling
Jun 27, 2026Zhenyu Liu, Huaze Tang, Shao-Lun HuangIn-Context Learning Exponential Family
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
In-Context World Modeling for Robotic Control
Jun 24, 2026Siyin Wang, Junhao Shi, Senyu Fei +4ICL for Robotics World Model Learning
1Fudan University · 2Shanghai Innovation Institute · 3Tongji University
The Generalization Spectrum: A Chromatographic Approach to Evaluating Learning Algorithms
Jun 24, 2026Jinghan Zhang, Zerui Cheng, Shiqi Chen +5Neural Network Generalization Reinforcement Learning
1ByteDance Seed · 2Hong Kong University of Science and Technology · 3Princeton University +1

Jul 15, 2026J/K move · Enter open · S save

Supervised Fine-Tuning vs. In-Context Learning: An Equilibrium Analysis of LLM Personalization under Congestion

Fengzhuo Zhang, Zhuoran Yang, Dirk Bergemann

Yale University, 30 Hillhouse Ave., New Haven, CT 06520, USA

Supervised Fine-Tuning LLM Serving LLM Personalization Cost-Aware Inference In-Context Learning

Open paper AskPDF arXiv

Large Language Models (LLMs) have revolutionized AI services, but a critical tension emerges: while personalization improves model performance, it consumes scarce computational resources that users must share. When should a user invest in expensive Supervised Fine-Tuning (SFT) versus lightweight In-Context Learning (ICL)? How does congestion from other users' personalization choices reshape these incentives? And what strategies should platforms adopt when offering multiple personalization algorithms? We develop a tractable framework for LLM serving that captures the statistical-economic trade-offs users face. Our analysis yields several surprising insights. First, we show that ICL and SFT dominate in different regimes, determined by an interplay between pretraining coverage and data signal-to-noise ratios, but congestion can flip these rankings. Second, equilibrium resource consumption exhibits pronounced non-monotonicity: improving pretraining precision reduces the congestion, while broader pretraining coverage and harder tasks sometimes increase it. Third, we prove that offering both personalization methods never hurts the platform's maximal profits, despite potentially increasing computational load. Experiments with GPT-2 on linear regression tasks validate our theoretical predictions about algorithm performance. Complementing these results, our review of documentation from 21 major AI platforms shows that the share offering both SFT and ICL increased from 9.5% in 2021 to 71.4% in 2025, consistent with our platform-design implications.