In-Context Learning

Also known as ICL

In-context learning (ICL) studies how a model can infer a task, rule, or preference from information in its prompt and apply it to new inputs, making adaptation possible without task-specific weight updates. Common approaches include few-shot demonstrations, retrieval and ordering strategies, synthetic-task pretraining for prior-fitted models, and test-time optimization of prompts or memory; evaluations range from synthetic regression and Markov tasks to language, vision, time-series, graph, and robotics benchmarks. Theory and mechanistic studies examine what algorithms attention and recurrent models implement, and how context length, training-task diversity, and architecture affect generalization. A central tension is that models may encode contextual information yet fail to use it reliably: performance can be sensitive to example quality, order, noise, and conflict with pretrained priors, while longer contexts increase inference costs.

Latest papers

404 papers64 papers in the last four weeks, up 52% on the four weeks before. 0.4% of all new papers.

Reinforcement Learning Elicits Contextual Learning of Unseen Language Translation
Jun 4, 2026Hanxu Hu, Zdeněk Šnajdr, Pinzhen Chen +2RL for Language Models In-Context Learning
University of Zurich · ETH Zurich · Queen’s University Belfast
LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agents
Jun 4, 2026Aofan Yu, Chenyu Zhou, Tianyi Xu +8LLM Agent Skill Learning In-Context Learning
Shanghai Jiao Tong University · Sun Yat-Sen University · Shanghai Innovation Institute +1
TS-ICL: A Flexible Time-Indexed Foundation Model for Time Series via In-Context Learning
Jun 4, 2026Etienne Le Naour, Tahar Nabil, Adrien PetraliaIrregular Time-Series Modeling Time Series Imputation
1EDF R&D, Palaiseau, France
Would you still call this Dax? Novel Visual References in VLMs and Humans
Jun 3, 2026Ada Defne Tür, Gaurav Kamath, Joyce Chai +2VLM Evaluation Vision-Language Models
McGill University · Mila Quebec AI Institute · University of Michigan - Ann Arbor +1
Activation-Based Active Learning for In-Context Learning: Challenges and Insights
Jun 3, 2026Yaseen M. Osman, Geoff V. Merrett, Stuart E. MiddletonIn-Context Example Selection Active Learning
School of Electronics and Computer Science, University of Southampton, United Kingdom
Fast & Faithful Function Vectors
Jun 3, 2026Minh An Pham, Anton Segeler, Thomas Wiegand +4Task Vectors LLM Interpretability
Fraunhofer Heinrich-Hertz-Institute, Berlin, Germany · Technische Universität Berlin, Berlin, Germany · BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany +1
Hyper-ICL: Attention Calibration with Hyperbolic Anchor Distillation for Multimodal In-Context Learning
Jun 3, 2026Niloufar Alipour Talemi, Hossein Kashiani, Fatemeh AfghahEfficient Multimodal Inference Multimodal ICL
Holcombe Department of Electrical and Computer Engineering, Clemson University, Clemson, United States.
OpenRFM: Dissecting Relational In-Context Learning
Jun 3, 2026Zhikai Chen, Junyu Yin, Jialiang Gu +5Relational Foundation Models Synthetic Data Pretraining
Michigan State University · George Mason University · Georgia Institute of Technology +1
An Asymptotic Theory of Chain-of-Thought in In-Context Learning
Jun 2, 2026Kaito Takanami, Cengiz PehlevanNeural Network Generalization CoT Reasoning
Department of Physics, Graduate School of Science, The University of Tokyo Tokyo, Japan · John A. Paulson School of Engineering and Applied Sciences, Harvard University Cambridge, MA, USA · Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University Cambridge, MA, USA +1
RA-LWLM: Retrieval-Augmented In-Context Localization with Wireless Foundation Models
Jun 1, 2026Guangjin Pan, Hui Chen, Hei Victor Cheng +1Wireless Communications In-Context Learning
Department of Electrical Engineering, Chalmers University of Technology, 41296 Gothenburg, Sweden · Department of Electrical and Computer Engineering, Aarhus University, Denmark
Med-HEAL: Analyzing and Mitigating Hallucinations in Medical LLMs with Hallucination-Aware In-Context Learning
May 31, 2026Yiming Liao, Zeno Franco, Jose Eduardo Lizarraga Mazaba +1LLM Self-Correction LLM Hallucination Mitigation
University of Maryland, Baltimore County Baltimore, Maryland, USA · Concordia University Wisconsin · Medical College of Wisconsin Milwaukee, Wisconsin USA
Images as Tables: In-Context Learning with TabPFN for Low-Data Detection of AI-Generated Images
May 30, 2026Jan Philip Walter, Shashank Agnihotri, Margret KeuperIn-Context Learning AI-Generated Image Detection
Machine Learning Group, University of Mannheim, Mannheim, Germany · Max-Planck-Institute for Informatics, Saarland Informatics Campus, Germany
Finite Certificates for In-Context Determinacy and a Threshold Theory of Emergence in Language Models
May 30, 2026Faruk Alpay, Hamdi AlakkadEmergent Abilities in Language Models LLM Evaluation
Department of Computer Engineering Bahcesehir University, Istanbul, Turkey · Department of Artificial Intelligence Engineering Bahcesehir University, Istanbul, Turkey
FOCUS: Forcing In-Context Object Localization through Visual Support Constraints and Policy Optimization
May 29, 2026Mohammed Asad Karim, Vinay Kumar VermaVision-Language Grounding In-Context Learning
Amazon, Seattle, USA
In-Context Reward Adaptation for Robust Preference Modeling
May 28, 2026Zhenyu Sun, Zheng Xu, Ermin WeiHuman Preference Modeling In-Context Learning
Northwestern University · Meta Superintelligence Labs
Attention as In-Context Empirical Bayes: A Two-Stage View via Particle Dynamics
May 28, 2026Matthew Smart, Soumya Ganguly, Nilava Metya +2Self-Attention Empirical Bayes
Lewis-Sigler Institute for Integrative Genomics, Princeton University, Princeton, NJ, USA · Department of Mathematics, Rutgers University, Piscataway, NJ, USA · Department of Physics and Astronomy, Rutgers University, Piscataway, NJ, USA +1
Understanding Generalization and Forgetting in In-Context Continual Learning
May 27, 2026Guangyu Li, Meng Ding, Lijie HuContinual Learning for LLMs In-Context Learning
1Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · University of Buffalo
Explaining is Harder Than Predicting Alone: Evaluating Concept-based Explanations of MLLMs as ICL Visual Classifiers
May 27, 2026Carmen Quiles-Ramírez, Leticia L. Rodríguez, Nicolás Martorell +1VLM Evaluation Multimodal Large Language Models
Universidad de Granada, Spain · Universidad de Buenos Aires, Argentina
Reflective Dialogue between Teacher and Solver Agents for Video Question Answering
May 27, 2026Takuya Murakawa, Toru TamakiVLM Adaptation Video QA
Nagoya Institute of Technology, Nagoya, Japan
Fine-Tuning Dynamics of In-Context Factual Recall in Transformers
May 26, 2026Ruomin Huang, Eshaan Nichani, Jason D. Lee +1Supervised Fine-Tuning Transformer Attention
Duke University · Princeton University · UC Berkeley
FSA-GRPO: Teaching Auditory LLMs to Use Few-shot Demonstrations
May 26, 2026Haolong Zheng, Siyin Wang, Xulin Fan +2RL for Language Models Few-Shot Domain Adaptation
University of Illinois Urbana Champaign · Tsinghua University
ICICLE: Expanding Retrieval with In-Context Documents
May 26, 2026Yu-Chen Den, Yung-Yu Shih, Zhi Rui Tam +4In-Context Retrieval Generative Retrieval
1National Taiwan University, Taipei, Taiwan · 2Johns Hopkins University, Baltimore, US
ContextGuard: Structured Self-Auditing for Context Learning in Language Models
May 26, 2026Hongbo Jin, Chi Wang, Haoran Tang +5LLM Auditing Instruction Following
Peking University · SCUT · Tsinghua University
Transformers Can Learn Posterior Predictive Distributions In-Context
May 26, 2026Gyeonghun Kang, Changwoo J. Lee, Xiang ChengBayesian Inference Transformer Attention
Department of Statistical Science, Duke University, Durham, NC, USA · Department of Electrical and Computer Engineering, Duke University, Durham, NC, USA
In-Context Optimization for Retrieval-Augmented Generation: A Gradient-Descent Perspective
May 25, 2026Mingchen Li, Jiatan Huang, Chuxu Zhang +2Retrieval-Augmented Generation Test-Time Adaptation
University of Massachusetts, Amherst · University of Connecticut · Emory University
When Correct Demonstrations Hurt: Rethinking the Role of Exemplars in In-Context Learning
May 25, 2026Chenghao Qiu, Chunli Peng, Yufeng Yang +2In-Context Learning
Texas A&M University
Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning
May 25, 2026Han-yu WangTransformer Interpretability Task Vectors
The University of Hong Kong
Context-CoT: Enhancing Context Learning via High-Quality Reasoning Synthesis
May 25, 2026Hongbo Jin, Mingnan Zhu, Jingqi Tian +6CoT Reasoning In-Context Learning
Peking University · Xiamen University · Tsinghua University
Advancing Graph Few-Shot Learning via In-Context Learning
May 23, 2026Renchu Guan, Yajun Wang, Chunli Guo +5Meta-Learning Few-Shot Learning
Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education
Self-Improving In-Context Learning
May 22, 2026Baturay Saglam, Dionysis KalogeriasLanguage Model Calibration In-Context Learning
Department of Electrical and Computer Engineering Yale University

Jun 4, 2026J/K move · Enter open · S save

Reinforcement Learning Elicits Contextual Learning of Unseen Language Translation

Hanxu Hu, Zdeněk Šnajdr, Pinzhen Chen, Jannis Vamvas+1

University of Zurich · ETH Zurich · Queen’s University Belfast

RL for Language Models In-Context Learning Low-Resource MT Machine Translation Cross-Lingual Transfer

Open paper AskPDF arXiv

Prior work has shown that large language models (LLMs) can translate unseen or low-resource languages by undergoing continued training or even by encoding a grammar book in their context. However, both methods typically overfit specific languages, with limited zero-shot transfer at test time. To translate extremely low-resource languages at scale, we argue that LLMs must acquire the meta-skill of utilizing in-context linguistic knowledge rather than memorizing specific languages. In this paper, we propose a reinforcement learning (RL) approach to unseen language translation given rich linguistic context, using a surface-level translation metric (chrF) as the reward. Empirically, despite the lightweight reward, our RL-trained models effectively extract and apply relevant linguistic information from the provided context, leading to better translations on completely unseen languages than in-context learning or supervised fine-tuning. Our analyses suggest that outcome-based RL can extend beyond conventional reasoning tasks like math and coding to serve as a recipe for language learning from context.