In-Context Learning

Also known as ICL

In-context learning (ICL) studies how a model can infer a task, rule, or preference from information in its prompt and apply it to new inputs, making adaptation possible without task-specific weight updates. Common approaches include few-shot demonstrations, retrieval and ordering strategies, synthetic-task pretraining for prior-fitted models, and test-time optimization of prompts or memory; evaluations range from synthetic regression and Markov tasks to language, vision, time-series, graph, and robotics benchmarks. Theory and mechanistic studies examine what algorithms attention and recurrent models implement, and how context length, training-task diversity, and architecture affect generalization. A central tension is that models may encode contextual information yet fail to use it reliably: performance can be sensitive to example quality, order, noise, and conflict with pretrained priors, while longer contexts increase inference costs.

Latest papers

404 papers64 papers in the last four weeks, up 52% on the four weeks before. 0.4% of all new papers.

RAS: Reflection-Augmented Scaling with In-Context Learning for Executable Cypher Query Generation
May 21, 2026Minseok Jung, Abhas Ricky, Muhammad Rameez ChatniExecution-Guided Code Generation Inference-Time Scaling
Cloudera 6220 America Center Dr. San Jose, CA 95002
From Residuals to Reasons: LLM-Guided Mechanism Inference from Tabular Data
May 21, 2026Mohammad R. Rezaei, Rahul G. KrishnanResidual Learning In-Context Learning
Department of Computer Science University of Toronto Vector Institute Toronto, ON, Canada
Point Cloud Sequence Encoding for Material-conditioned Graph Network Simulators
May 20, 2026Philipp Dahlinger, Balázs Gyenes, Niklas Freymuth +6Sim-to-Real Transfer Point Cloud Learning
1Autonomous Learning Robots · 2Methods for Big Data · Institute of Vehicle System Technology
Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning
May 20, 2026Jihoon Kwon, Jiwon Choi, Jy-yong SohnTask Vectors Distribution Matching
Seoul National University · Yonsei University
Do as I Say, Not as I Do: Instruction-Induction Conflict in LLMs
May 19, 2026Carolina Camassa, Derek ShillerLLM Evaluation Language Model Self-Assessment
Future Impact Group · Rethink Priorities
Easier to Judge than to Find: Predicting In-Context Learning Success for Demonstration Selection
May 18, 2026Haochun Wang, Chaofen Yang, Jiatong Liu +5LLM Inference Efficiency In-Context Example Selection
Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology, China.
Geometry-Aware Uncertainty Coresets for Robust Visual In-Context Learning in Histopathology
May 18, 2026Franciskus Xaverius Erick, Johanna Paula Müller, Bernhard KainzVLM Robustness In-Context Learning
FAU Erlangen-Nürnberg, Erlangen, DE · Department of Computing, Imperial College London, London, UK
Temporal Task Diversity: Inductive Biases Under Non-Stationarity in Synthetic Sequence Modelling
May 18, 2026Afiq Abdillah Effiezal Aswadi, Oliver Britton, Ross Baker +1Neural Network Generalization In-Context Learning
1Independent · University of Oxford
Context Memorization for Efficient Long Context Generation
May 18, 2026Yasuyuki Okoshi, Hao Mark Chen, Guanxi Lu +3Memory-Augmented Language Models Long-Context Language Model Inference
Institute of Science Tokyo, Japan · Imperial College London, UK
Large language models reorganize representational geometry during in-context learning
May 16, 2026Hua-Dong Xiong, Li Ji-An, Robert C. Wilson +2Neural Representation Geometry Representation Disentanglement
School of Psychological and Brain Sciences, Georgia Tech · Department of Psychology, New York University · School of Psychological and Brain Sciences, Georgia Tech 3 Center of Excellence for Computational Cognition, Georgia Tech +2
ACIL: Auto Chain of Thoughts for In-Context Learning
May 16, 2026Rui ChuLLM Prompting CoT Reasoning
In-context learning enables continental-scale subsurface temperature prediction from sparse local observations
May 15, 2026Daniel O'Malley, Christopher W. Johnson, Javier E. Santos +10In-Context Learning
Los Alamos National Laboratory
How Few-Shot Examples Add Up: A Causal Decomposition of Function Vectors in In-Context Learning
May 15, 2026Entang Wang, Yiwei Wang, Aleksandra Bakalova +1Transformer Interpretability Task Vectors
Saarland Informatics Campus, Saarland Univesity, Saarbrücken, Germany.
KGPFN: Unlocking the Potential of Knowledge Graph Foundation Model via In-Context Learning
May 14, 2026Yisen Gao, Jiaxin Bai, Haoyu Huang +5Graph Neural Networks In-Context Learning
Department of Computer Science and Engineering, HKUST, Hong Kong, China · Department of Computer Science and Engineering, HKBU, Hong Kong, China
In-Context Learning for Data-Driven Censored Inventory Control
May 14, 2026Sohom Mukherjee, Anh-Duy Pham, Richard Pibernik +1Inventory Control Bandit Convex Optimization
Julius-Maximilians-Universität Würzburg · National University of Singapore Zaragoza Logistics Center
Many-Shot CoT-ICL: Making In-Context Learning Truly Learn
May 13, 2026Tsz Ting Chung, Lemao Liu, Mo Yu +1CoT Reasoning In-Context Learning
Hong Kong University of Science and Technology · Fudan University · Wechat AI, Tencent.
IV-ICL: Bounding Causal Effects with Instrumental Variables via In-Context Learning
May 13, 2026Vahid Balazadeh, Hamidreza Kamkari, Medha Barath +2Causal Effect Estimation Parameter Identifiability
University of Toronto · Vector Institute · MIT CSAIL +1
VIP-COP: Context Optimization for Tabular Foundation Models
May 13, 2026Yilong Chen, Xueying Ding, Leman AkogluIn-Context Example Selection Tabular Foundation Models
Carnegie Mellon University
In-Context Learning Operates as Concept Subspace Learning
May 12, 2026Wei Tang, Xinyan Jiang, Fakhri Karray +1Representation Learning Mechanistic Interpretability
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · Shanghai Advanced Research Institute, Chinese Academy of Science
Learning, Fast and Slow: Towards LLMs That Adapt Continually
May 12, 2026Rishabh Tiwari, Kusha Sareen, Lakshya A Agrawal +6Continual Learning for LLMs RL for Language Model Reasoning
UC Berkeley · Eragon · Mila +3
Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space
May 12, 2026Eric Bigelow, Raphaël Sarfati, Daniel Wurgaft +5LLM Interpretability In-Context Learning
In-context learning to predict critical transitions in dynamical systems
May 12, 2026Yunus Sevinchan, Juan Nathaniel, Kai Ueltzhöffer +8Dynamical Systems Early Warning Detection
kausable Heidelberg, Germany · Columbia University New York, USA · University of Edinburgh Scotland, UK
Instruct-ICL: Instruction-Guided In-Context Learning for Post-Disaster Damage Assessment
May 12, 2026Armin Zarbaft, Ehsan Karimi, Nhut Le +1Disaster Damage Assessment LLM Prompting
Lehigh Uniersity Pennsylvania, USA
Self-Attention as a Covariance Readout: A Unified View of In-Context Learning and Repetition
May 11, 2026Haoren Xu, Guanhua FangSoftmax Attention Repetition in Language Generation
Fudan University
One for All: A Non-Linear Transformer can Enable Cross-Domain Generalization for In-Context Reinforcement Learning
May 10, 2026Bowen He, Juncheng Dong, Lin Lin +1Reinforcement Learning Transformer
Duke University, Durham, U.S.A.
Large Language Models for Sequential Decision-Making: Improving In-Context Learning via Supervised Fine-Tuning
May 9, 2026Minmin Zhang, Sina Aghaei, Soroush SaghafianSupervised Fine-Tuning Sequential Decision Making
Harvard University
PromptDx: Differentiable Prompt Tuning for Multimodal In-Context Alzheimer's Diagnosis
May 9, 2026Lujia Zhong, Yihao Xia, Shuo Huang +2Multimodal ICL Medical Diagnosis
Stevens Neuroimaging and Informatics Institute, Keck School of Medicine, University of Southern California · Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California · Alfred E. Mann Department of Biomedical Engineering, Viterbi School of Engineering, University of Southern California
Transformers Can Implement Preconditioned Richardson Iteration for In-Context Gaussian Kernel Regression
May 8, 2026Mingsong Yan, Dongyang Li, Charles Kulick +1Softmax Attention Transformer Interpretability
Department of Mathematics, University of California, Santa Barbara, CA
Belief or Circuitry? Causal Evidence for In-Context Graph Learning
May 8, 2026Katharine Kowalyshyn, Timothy Duggan, Daniel Little +1Transformer Interpretability LLM Reasoning with Graphs
Department of Computer Science, Tufts University, Medford, United States
Context-Gated Associative Retrieval: From Theory to Transformers
May 8, 2026Moulik Choraria, Argyrios Gerogiannis, Vidhata Jayaraman +2Transformer Interpretability In-Context Learning
UIUC · University of Minnesota, Twin Cities · Stony Brook University

May 21, 2026J/K move · Enter open · S save

RAS: Reflection-Augmented Scaling with In-Context Learning for Executable Cypher Query Generation

Minseok Jung, Abhas Ricky, Muhammad Rameez Chatni

Cloudera 6220 America Center Dr. San Jose, CA 95002

Execution-Guided Code Generation Inference-Time Scaling Text-to-Cypher In-Context Learning Text Generation

Open paper AskPDF arXiv

Inference-time scaling can reduce errors in structured query generation, but methods to allocate the compute for query code generation remains underexplored. We study Text2Cypher, where language models generate Cypher queries that execute against property graph databases. Non-executable queries constitute a distinct syntactic failure separate from semantic inaccuracy: a syntax error triggers a system-generated error message from the database. These error messages are typically discarded at inference time rather than leveraged through in-context learning (ICL). We compare two inference methods: Independent Scaling (IS), which performs memoryless resampling, and Reflection-Augmented Scaling (RAS), which conditions each new attempt on prior execution feedback via ICL. Across three Neo4j datasets and five code-specialized language models, RAS reduces the Query Execution Error Rate by 41--50% at n{=}5, outperforming IS at 32--38%. Execution errors are not merely failures to discard but actionable feedback, and structuring inference-time compute around them is a more efficient path to executability than scaling independent samples.