Context Learning

In-context learning (ICL) is a paradigm shift in machine learning, focusing on enabling models to adapt to new tasks using only a few examples provided within the input, without requiring parameter updates. Current research emphasizes understanding ICL's mechanisms, particularly within transformer-based large language models, and improving its effectiveness through techniques like enhanced example selection, chain-of-thought prompting, and addressing issues such as spurious correlations and copy bias. This research is significant because ICL offers a more efficient and adaptable approach to many machine learning problems, impacting fields ranging from natural language processing and computer vision to scientific computing and beyond.

Papers

October 6, 2024

October 3, 2024

October 2, 2024

October 1, 2024

Exploring the Learning Capabilities of Language Models using LEVERWORLDS
Eitan Wagner, Amir Feder, Omri Abend
Language Model Context Learning Transformer Language Model Modern Language Model Generative Learning

September 30, 2024

TaskComplexity: A Dataset for Task Complexity Classification with In-Context Learning, FLAN-T5 and GPT-4o Benchmarks
Areeg Fahad Rasheed, M. Zarkoosh, Safa F. Abbas, Sana Sabah Al-Azzawi
Data Set Context Learning GPT 4 Novel Dataset Problem Solving Programming Task Task Complexity Task Recognition

Context Learning

Papers

Transformers learn variable-order Markov chains in-context

Task Diversity Shortens the ICL Plateau

Deeper Insights Without Updates: The Power of In-Context Learning Over Fine-Tuning

Multimodal 3D Fusion and In-Situ Learning for Spatially Aware AI

GAMformer: In-Context Learning for Generalized Additive Models

Revisiting In-context Learning Inference Circuit in Large Language Models

Enhanced Transformer architecture for in-context learning of dynamical systems

In-context Learning in Presence of Spurious Correlations

RIPPLECOT: Amplifying Ripple Effect of Knowledge Editing in Language Models via Chain-of-Thought In-Context Learning

Calibrate to Discriminate: Improve In-Context Learning with Label-Free Comparative Inference

GraphIC: A Graph-Based In-Context Example Retrieval Model for Multi-Step Reasoning

Bayes' Power for Explaining In-Context Learning Generalizations

In-Context Transfer Learning: Demonstration Synthesis by Transferring Similar Tasks

Disentangling Latent Shifts of In-Context Learning Through Self-Training

Mitigating Copy Bias in In-Context Learning through Neuron Pruning

Sparse Autoencoders Reveal Temporal Difference Learning in Large Language Models

Transformers Handle Endogeneity in In-Context Linear Regression

BordIRlines: A Dataset for Evaluating Cross-lingual Retrieval-Augmented Generation

Exploring the Learning Capabilities of Language Models using LEVERWORLDS

TaskComplexity: A Dataset for Task Complexity Classification with In-Context Learning, FLAN-T5 and GPT-4o Benchmarks