Context Learning

In-context learning (ICL) is a paradigm shift in machine learning, focusing on enabling models to adapt to new tasks using only a few examples provided within the input, without requiring parameter updates. Current research emphasizes understanding ICL's mechanisms, particularly within transformer-based large language models, and improving its effectiveness through techniques like enhanced example selection, chain-of-thought prompting, and addressing issues such as spurious correlations and copy bias. This research is significant because ICL offers a more efficient and adaptable approach to many machine learning problems, impacting fields ranging from natural language processing and computer vision to scientific computing and beyond.

Papers

June 11, 2024

June 10, 2024

In-Context Learning and Fine-Tuning GPT for Argument Mining
Jérémie Cabessa, Hugo Hernault, Umer Mushtaq
Context Learning Argument Classification

June 7, 2024

BERTs are Generative In-Context Learners
David Samuel
Language Model Context Learning Causal Language

June 6, 2024

June 5, 2024

June 3, 2024

June 2, 2024

Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective
Fabian Falck, Ziyu Wang, Chris Holmes
Context Learning Bayesian Inference Martingale Difference

May 31, 2024

May 30, 2024

Context Learning

Papers

Efficiently Exploring Large Language Models for Document-Level Machine Translation with In-context Learning

AIM: Let Any Multi-modal Large Language Models Embrace Efficient In-Context Learning

In-Context Learning and Fine-Tuning GPT for Argument Mining

BERTs are Generative In-Context Learners

Exploring the Zero-Shot Capabilities of Vision-Language Models for Improving Gaze Following

FastGAS: Fast Graph-based Annotation Selection for In-Context Learning

Improving In-Context Learning with Prediction Feedback for Sentiment Analysis

Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers

Learning to grok: Emergence of in-context learning and skill composition in modular arithmetic tasks

E-ICL: Enhancing Fine-Grained Emotion Recognition through the Lens of Prototype Theory

Eliciting the Priors of Large Language Models using Iterated In-Context Learning

In-Context Learning of Physical Properties: Few-Shot Adaptation to Out-of-Distribution Molecular Graphs

Universal In-Context Approximation By Prompting Fully Recurrent Models

Demonstration Augmentation for Zero-shot In-context Learning

Selectively Answering Visual Questions

Is In-Context Learning in Large Language Models Bayesian? A Martingale Perspective

From Unstructured Data to In-Context Learning: Exploring What Tasks Can Be Learned and When

In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought

Is In-Context Learning Sufficient for Instruction Following in LLMs?

Why Larger Language Models Do In-context Learning Differently?