Context Learning

In-context learning (ICL) is a paradigm shift in machine learning, focusing on enabling models to adapt to new tasks using only a few examples provided within the input, without requiring parameter updates. Current research emphasizes understanding ICL's mechanisms, particularly within transformer-based large language models, and improving its effectiveness through techniques like enhanced example selection, chain-of-thought prompting, and addressing issues such as spurious correlations and copy bias. This research is significant because ICL offers a more efficient and adaptable approach to many machine learning problems, impacting fields ranging from natural language processing and computer vision to scientific computing and beyond.

Papers

October 12, 2023

October 10, 2023

October 8, 2023

October 7, 2023

The Cost of Down-Scaling Language Models: Fact Recall Deteriorates before In-Context Learning
Tian Jin, Nolan Clement, Xin Dong, Vaishnavh Nagarajan, Michael Carbin, Jonathan Ragan-Kelley, Gintare Karolina Dziugaite
Large Language Model Context Learning Context Example Hidden CoST Strong Scaling Scaling Method

October 5, 2023

October 4, 2023

October 1, 2023

Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning
Mustafa Shukor, Alexandre Rame, Corentin Dancette, Matthieu Cord
Context Learning Large Multimodal Model Task Performance Multimodal in Context Learning

September 30, 2023

September 29, 2023

Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering
Han Zhou, Xingchen Wan, Lev Proleev, Diana Mincu, Jilin Chen, Katherine Heller, Subhrajit Roy
Large Language Model Context Learning Language Understanding Prompt Engineering Contextual Biasing Global Calibration Experimental Batch Correction Method

September 27, 2023

Context Learning

Papers

Not All Demonstration Examples are Equally Beneficial: Reweighting Demonstration Examples for In-Context Learning

Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning Ability

SEER : A Knapsack approach to Exemplar Selection for In-Context HybridQA

Jailbreak and Guard Aligned Language Models with Only Few In-Context Demonstrations

In-Context Convergence of Transformers

Lightweight In-Context Tuning for Multimodal Unified Models

Guideline Learning for In-context Information Extraction

The Cost of Down-Scaling Language Models: Fact Recall Deteriorates before In-Context Learning

Procedural Text Mining with Large Language Models

Fine-tune Language Models to Approximate Unbiased In-context Learning

Understanding In-Context Learning in Transformers and LLMs by Learning to Learn Discrete Functions

DQ-LoRe: Dual Queries with Low Rank Approximation Re-ranking for In-Context Learning

Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning

Dynamic Demonstrations Controller for In-Context Learning

Decoding In-Context Learning: Neuroscience-inspired Analysis of Representations in Large Language Models

Towards LLM-based Fact Verification on News Claims with a Hierarchical Step-by-Step Prompting Method

Understanding In-Context Learning from Repetitions

Batch Calibration: Rethinking Calibration for In-Context Learning and Prompt Engineering

MKRAG: Medical Knowledge Retrieval Augmented Generation for Medical Question Answering

Generative Speech Recognition Error Correction with Large Language Models and Task-Activating Prompting