Context Learning

In-context learning (ICL) is a paradigm shift in machine learning, focusing on enabling models to adapt to new tasks using only a few examples provided within the input, without requiring parameter updates. Current research emphasizes understanding ICL's mechanisms, particularly within transformer-based large language models, and improving its effectiveness through techniques like enhanced example selection, chain-of-thought prompting, and addressing issues such as spurious correlations and copy bias. This research is significant because ICL offers a more efficient and adaptable approach to many machine learning problems, impacting fields ranging from natural language processing and computer vision to scientific computing and beyond.

Papers

August 26, 2024

Focused Large Language Models are Stable Many-Shot Learners
Peiwen Yuan, Shaoxiong Feng, Yiwei Li, Xinglin Wang, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Heda Wang, Yao Hu, Kan Li
Large Language Model Context Learning Training Free Task Adaptation Stable Learning

August 23, 2024

August 22, 2024

August 21, 2024

Memorization in In-Context Learning
Shahriar Golchin, Mihai Surdeanu, Steven Bethard, Eduardo Blanco, Ellen Riloff
Large Language Model Training Data Context Learning Zero Shot Learning Limited Memorization

August 20, 2024

Revisiting VerilogEval: Newer LLMs, In-Context Learning, and Specification-to-RTL Tasks
Nathaniel Pinckney, Christopher Batten, Mingjie Liu, Haoxing Ren, Brucek Khailany
Large Language Model Context Learning Hardware Description Language Register Transfer Level

August 19, 2024

August 18, 2024

Out-of-distribution generalization via composition: a lens through induction heads in Transformers
Jiajun Song, Zhuoyan Xu, Yiqiao Zhong
Large Language Model Strong Generalization Transformer Megatron Decepticons Context Learning Camera Lens Distribution Generalization Compositional Ability Joint Representation Head Transformer

August 16, 2024

Adaptive Guardrails For Large Language Models via Trust Modeling and In-Context Learning
Jinwei Hu, Yi Dong, Xiaowei Huang
Context Learning Guardrail Model Human Trust Trust Model Intelligent Trust Cloud Management Method

August 14, 2024

August 12, 2024

Towards Autonomous Agents: Adaptive-planning, Reasoning, and Acting in Language Models
Yen-Che Hsiao, Abhishek Dutta
Language Model Context Learning Complex Reasoning Autonomous Agent Language Agent Adaptive Planning Autonomous Language Agent Many Autonomous Intelligent Agent

August 10, 2024

LaiDA: Linguistics-aware In-context Learning with Data Augmentation for Metaphor Components Identification
Hongde Liu, Chenyuan He, Feiyang Meng, Changyong Niu, Yuxiang Jia
Large Language Model Data Augmentation Context Learning Shared Task Metaphor Detection

August 9, 2024

SCOI: Syntax-augmented Coverage-based In-context Example Selection for Machine Translation
Chenming Tang, Zhixiang Wang, Yunfang Wu
Context Learning Machine Translation Context Example Syntactic Representation

August 8, 2024

August 6, 2024

Pre-training and in-context learning IS Bayesian inference a la De Finetti
Naimeng Ye, Hanming Yang, Andrew Siah, Hongseok Namkoong
Context Learning Bayesian Inference Sequence to Sequence Posterior Inference Sequence Modeling Sequence Model De Finetti

Context Learning

Papers

Focused Large Language Models are Stable Many-Shot Learners

In-Context Learning with Reinforcement Learning for Incomplete Utterance Rewriting

Multimodal Contrastive In-Context Learning

Transformers are Minimax Optimal Nonparametric In-Context Learners

Evidence-backed Fact Checking using RAG and Few-Shot In-Context Learning with LLMs

Memorization in In-Context Learning

Revisiting VerilogEval: Newer LLMs, In-Context Learning, and Specification-to-RTL Tasks

In-Context Learning with Representations: Contextual Generalization of Trained Transformers

Strategic Demonstration Selection for Improved Fairness in LLM In-Context Learning

Out-of-distribution generalization via composition: a lens through induction heads in Transformers

Adaptive Guardrails For Large Language Models via Trust Modeling and In-Context Learning

Fast Training Dataset Attribution via In-Context Learning

Cropper: Vision-Language Model for Image Cropping through In-Context Learning

Large Language Models Know What Makes Exemplary Contexts

Towards Autonomous Agents: Adaptive-planning, Reasoning, and Acting in Language Models

LaiDA: Linguistics-aware In-context Learning with Data Augmentation for Metaphor Components Identification

SCOI: Syntax-augmented Coverage-based In-context Example Selection for Machine Translation

How Transformers Utilize Multi-Head Attention in In-Context Learning? A Case Study on Sparse Linear Regression

Enhancing Robustness of Retrieval-Augmented Language Models with In-Context Learning

Pre-training and in-context learning IS Bayesian inference a la De Finetti