Context Learning

In-context learning (ICL) is a paradigm shift in machine learning, focusing on enabling models to adapt to new tasks using only a few examples provided within the input, without requiring parameter updates. Current research emphasizes understanding ICL's mechanisms, particularly within transformer-based large language models, and improving its effectiveness through techniques like enhanced example selection, chain-of-thought prompting, and addressing issues such as spurious correlations and copy bias. This research is significant because ICL offers a more efficient and adaptable approach to many machine learning problems, impacting fields ranging from natural language processing and computer vision to scientific computing and beyond.

Papers

March 14, 2023

March 9, 2023

ICL-D3IE: In-Context Learning with Diverse Demonstrations Updating for Document Information Extraction
Jiabang He, Lei Wang, Yi Hu, Ning Liu, Hui Liu, Xing Xu, Heng Tao Shen
Large Language Model Natural Language Processing Context Learning Noisy Demonstration Different Context Document Level Information Extraction

March 7, 2023

Larger language models do in-context learning differently
Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, Tengyu Ma
Context Learning Context Example Semantic Label Larger Language Model Semantic Prior

March 6, 2023

OpenICL: An Open-Source Framework for In-context Learning
Zhenyu Wu, YaoXiang Wang, Jiacheng Ye, Jiangtao Feng, Jingjing Xu, Yu Qiao, Zhiyong Wu
Large Language Model Context Learning NLP Task Semantic Parsing

February 27, 2023

Finding Support Examples for In-Context Learning
Xiaonan Li, Xipeng Qiu
Language Model Context Learning Context Example Support Set

February 22, 2023

How Does In-Context Learning Help Prompt Tuning?
Simeng Sun, Yang Liu, Dan Iter, Chenguang Zhu, Mohit Iyyer
Context Learning Prompt Tuning Prompt Embeddings Parameter Efficient Adaptation Instruction Aware Prompt Tuning

February 21, 2023

In-context Example Selection with Influences
Tai Nguyen, Eric Wong
Context Learning External Influence Context Example Negative Example Exemplar Selection

February 14, 2023

ScatterShot: Interactive In-context Example Curation for Text Transformation
Tongshuang Wu, Hua Shen, Daniel S. Weld, Jeffrey Heer, Marco Tulio Ribeiro
Context Learning Context Example Text Transformation Attribute Pattern Reduce SCATTER

February 13, 2023

Distinguishability Calibration to In-Context Learning
Hongjing Li, Hanqi Yan, Yanran Li, Li Qian, Yulan He, Lin Gui
Fine Grained Context Learning Prompt Based Learning Output Embeddings Fine Grained Token Leave One Out Distinguishability

February 12, 2023

Stabilized In-Context Learning with Pre-trained Language Models for Few Shot Dialogue State Tracking
Derek Chen, Kun Qian, Zhou Yu
Pre Trained Language Model Context Learning Context Example Dialogue State Tracking Word Length

February 11, 2023

Compositional Exemplars for In-context Learning
Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu, Lingpeng Kong
Context Learning Context Example Compositional Structure Efficient in Context Learning

February 10, 2023

Context Understanding in Computer Vision: A Survey
Xuan Wang, Zhigang Zhu
Timely Survey Computer Vision Context Learning Contextual Information Contextual Understanding Appearance Information

February 9, 2023

In-Context Learning with Many Demonstration Examples
Mukai Li, Shansan Gong, Jiangtao Feng, Yiheng Xu, Jun Zhang, Zhiyong Wu, Lingpeng Kong
Context Learning Pre Trained Large Language Model Efficient Transformer Transformer Backbone Long Range Language Demonstration Example

February 3, 2023

Towards Few-Shot Identification of Morality Frames using In-Context Learning
Shamik Roy, Nishanth Sridhar Nakshatri, Dan Goldwasser
Context Learning NLP Field Person Identification Psycholinguistic Research Shot in Context Learning Moral Value Moral Theory

February 2, 2023

Using In-Context Learning to Improve Dialogue Safety
Nicholas Meade, Spandana Gella, Devamanyu Hazarika, Prakhar Gupta, Di Jin, Siva Reddy, Yang Liu, Dilek Hakkani-Tür
Context Learning Dialogue Agent Conversational Large Language Model Dialogue Safety

January 31, 2023

What Makes Good Examples for Visual In-Context Learning?
Yuanhan Zhang, Kaiyang Zhou, Ziwei Liu
Context Learning Context Example Large Vision Model Visual in Context Learning

January 27, 2023

Large Language Models Are Latent Variable Models: Explaining and Finding Good Demonstrations for In-Context Learning
Xinyi Wang, Wanrong Zhu, Michael Saxon, Mark Steyvers, William Yang Wang
Context Learning Shot Learning Latent Variable Model Pre Trained Large Language Model Effective Demonstration

January 20, 2023

Transforming Unstructured Text into Data with Context Rule Assisted Machine Learning (CRAML)
Stephen Meisenbacher, Peter Norlander
Context Learning Raw Data Supervised Learning Unstructured Text Text Classification Model

January 17, 2023

Transformers as Algorithms: Generalization and Stability in In-context Learning
Yingcong Li, M. Emrullah Ildiz, Dimitris Papailiopoulos, Samet Oymak
Strong Generalization Transformer Megatron Decepticons Practical Algorithm Context Learning Core Stability Generalization Bound Multitask Learning Attention Based Architecture Online Inference