Context Learning

In-context learning (ICL) is a paradigm shift in machine learning, focusing on enabling models to adapt to new tasks using only a few examples provided within the input, without requiring parameter updates. Current research emphasizes understanding ICL's mechanisms, particularly within transformer-based large language models, and improving its effectiveness through techniques like enhanced example selection, chain-of-thought prompting, and addressing issues such as spurious correlations and copy bias. This research is significant because ICL offers a more efficient and adaptable approach to many machine learning problems, impacting fields ranging from natural language processing and computer vision to scientific computing and beyond.

Papers

December 19, 2023

Can Transformers Learn Sequential Function Classes In Context?
Ryan Campbell, Emma Guo, Evan Hu, Reya Vir, Ethan Hsiao
Transformer Megatron Decepticons Context Learning Transformer Model Context Information Function Class

December 17, 2023

kNN-ICL: Compositional Task-Oriented Parsing Generalization with Nearest Neighbor In-Context Learning
Wenting Zhao, Ye Liu, Yao Wan, Yibo Wang, Qingyang Wu, Zhongfen Deng, Jiangshu Du, Shuaiqi Liu, Yunlong Xu, Philip S. Yu
Context Learning Semantic Parsing Natural Language Prompt KNN Clip

December 16, 2023

Stochastic Bayesian Optimization with Unknown Continuous Context Distribution via Kernel Density Estimation
Xiaobin Huang, Lei Song, Ke Xue, Chao Qian
Context Learning Optimization Purpose Bayesian Optimization Kernel Density Context Distribution

December 15, 2023

Low-resource classification of mobility functioning information in clinical sentences using large language models
Tuan Dung Le, Thanh Duong, Thanh Thieu
Large Language Model Context Learning MAESTRO Dataset Full Information Task Specific Clinical Text Mobility Management Low Resource Text Classification

December 14, 2023

December 13, 2023

Reconciling Shared versus Context-Specific Information in a Neural Network Model of Latent Causes
Qihong Lu, Tan T. Nguyen, Qiong Zhang, Uri Hasson, Thomas L. Griffiths, Jeffrey M. Zacks, Samuel J. Gershman, Kenneth A. Norman
Context Learning Context Information Neural Network Model LLM Inference Latent Causal Sharing Matter Contextual Structure Common Cause

December 12, 2023

December 11, 2023

December 7, 2023

December 6, 2023

December 5, 2023

Prompt Optimization via Adversarial In-Context Learning
Xuan Long Do, Yiran Zhao, Hannah Brown, Yuxi Xie, James Xu Zhao, Nancy F. Chen, Kenji Kawaguchi, Michael Shieh, Junxian He
Context Learning Prompt Optimization Adversarial Loss Traditional Adversarial

Context Learning

Papers

Can Transformers Learn Sequential Function Classes In Context?

kNN-ICL: Compositional Task-Oriented Parsing Generalization with Nearest Neighbor In-Context Learning

Stochastic Bayesian Optimization with Unknown Continuous Context Distribution via Kernel Density Estimation

Low-resource classification of mobility functioning information in clinical sentences using large language models

A Comparative Analysis of Fine-Tuned LLMs and Few-Shot Learning of LLMs for Financial Sentiment Analysis

MotherNet: A Foundational Hypernetwork for Tabular Classification

Reconciling Shared versus Context-Specific Information in a Neural Network Model of Latent Causes

Can LLM find the green circle? Investigation and Human-guided tool manipulation for compositional generalization

Comparable Demonstrations are Important in In-Context Learning: A Novel Perspective on Demonstration Selection

ICL Markup: Structuring In-Context Learning using Soft-Token Tags

Extracting Self-Consistent Causal Insights from Users Feedback with LLMs and In-context Learning

Flexible visual prompts for in-context learning in computer vision

Transformers Implement Functional Gradient Descent to Learn Non-Linear Functions In Context

MMICT: Boosting Multi-Modal Fine-Tuning with In-Context Examples

A Study on the Calibration of In-context Learning

Cost-Effective In-Context Learning for Entity Resolution: A Design Space Exploration

Skeleton-in-Context: Unified Skeleton Sequence Modeling with In-Context Learning

Context Diffusion: In-Context Aware Image Generation

Generalization to New Sequential Decision Making Tasks with In-Context Learning

Prompt Optimization via Adversarial In-Context Learning