Context Learning

In-context learning (ICL) is a paradigm shift in machine learning, focusing on enabling models to adapt to new tasks using only a few examples provided within the input, without requiring parameter updates. Current research emphasizes understanding ICL's mechanisms, particularly within transformer-based large language models, and improving its effectiveness through techniques like enhanced example selection, chain-of-thought prompting, and addressing issues such as spurious correlations and copy bias. This research is significant because ICL offers a more efficient and adaptable approach to many machine learning problems, impacting fields ranging from natural language processing and computer vision to scientific computing and beyond.

Papers

March 19, 2024

March 18, 2024

Towards Understanding the Relationship between In-context Learning and Compositional Generalization
Sungjun Han, Sebastian Padó
Strong Generalization Context Learning Inductive Bias Compositional Generalization Human Relationship

March 15, 2024

The Whole is Better than the Sum: Using Aggregated Demonstrations in In-Context Learning for Sequential Recommendation
Lei Wang, Ee-Peng Lim
Large Language Model Context Learning Sequential Recommendation Whole Body Partial Sum Effective Demonstration

March 14, 2024

Rectifying Demonstration Shortcut in In-Context Learning
Joonwon Jang, Sanghwan Jang, Wonbin Kweon, Minjin Jeon, Hwanjo Yu
Large Language Model Context Learning Expert Demonstration Semantic Prior Optimal Demonstration

March 12, 2024

In-context learning enables multimodal large language models to classify cancer pathology images
Dyke Ferber, Georg Wölflein, Isabella C. Wiest, Marta Ligero, Srividhya Sainath, Narmin Ghaffari Laleh, Omar S. M. El Nahhas, Gustav Müller-Franzes, Dirk Jäger, Daniel Truhn, Jakob Nikolas Kather
Context Learning Multimodal Large Language Model Medical Image Analysis Medical Image Classification Cancer Detection Medical Image Analysis Task

March 11, 2024

March 10, 2024

In-context Prompt Learning for Test-time Vision Recognition with Frozen Vision-language Model
Junhui Yin, Xinyu Zhang, Lin Wu, Xiaojie Wang
Context Learning Prompt Learning Pre Trained Vision Language Model Test Time Prompt Tuning Frozen Vision Language

March 9, 2024

Few-Shot Cross-Lingual Transfer for Prompting Large Language Models in Low-Resource Languages
Christopher Toukmaji
Large Language Model Fine Tuning Context Learning Low Resource Language Cross Lingual Transfer Large Pre Trained Language Model Shot Prompting

March 8, 2024

March 5, 2024

March 4, 2024

Analyzing and Adapting Large Language Models for Few-Shot Multilingual NLU: Are We There Yet?
Evgeniia Razumovskaia, Ivan Vulić, Anna Korhonen
Natural Language Processing Context Learning Instruction Tuning Supervised Fine Tuning Language Adaptation Shot Cross Lingual

March 3, 2024

March 2, 2024

Context Learning

Papers

VL-ICL Bench: The Devil in the Details of Multimodal In-Context Learning

Towards Multimodal In-Context Learning for Vision & Language Models

Towards Understanding the Relationship between In-context Learning and Compositional Generalization

The Whole is Better than the Sum: Using Aggregated Demonstrations in In-Context Learning for Sequential Recommendation

Rectifying Demonstration Shortcut in In-Context Learning

In-context learning enables multimodal large language models to classify cancer pathology images

MEND: Meta dEmonstratioN Distillation for Efficient and Effective In-Context Learning

In-context Exploration-Exploitation for Reinforcement Learning

One size doesn't fit all: Predicting the Number of Examples for In-Context Learning

In-context Prompt Learning for Test-time Vision Recognition with Frozen Vision-language Model

Few-Shot Cross-Lingual Transfer for Prompting Large Language Models in Low-Resource Languages

DP-TabICL: In-Context Learning with Differentially Private Tabular Data

Concept-aware Data Construction Improves In-context Learning of Language Models

How Well Can Transformers Emulate In-context Newton's Method?

JMI at SemEval 2024 Task 3: Two-step approach for multimodal ECAC using in-context learning with GPT and instruction-tuned Llama models

Analyzing and Adapting Large Language Models for Few-Shot Multilingual NLU: Are We There Yet?

Transformers for Supervised Online Continual Learning

Revisiting Dynamic Evaluation: Online Adaptation for Large Language Models

Distilling Text Style Transfer With Self-Explanation From LLMs

FaiMA: Feature-aware In-context Learning for Multi-domain Aspect-based Sentiment Analysis