Context Learning

In-context learning (ICL) is a paradigm shift in machine learning, focusing on enabling models to adapt to new tasks using only a few examples provided within the input, without requiring parameter updates. Current research emphasizes understanding ICL's mechanisms, particularly within transformer-based large language models, and improving its effectiveness through techniques like enhanced example selection, chain-of-thought prompting, and addressing issues such as spurious correlations and copy bias. This research is significant because ICL offers a more efficient and adaptable approach to many machine learning problems, impacting fields ranging from natural language processing and computer vision to scientific computing and beyond.

Papers

November 25, 2024

November 24, 2024

Can a Large Language Model Learn Matrix Functions In Context?
Paimon Goulart, Evangelos E. Papalexakis
Fine Tuning Context Learning Context Information High Dimensional Problem

November 22, 2024

Optimizing Social Media Annotation of HPV Vaccine Skepticism and Misinformation Using Large Language Models: An Experimental Evaluation of In-Context Learning and Fine-Tuning Stance Detection Across Multiple Models
Luhang Sun, Varsha Pendyala, Yun-Shiuan Chuang, Shanglin Yang, Jonathan Feldman, Andrew Zhao, Munmun De Choudhury, Sijia Yang, Dhavan Shah
Context Learning Stance Detection Text Annotation Vaccine Hesitancy

November 18, 2024

Re-examining learning linear functions in context
Omar Naim, Guilhem Fouilhé, Nicholas Asher
Training Data Full Model Context Learning Transformer Based Model Context Information Linear Function

November 17, 2024

November 16, 2024

One-Layer Transformer Provably Learns One-Nearest Neighbor In Context
Zihao Li, Yuan Cao, Cheng Gao, Yihan He, Han Liu, Jason M. Klusowski, Jianqing Fan, Mengdi Wang
Context Learning Transformer Model Context Information Layer Transformer Softmax Attention

November 15, 2024

On the Privacy Risk of In-context Learning
Haonan Duan, Adam Dziedzic, Mohammad Yaghini, Nicolas Papernot, Franziska Boenisch
Large Language Model Context Learning Shot Learning Membership Inference Attack Privacy Risk Natural Language Prompt

November 14, 2024

StreamAdapter: Efficient Test Time Adaptation from Contextual Streams
Dilxat Muhtar, Yelong Shen, Yaming Yang, Xiaodong Liu, Yadong Lu, Jianfeng Liu, Yuefeng Zhan, Hao Sun, Weiwei Deng, Feng Sun, Xueliang Zhang, Jianfeng Gao, Weizhu Chen, Qi Zhang
Large Language Model Context Learning Test Time Adaptation Efficient Inference Inference Cost

November 13, 2024

XiYan-SQL: A Multi-Generator Ensemble Framework for Text-to-SQL
Yingqi Gao, Yifu Liu, Xiaoxia Li, Xiaorong Shi, Yin Zhu, Yiming Wang, Shiqi Li, Wei Li, Yuntao Hong, Zhiling Luo, Jinyang Gao, Liyu Mou, Yu Li
Large Language Model Context Learning Entity Recognition Text to SQL

November 12, 2024

Decision Feedback In-Context Symbol Detection over Block-Fading Channels
Li Fan, Jing Yang, Cong Shen
Context Learning Pre Trained Transformer Human Decision Symbol Detection Large Scale Fading Transformer Based Decoder

November 10, 2024

In-Context Learning for Preserving Patient Privacy: A Framework for Synthesizing Realistic Patient Portal Messages
Joseph Gatto, Parker Seegmiller, Timothy E. Burdick, Sarah Masud Preum
New Framework Context Learning Patient Friendly Language Sensitive Patient Healthcare Workflow Patient Privacy Synthetic Patient Physician Dialogue

November 8, 2024

WeatherGFM: Learning A Weather Generalist Foundation Model via In-context Learning
Xiangyu Zhao, Zhiwang Zhou, Wenlong Zhang, Yihao Liu, Xiangyu Chen, Junchao Gong, Hao Chen, Ben Fei, Shiqi Chen, Wanli Ouyang, Xiao-Ming Wu, Lei Bai
Foundation Model Context Learning Image Translation Weather Forecasting Climate Data Visual Foundation Model

November 7, 2024

November 6, 2024

November 5, 2024

Mixtures of In-Context Learners
Giwon Hong, Emile van Krieken, Edoardo Ponti, Nikolay Malkin, Pasquale Minervini
Context Learning Mixture Component Noisy Demonstration Different Context Transformer Based LLM Context Learner

Context Learning

Papers

MICAS: Multi-grained In-Context Adaptive Sampling for 3D Point Cloud Processing

Med-PerSAM: One-Shot Visual Prompt Tuning for Personalized Segment Anything Model in Medical Domain

Can a Large Language Model Learn Matrix Functions In Context?

Optimizing Social Media Annotation of HPV Vaccine Skepticism and Misinformation Using Large Language Models: An Experimental Evaluation of In-Context Learning and Fine-Tuning Stance Detection Across Multiple Models

Re-examining learning linear functions in context

AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers

SymDPO: Boosting In-Context Learning of Large Multimodal Models with Symbol Demonstration Direct Preference Optimization

One-Layer Transformer Provably Learns One-Nearest Neighbor In Context

On the Privacy Risk of In-context Learning

StreamAdapter: Efficient Test Time Adaptation from Contextual Streams

XiYan-SQL: A Multi-Generator Ensemble Framework for Text-to-SQL

Decision Feedback In-Context Symbol Detection over Block-Fading Channels

In-Context Learning for Preserving Patient Privacy: A Framework for Synthesizing Realistic Patient Portal Messages

WeatherGFM: Learning A Weather Generalist Foundation Model via In-context Learning

Adversarial Robustness of In-Context Learning in Transformers for Linear Regression

Vision Language Models are In-Context Value Learners

Bio-xLSTM: Generative modeling, representation and in-context learning of biological and chemical sequences

Can Custom Models Learn In-Context? An Exploration of Hybrid Architecture Performance on In-Context Learning Tasks

EXPLORA: Efficient Exemplar Subset Selection for Complex Reasoning

Mixtures of In-Context Learners