Context Learning
In-context learning (ICL) is a paradigm shift in machine learning, focusing on enabling models to adapt to new tasks using only a few examples provided within the input, without requiring parameter updates. Current research emphasizes understanding ICL's mechanisms, particularly within transformer-based large language models, and improving its effectiveness through techniques like enhanced example selection, chain-of-thought prompting, and addressing issues such as spurious correlations and copy bias. This research is significant because ICL offers a more efficient and adaptable approach to many machine learning problems, impacting fields ranging from natural language processing and computer vision to scientific computing and beyond.
Papers
Is Mamba Capable of In-Context Learning?
Riccardo Grazzi, Julien Siems, Simon Schrodi, Thomas Brox, Frank Hutter
Automatic Combination of Sample Selection Strategies for Few-Shot Learning
Branislav Pecher, Ivan Srba, Maria Bielikova, Joaquin Vanschoren
How do Large Language Models Learn In-Context? Query and Key Matrices of In-Context Heads are Two Towers for Metric Learning
Zeping Yu, Sophia Ananiadou
Solution-oriented Agent-based Models Generation with Verifier-assisted Iterative In-context Learning
Tong Niu, Weihao Zhang, Rong Zhao
The Developmental Landscape of In-Context Learning
Jesse Hoogland, George Wang, Matthew Farrugia-Roberts, Liam Carroll, Susan Wei, Daniel Murfet
Entire Chain Uplift Modeling with Context-Enhanced Learning for Intelligent Marketing
Yinqiu Huang, Shuli Wang, Min Gao, Xue Wei, Changhao Li, Chuan Luo, Yinhua Zhu, Xiong Xiao, Yi Luo
Can MLLMs Perform Text-to-Image In-Context Learning?
Yuchen Zeng, Wonjun Kang, Yicong Chen, Hyung Il Koo, Kangwook Lee
Transformers Learn Nonlinear Features In Context: Nonconvex Mean-field Dynamics on the Attention Landscape
Juno Kim, Taiji Suzuki
In-Context Learning for Few-Shot Nested Named Entity Recognition
Meishan Zhang, Bin Wang, Hao Fei, Min Zhang
Can Large Language Models Understand Context?
Yilun Zhu, Joel Ruben Antony Moniz, Shruti Bhargava, Jiarui Lu, Dhivya Piraviperumal, Site Li, Yuan Zhang, Hong Yu, Bo-Hsiang Tseng
Unlearnable Algorithms for In-context Learning
Andrei Muresanu, Anvith Thudi, Michael R. Zhang, Nicolas Papernot
Theoretical Understanding of In-Context Learning in Shallow Transformers with Unstructured Data
Yue Xing, Xiaofeng Lin, Chenheng Xu, Namjoon Suh, Qifan Song, Guang Cheng
Automated Root Causing of Cloud Incidents using In-Context Learning with GPT-4
Xuchao Zhang, Supriyo Ghosh, Chetan Bansal, Rujia Wang, Minghua Ma, Yu Kang, Saravan Rajmohan
A Unified Approach to Emotion Detection and Task-Oriented Dialogue Modeling
Armand Stricker, Patrick Paroubek
Tyche: Stochastic In-Context Learning for Medical Image Segmentation
Marianne Rakic, Hallee E. Wong, Jose Javier Gonzalez Ortiz, Beth Cimini, John Guttag, Adrian V. Dalca