Shot in Context Learning

Shot-in-context learning (ICL) explores how large language models (LLMs) can perform tasks with minimal explicit training, leveraging a few examples provided within the input prompt. Current research focuses on improving ICL's robustness and efficiency, particularly through techniques like chain-of-thought prompting, multimodal task vectors for compressing many examples, and methods to mitigate biases and hallucinations. This research is significant because it promises more efficient and adaptable AI systems, impacting diverse fields from healthcare diagnostics and molecular design to environmental monitoring and multilingual natural language processing.

Papers

February 16, 2024

Let's Learn Step by Step: Enhancing In-Context Learning Ability with Curriculum Learning
Yinpeng Liu, Jiawei Liu, Xiang Shi, Qikai Cheng, Yong Huang, Wei Lu
Large Language Model Context Learning Curriculum Learning Open Source Large Language Model Critique Ability Different Context Cross Over Step Shot in Context Learning Optimal Demonstration

January 21, 2024

In-context Learning with Retrieved Demonstrations for Language Models: A Survey
Man Luo, Xin Xu, Yue Liu, Panupong Pasupat, Mehran Kazemi
Language Model Timely Survey Context Learning Retrieval Model Pre Trained Large Language Model Shot in Context Learning Demonstration Retrieval

November 26, 2023

Learning to Skip for Language Modeling
Dewen Zeng, Nan Du, Tao Wang, Yuanzhong Xu, Tao Lei, Zhifeng Chen, Claire Cui
Language Model LeArning Abstract Large Scale Language Model Shot in Context Learning Skip and Recover

November 15, 2023

Few-shot Transfer Learning for Knowledge Base Question Answering: Fusing Supervised Models with In-Context Learning
Mayur Patidar, Riya Sawhney, Avinash Singh, Biswajit Chatterjee, Mausam, Indrajit Bhattacharya
Context Learning Supervised Learning Knowledge Retrieval Shot in Context Learning Knowledge Base Question Answering Shot Transfer Learning

October 16, 2023

LLM4SGG: Large Language Models for Weakly Supervised Scene Graph Generation
Kibum Kim, Kanghoon Yoon, Jaehyeong Jeon, Yeonjun In, Jinyoung Moon, Donghyun Kim, Chanyoung Park
Weakly Supervised Scene Graph Generation Shot in Context Learning Fully Supervised Visual Genome

September 27, 2023

Generative Speech Recognition Error Correction with Large Language Models and Task-Activating Prompting
Chao-Han Huck Yang, Yile Gu, Yi-Chieh Liu, Shalini Ghosh, Ivan Bulyko, Andreas Stolcke
Large Language Model Context Learning Shot in Context Learning Domain Specific Language Model Context Prompting Generative Error Correction

June 19, 2023

Multilingual Few-Shot Learning via Language Model Retrieval
Genta Indra Winata, Liang-Kang Huang, Soumya Vadlamannati, Yash Chandarana
Language Model LeArning Abstract Transformer Based Language Model Shot in Context Learning Question Classification Multilingual Task Shot Data

May 24, 2023

May 8, 2023

GersteinLab at MEDIQA-Chat 2023: Clinical Note Summarization from Doctor-Patient Conversations through Fine-tuning and In-context Learning
Xiangru Tang, Andrew Tran, Jeffrey Tan, Mark Gerstein
Large Language Model Fine Tuning Context Learning Dialogue Summarization Shot in Context Learning Clinical Note Summarization Doctor Patient

May 2, 2023

February 3, 2023

Towards Few-Shot Identification of Morality Frames using In-Context Learning
Shamik Roy, Nishanth Sridhar Nakshatri, Dan Goldwasser
Context Learning NLP Field Person Identification Psycholinguistic Research Shot in Context Learning Moral Value Moral Theory

January 19, 2023

Batch Prompting: Efficient Inference with Large Language Model APIs
Zhoujun Cheng, Jungo Kasai, Tao Yu
Large Language Model Efficient Inference Inference Cost Shot in Context Learning

December 21, 2022

Prompt-Augmented Linear Probing: Scaling beyond the Limit of Few-shot In-Context Learners
Hyunsoo Cho, Hyuhng Joon Kim, Junyeob Kim, Sang-Woo Lee, Sang-goo Lee, Kang Min Yoo, Taeuk Kim
Language Model Context Learning Continuum Limit Large Scale Language Model Linear Probing Shot in Context Learning

December 19, 2022

Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environments
Yu Gu, Xiang Deng, Yu Su
Language Model Deep Learning Language Understanding Real World Environment Proposal Balance Refinement Shot in Context Learning Discriminative Reply

July 2, 2022

Rationale-Augmented Ensembles in Language Models
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Denny Zhou
Language Model Multi Step Reasoning Shot in Context Learning Meta Prompting

May 11, 2022

Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning
Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, Colin Raffel
Pre Trained Language Model Context Learning Parameter Efficient Fine Tuning Shot in Context Learning Cost Effective

May 3, 2022

Improving In-Context Few-Shot Learning via Self-Supervised Training
Mingda Chen, Jingfei Du, Ramakanth Pasunuru, Todor Mihaylov, Srini Iyer, Veselin Stoyanov, Zornitsa Kozareva
LeArning Abstract Self Supervision Self Supervised Pretraining Different Context Self Supervised Training Self Supervised Objective Shot in Context Learning

April 28, 2022

On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Model
Seongjin Shin, Sang-Woo Lee, Hwijeen Ahn, Sungdong Kim, HyoungSeok Kim, Boseop Kim, Kyunghyun Cho, Gichang Lee, Woomyoung Park, Jung-Woo Ha, Nako Sung
Context Learning Shot Learning Mixed Effect Large Scale Language Model Training Corpus Shot in Context Learning Pre Training Corpus Context Learning Ability

Shot in Context Learning

Papers

Let's Learn Step by Step: Enhancing In-Context Learning Ability with Curriculum Learning

In-context Learning with Retrieved Demonstrations for Language Models: A Survey

Learning to Skip for Language Modeling

Few-shot Transfer Learning for Knowledge Base Question Answering: Fusing Supervised Models with In-Context Learning

LLM4SGG: Large Language Models for Weakly Supervised Scene Graph Generation

Generative Speech Recognition Error Correction with Large Language Models and Task-Activating Prompting

Multilingual Few-Shot Learning via Language Model Retrieval

Revisiting Parallel Context Windows: A Frustratingly Simple Alternative and Chain-of-Thought Deterioration

Exploring Diverse In-Context Configurations for Image Captioning

GersteinLab at MEDIQA-Chat 2023: Clinical Note Summarization from Doctor-Patient Conversations through Fine-tuning and In-context Learning

Few-shot In-context Learning for Knowledge Base Question Answering

Why So Gullible? Enhancing the Robustness of Retrieval-Augmented Models against Counterfactual Noise

Towards Few-Shot Identification of Morality Frames using In-Context Learning

Batch Prompting: Efficient Inference with Large Language Model APIs

Prompt-Augmented Linear Probing: Scaling beyond the Limit of Few-shot In-Context Learners

Don't Generate, Discriminate: A Proposal for Grounding Language Models to Real-World Environments

Rationale-Augmented Ensembles in Language Models

Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning

Improving In-Context Few-Shot Learning via Self-Supervised Training

On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Model