Multimodal ICL

ICL: In-Context Learning

Ask about this topic

Overview

Multimodal in-context learning asks whether models can infer task rules or acquire useful information from examples at inference time, without updating their weights. Common approaches retrieve or learn to select demonstrations, structure prompts for reasoning, and compress examples into task vectors, adapters, or reusable memory; applications include visual question answering, classification, audio understanding, and video interaction. Benchmarks increasingly test not only answers but whether models ground context, apply new rules, and adapt to changing criteria. A central tension is that strong zero-shot performance does not guarantee effective use of demonstrations: multimodal grounding, noisy or irrelevant context, and transfer from examples to new queries remain persistent challenges.

Latest papers

CardsList

49 papers9 papers in the last four weeks, up 50% on the four weeks before. 0.1% of all new papers.

  1. Predicting Music Valence and Arousal via Textual and Multimodal In-Context Learning in Large Language Models

    Oct 22, 2026Chi-En Dai, Chih-Pin Tan, Yi-Hsuan YangValence-Arousal ModelingEmotion Recognition

    Graduate Institute of Communication Engineering, National Taiwan University, Taiwan

  2. Watch and Learn: Teaching Visual In-Context Learning by Removing Textual Shortcuts

    Oct 22, 2026Nimrod Shabtay, Asaf Yehudai, Avihu Dekel +2Multimodal ICLIn-Context Learning

    IBM Research · Tel-Aviv University

  3. Visual Grounding First, Multimodal In-context Learning Follows

    Oct 22, 2026Minhyuk Seo, Minjae Lee, Chaeeun Lee +4Multimodal ICLVision-Language Grounding

    UC Berkeley · Seoul National University · Xero +2

  4. ActICL: Post-Training for Agentic Multimodal In-Context Learning

    Oct 14, 2026Yanshu Li, Ruixiang Tang, Junyi Jessy Li +1Multimodal ICLRetrieval-Augmented ICL

    University of Texas at Austin · 2Rutgers University

  5. TraceICL: Thread-Resolved Demonstration Curation with Consistency Calibration for Robust Multimodal In-Context Learning

    Oct 7, 2026Yigit Yilmaz, Elena Petrova, Mehmet Kaya +2Multimodal ICLIn-Context Example Selection

    Bandirma Onyedi Eylul University

  6. Two Vectors Replace In-Context Demos: Structured Task Adaptation via Embeddings

    Oct 6, 2026Xi Ding, Naichen Shi, Jiawei ZhangMultimodal ICLTask Vectors

    University of Wisconsin–Madison · Northwestern University

  7. MCD: Causal Distillation of Multimodal In-Context Learning in Large Vision-Language Models

    Sep 30, 2026Yanshu Li, Jiaqian Li, Canran Xiao +3Cross-Modal Knowledge DistillationMultimodal ICL

    Brown University · University of Alabama at Birmingham · Hanyang University

  8. Are In-Context Images Worth 10 Dimensions?

    Sep 29, 2026Adhemar de Senneville, Xavier Bou, Jérémy Anger +2Vision-Language ModelsMultimodal ICL

    Université Paris-Saclay, CNRS, ENS Paris-Saclay, Centre Borelli, Paris, France · Pôle recherche de l’AMIAD, Palaiseau, France · Institut Universitaire de France, Paris, France

  9. Exploring In-Context Learning for Handwritten Text Recognition

    Sep 29, 2026Eric Ayllon, Abel Gandia, Jorge Calvo-ZaragozaIn-Context Example SelectionVision-Language Models

  10. M2^2PFN: End-to-End Disentangled Alignment for Generalizable Multimodal In-Context Learning in Alzheimer's Disease

    Sep 23, 2026Lujia Zhong, Shuo Huang, Jianwei Zhang +2Multimodal Disentangled Representation LearningMultimodal ICL

    Stevens Neuroimaging and Informatics Institute, Keck School of Medicine, University of Southern California, Los Angeles, CA 90033, USA · Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA 90089, USA · Alfred E. Mann Department of Biomedical Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, CA 90089, USA

  11. V-ICAL Bench: Evaluating Video In-Context Learning for Multimodal Agents in Interactive Environments

    Sep 14, 2026Ziqian Fan, Shibo Xu, Junjie Li +8Multimodal ICLAI Agent Benchmarks

    South China University of Technology · Shanghai Jiao Tong University · Fudan University +4

  12. TwinICL: Diagnosing Multimodal In-Context Learning through Paired Counterfactuals

    Sep 14, 2026Zihan Xue, Po-Yi Lu, Serhii Honcharenko +5Multimodal ICLModality Gap in VLMs

    University of California, Los Angeles · 2National Taiwan University · 3Texas A&M University +2

  13. Convergent Emergence of In-Context Learning Across Modalities

    Sep 12, 2026Nathan Breslow, Seungwook Han, Daniel Hyunsoo Lee +3In-Context LearningMultimodal ICL

    Johns Hopkins University · MIT · University of Illinois Urbana-Champaign

  14. Beyond Surface Imitation: Contrastive Modeling for Reasoning Path Alignment in Multimodal In-Context Learning

    Sep 9, 2026Mingbo Yang, Wenqiang Wang, Zhaolu Kang +4Contrastive LearningMultimodal ICL

    Sun Yat-Sen University Shenzhen, Guangdong, China · Peking University Beijing, Beijing, China · Sun Yat-Sen University Shenzhen, Guangdong, China Pengcheng Laboratory +1

  15. Understanding In-Context Multimodal Jailbreaks via Posterior Reweighting

    Sep 8, 2026Xu Zhang, Dev Mistry, Xiang Xu +1Multimodal Jailbreak AttacksAdversarial Robustness of Language Models

  16. Pretraining Reusable Inference Across Views with Synthetic Task Priors

    Aug 19, 2026Jielong Lu, Zhihao Wu, Jiajun Yu +2Multi-View LearningMultimodal ICL

    Zhejiang University, Hangzhou, China · Hong Kong Baptist University, Hong Kong SAR, China

  17. VicEdit: Learning to Edit Videos from Visual In-Context Examples

    Aug 17, 2026Yuji Wang, Teng Hu, Yuheng Chen +6Video EditingInstruction-Based Video Editing

    Shanghai Jiao Tong University · Tencent Youtu Lab · Zhejiang University

  18. Boosting Knowledge-based Visual Question Answering with Structured Context Reasoning

    Aug 17, 2026Qiyou Liu, Yong Zhang, Jianjie Luo +2Knowledge-Based VQAVLM Reasoning

    School of Computer Science, Guangdong University of Technology, Guangzhou, China · School of Artificial Intelligence, Shenzhen University, Shenzhen, China · Graduate School of Advanced Science and Engineering, Hiroshima University, Hiroshima, Japan

  19. When Is a Task Vector Enough? An Empirical Theory of Implicit Multimodal ICL

    Aug 13, 2026Jiaqian LiMultimodal ICLTask Vectors

    Brown University

  20. MAG: MAnifold Guided Semi-Supervised Multi-modal In-Context Learning

    Aug 13, 2026Zirui Cheng, Xun Xu, Tiankai Chen +7In-Context Example SelectionMultimodal ICL

    College of Design and Engineering, National University of Singapore · Institute for Infocomm Research, ASTAR, Singapore

  21. In-Context Collapse in Vision-Language Models and How to Mitigate it?

    Aug 3, 2026Mohammad RostamiVLM RobustnessVision-Language Models

    Amazon Generative AI Innovation Center∗, USA.

  22. MoRoute: Dynamic Routing for In-Context Multimodal Video Generation

    Jul 31, 2026Chong Gao, Jie Ma, Zhan Peng +5Video Diffusion ModelsMultimodal ICL

    Sun Yat-sen University · Orange Team, Moku Lab, HUJING Digital Media & Entertainment Group · Huazhong University of Science and Technology

  23. CLBench-V: Evaluating Multimodal Context Learning from Grounding to Knowledge Acquisition

    Jul 28, 2026Lai Wei, Chengqi Li, Jiapeng Li +3Multimodal GroundingMultimodal ICL

    School of Computer Science, Shanghai Jiao Tong University · 2Zhongguancun Academy · 3Shanghai Innovation Institute

  24. In-Context Learning for Wound Classification with Small Multimodal Language Models

    Jul 21, 2026George Martvel, Oskar Gustafsson, John Pavia +1Multimodal ICLMultimodal Large Language Models

    1Jönköping University, Jönköping, Sweden · 2Halmstad University, Halmstad, Sweden · 3Mölnlycke Health Care +1

  25. Tree-of-Thoughts Reasoning for Text-to-Image In-Context Learning

    Jul 8, 2026Stepanida Alekseeva, Jenifer Kalafatovich, Seong-Whan LeeTree of ThoughtsMultimodal ICL

    Department of Artificial Intelligence, Korea University, Seoul, Korea.

  26. Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection

    Jul 4, 2026Runzhi Deng, Yundi Hu, Yiming Zhong +5Multimodal ICLIndustrial Anomaly Detection

    Nanjing University, Nanjing, Jiangsu, China · China Mobile Zijin Innovation Institute, Nanjing, Jiangsu, China · Southeast University, Nanjing, Jiangsu, China

  27. Retrieved Images as Visual Thought: Training-Free Multimodal In-Context Learning for the Open-vs-Closed Gap

    Jul 1, 2026Bingchen Huang, Zhiling Wang, Yifu Chen +1Multimodal RAGMultimodal ICL

    Meituan

  28. Criterion-Conditional In-Context Learning: Evaluating Criterion-Shift Adaptation in Vision-Language Models

    Jun 30, 2026Kaiyun Yang, Ruilin Yang, Zhimin Yao +2VLM EvaluationMultimodal ICL

    University of Science and Technology of China, Hefei, China · Megvii Technology Inc., Beijing, China.

  29. Unlocking In-Context Learning in Audio-Language Models from Decentralized Medical Audio

    Jun 22, 2026Ran Piao, Tsai-Ning Wang, Martijn den Dekker +4Few-Shot Audio ClassificationMultimodal ICL

    Eindhoven University of Technology, The Netherlands · Erasmus MC, The Netherlands · Rijnstate Hospital, The Netherlands +1

  30. MAF: Multimodal Adaptive Few-shot Prompting for Sentiment Analysis with MLLMs

    Jun 14, 2026Hangling XieMultimodal ICLMultimodal Large Language Models

    Nanjing University of Posts and Telecommunications, Wenyuan Road No.9, Xixia District, Nanjing City, Jiangsu Province, China