Task Vectors

Follow this topic Ask about this topic

Momentum

2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 17

All topics

Two Vectors Replace In-Context Demos: Structured Task Adaptation via Embeddings
Oct 6, 2026Xi Ding, Naichen Shi, Jiawei ZhangMultimodal ICL Task Vectors
University of Wisconsin–Madison · Northwestern University
No Task Vector Is an Island: A Comprehensive Study on the Composability of Task Vectors from On-Policy Distillation
Sep 30, 2026Jingang Zhou, Feiyu Han, Han Zhu +6Task Vectors Task Vector Merging
Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Ant Group
Training-Free Task Vectors for LLM Behavioral Control
Sep 8, 2026Gabriel J. Perin, Lucas Boscaini, André Araujo +1Task Vectors AI Control
University of São Paulo · Google · Google DeepMind
When Is a Task Vector Enough? An Empirical Theory of Implicit Multimodal ICL
Aug 13, 2026Jiaqian LiMultimodal ICL Task Vectors
Brown University
Revisiting the Systematicity in Negation in the Era of In-Context Learning
Jun 15, 2026Hitomi Yanaka, Taisei YamamotoTask Vectors In-Context Learning
The University of Tokyo · Riken · Tohoku University
Fast & Faithful Function Vectors
Jun 3, 2026Minh An Pham, Anton Segeler, Thomas Wiegand +4Task Vectors LLM Interpretability
Fraunhofer Heinrich-Hertz-Institute, Berlin, Germany · Technische Universität Berlin, Berlin, Germany · BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany +1
Interpretability-Guided Layer Selection over Subspace Projection: SAEs as Stethoscopes, Not Scalpels, for Raw Task Vector Model Editing
May 27, 2026Li Lei, Madalina Ciobanu, Qingqing Mao +1Task Vectors LLM Interpretability
Incept Labs Houston, TX · Incept Labs, Houston, TX · Titan Holdings, San Francisco, CA
Bilinear Coordinate Alignment for Training-Free Task-Vector Transfer
May 27, 2026Jungyong Son, Jinwook Jung, Minhee Park +1Task Vectors Transfer Learning
Dept. of Artificial Intelligence, 2 Dept. of Data Science Hanyang University
Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning
May 25, 2026Han-yu WangTransformer Interpretability Task Vectors
The University of Hong Kong
Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning
May 20, 2026Jihoon Kwon, Jiwon Choi, Jy-yong SohnTask Vectors Distribution Matching
Seoul National University · Yonsei University
How Few-Shot Examples Add Up: A Causal Decomposition of Function Vectors in In-Context Learning
May 15, 2026Entang Wang, Yiwei Wang, Aleksandra Bakalova +1Transformer Interpretability Task Vectors
Saarland Informatics Campus, Saarland Univesity, Saarbrücken, Germany.
CapVector: Learning Transferable Capability Vectors in Parametric Space for Vision-Language-Action Models
May 11, 2026Wenxuan Song, Han Zhao, Fuhao Li +7Task Vectors VLM Adaptation
1HKUST (GZ) · 2Zhejiang University · 3Westlake University +2
Task Vector Geometry Underlies Dual Modes of Task Inference in Transformers
May 5, 2026Hao Yan, Haolin Yang, Yiqiao ZhongTransformer Interpretability Neural Representation Geometry
University of Wisconsin–Madison · University of Chicago
Auto-FlexSwitch: Efficient Dynamic Model Merging via Learnable Task Vector Compression
Apr 30, 2026Junqi Gao, Dazhi Zhang, Zhichang Guo +3Task Vectors Task Vector Merging
School of Mathematics, Harbin Institute of Technology, Harbin, P. R. China · Shanghai Artificial Intelligence Laboratory, Shanghai, P. R. China
Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation
Apr 21, 2026Nurkhan Laiyk, Gerard I. Gállego, Javier Ferrando +1Multilingual Language Models Task Vectors
Mohamed bin Zayed University of Artificial Intelligence · Universitat Politècnica de Catalunya · Cantina Labs
Understanding and Enforcing Weight Disentanglement in Task Arithmetic
Apr 18, 2026Shangge Liu, Yuehan Yin, Lei Wang +5Task Vectors Weight-Space Analysis
State Key Laboratory for Novel Software Technology, Nanjing University, China · University of Wollongong, Australia · Nanyang Technological University, Singapore
Task Vector Bases: A Unified and Scalable Framework for Compressed Task Arithmetic
Feb 3, 2025Siqi Zeng, Yifei He, Meitong Liu +5Model Compression Task Vectors
University of Illinois Urbana-Champaign · University of Pennsylvania · Okinawa Institute of Science and Technology

Oct 6, 2026, cs.CLJ/K move · Enter open · S save

Two Vectors Replace In-Context Demos: Structured Task Adaptation via Embeddings

Xi Ding, Naichen Shi, Jiawei Zhang

University of Wisconsin–Madison · Northwestern University

Multimodal ICL Task Vectors VLM Adaptation Multimodal Large Language Models Parameter-Efficient Fine-Tuning

Open paper AskPDF arXiv

In-context learning (ICL) adapts frozen large multimodal models (LMMs) to new tasks from a few demonstrations (demos), but re-encodes them at every query, where each demo image adds up to hundreds of visual tokens. Demo-free methods remove this cost with a compact task state. However, they add it at locations searched per task or at every decoder layer, where task parameters grow with depth. Moreover, inserted tokens or keys cannot change how the original prompt divides its attention within a layer. To address these issues, we propose Structured Task Adaptation via Embeddings (STAVE), which replaces demos with two task-specific vectors added to existing input embeddings. Specifically, a readout vector updates the answer-producing tokens and a context vector updates the other structural token groups. Both are trained with answer labels on prompts with and without demos. We justify these design choices theoretically using a first-order analysis of the loss and a margin bound. Extensive experiments on six LMMs and five large language models show that STAVE matches or outperforms state-of-the-art methods on multimodal tasks with far fewer task parameters and surpasses 15-shot ICL and prior task vectors on 18 text tasks, all at zero-shot inference cost.

Momentum

Related topics

Latest papers 17