Task Vectors

Follow this topic Ask about this topic

Task vectors represent task-specific behavior as directions or compact changes in a model, either in its weights or in its activations during inference. Researchers use weight differences for model merging, editing, transfer, and compression, while function vectors extracted from activations can steer behavior or capture in-context learning without updating weights. These methods are tested on language and vision-language tasks, including classification, reasoning, behavioral control, and multimodal question answering. A central challenge is that vectors can compose and transfer across tasks or models, yet interference, query-specific behavior, and differences between model architectures can make simple addition or static steering unreliable.

Latest papers

21 papers4 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

From Parameter Rebasin to Functional Transfer Across Heterogeneous Transformers
Oct 23, 2026Filippo Rinaldi, Aniello Panariello, Giacomo Salici +4Transfer Learning Task Vectors
University of Modena and Reggio Emilia, Italy · University of Florence, Italy
Two Vectors Replace In-Context Demos: Structured Task Adaptation via Embeddings
Oct 6, 2026Xi Ding, Naichen Shi, Jiawei ZhangMultimodal ICL Task Vectors
University of Wisconsin–Madison · Northwestern University
Predicting and Repairing Merge Collapse in Large Language Models
Oct 2, 2026Jungseob Lee, Seungyoon Lee, Sugyeong Eo +3Language Model Merging Task Vector Merging
Korea University · Yonsei University Mirae Campus · Sookmyung Women’s University +1
No Task Vector Is an Island: A Comprehensive Study on the Composability of Task Vectors from On-Policy Distillation
Sep 30, 2026Jingang Zhou, Feiyu Han, Han Zhu +6Task Vectors Task Vector Merging
Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Ant Group
Training-Free Task Vectors for LLM Behavioral Control
Sep 8, 2026Gabriel J. Perin, Lucas Boscaini, André Araujo +1Task Vectors AI Control
University of São Paulo · Google · Google DeepMind
When Is a Task Vector Enough? An Empirical Theory of Implicit Multimodal ICL
Aug 13, 2026Jiaqian LiMultimodal ICL Task Vectors
Brown University
Revisiting the Systematicity in Negation in the Era of In-Context Learning
Jun 15, 2026Hitomi Yanaka, Taisei YamamotoTask Vectors In-Context Learning
The University of Tokyo · Riken · Tohoku University
Fast & Faithful Function Vectors
Jun 3, 2026Minh An Pham, Anton Segeler, Thomas Wiegand +4Task Vectors LLM Interpretability
Fraunhofer Heinrich-Hertz-Institute, Berlin, Germany · Technische Universität Berlin, Berlin, Germany · BIFOLD – Berlin Institute for the Foundations of Learning and Data, Berlin, Germany +1
Interpretability-Guided Layer Selection over Subspace Projection: SAEs as Stethoscopes, Not Scalpels, for Raw Task Vector Model Editing
May 27, 2026Li Lei, Madalina Ciobanu, Qingqing Mao +1Task Vectors LLM Interpretability
Incept Labs Houston, TX · Incept Labs, Houston, TX · Titan Holdings, San Francisco, CA
Bilinear Coordinate Alignment for Training-Free Task-Vector Transfer
May 27, 2026Jungyong Son, Jinwook Jung, Minhee Park +1Orthogonal Procrustes Alignment Task Vectors
Dept. of Artificial Intelligence, 2 Dept. of Data Science Hanyang University
Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning
May 25, 2026Han-yu WangTransformer Interpretability Task Vectors
The University of Hong Kong
Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning
May 20, 2026Jihoon Kwon, Jiwon Choi, Jy-yong SohnTask Vectors Distribution Matching
Seoul National University · Yonsei University
How Few-Shot Examples Add Up: A Causal Decomposition of Function Vectors in In-Context Learning
May 15, 2026Entang Wang, Yiwei Wang, Aleksandra Bakalova +1Transformer Interpretability Task Vectors
Saarland Informatics Campus, Saarland Univesity, Saarbrücken, Germany.
CapVector: Learning Transferable Capability Vectors in Parametric Space for Vision-Language-Action Models
May 11, 2026Wenxuan Song, Han Zhao, Fuhao Li +7Task Vectors VLM Adaptation
1HKUST (GZ) · 2Zhejiang University · 3Westlake University +2
Task Vector Geometry Underlies Dual Modes of Task Inference in Transformers
May 5, 2026Hao Yan, Haolin Yang, Yiqiao ZhongTransformer Interpretability Neural Representation Geometry
University of Wisconsin–Madison · University of Chicago
Auto-FlexSwitch: Efficient Dynamic Model Merging via Learnable Task Vector Compression
Apr 30, 2026Junqi Gao, Dazhi Zhang, Zhichang Guo +3Task Vectors Task Vector Merging
School of Mathematics, Harbin Institute of Technology, Harbin, P. R. China · Shanghai Artificial Intelligence Laboratory, Shanghai, P. R. China
Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation
Apr 21, 2026Nurkhan Laiyk, Gerard I. Gállego, Javier Ferrando +1Multilingual Language Models Task Vectors
Mohamed bin Zayed University of Artificial Intelligence · Universitat Politècnica de Catalunya · Cantina Labs
Understanding and Enforcing Weight Disentanglement in Task Arithmetic
Apr 18, 2026Shangge Liu, Yuehan Yin, Lei Wang +5Task Vectors Weight-Space Analysis
State Key Laboratory for Novel Software Technology, Nanjing University, China · University of Wollongong, Australia · Nanyang Technological University, Singapore
DeCoVec: Building Decoding Space based Task Vector for Large Language Models via In-Context Learning
Apr 13, 2026Feiyang Li, Yile WangLanguage Model Steering Task Vectors
College of Computer Science and Software Engineering, Shenzhen University
Fast-dVLA: Accelerating Discrete Diffusion VLA to Real-Time Performance
Mar 26, 2026Wenxuan Song, Jiayi Chen, Shuai Chen +8VLA Model Training Vision-Language-Action Models
The Hong Kong University of Science and Technology (Guangzhou) · ShanghaiTech University · Shanghai Institute of Technical Physics, CAS +3
Task Vector Bases: A Unified and Scalable Framework for Compressed Task Arithmetic
Feb 3, 2025Siqi Zeng, Yifei He, Meitong Liu +5Model Compression Task Vectors
University of Illinois Urbana-Champaign · University of Pennsylvania · Okinawa Institute of Science and Technology

Oct 23, 2026J/K move · Enter open · S save

From Parameter Rebasin to Functional Transfer Across Heterogeneous Transformers

Filippo Rinaldi, Aniello Panariello, Giacomo Salici, Simone Magistri+3

University of Modena and Reggio Emilia, Italy · University of Florence, Italy

Transfer Learning Task Vectors

Open paper AskPDF arXiv

Adapting pretrained Transformers to downstream tasks produces task-specific updates that are expensive to reproduce for every model variant. Reusing these updates across different depths and widths is challenging because their parameters have incompatible shapes and do not share a canonical coordinate system. We introduce Ariadne, a training-free method that transfers task-specific behavior across heterogeneous Transformers without resizing the source model or transporting its parameters. Rather than treating the source task vector, i.e. the difference in parameter space between the fine-tuned and the base models, as the object to move, Ariadne characterizes fine-tuning by the activation changes it induces at matched blocks. First, it aligns these effects to a target base through local Procrustes maps. Then, it directly synthesizes a native target update by closed-form ridge regression. Across Transformer expansion, reduction and language-model transfer, Ariadne improves target-model performance while substantially reducing the time and memory required to construct an update relative to previous cross-architecture rebasin approaches. These results show that local functional effects provide an efficient alternative to parameter transport.