Overview

Task vectors represent task-specific behavior as directions or compact changes in a model, either in its weights or in its activations during inference. Researchers use weight differences for model merging, editing, transfer, and compression, while function vectors extracted from activations can steer behavior or capture in-context learning without updating weights. These methods are tested on language and vision-language tasks, including classification, reasoning, behavioral control, and multimodal question answering. A central challenge is that vectors can compose and transfer across tasks or models, yet interference, query-specific behavior, and differences between model architectures can make simple addition or static steering unreliable.

Latest papers

CardsList

24 papers4 papers in the last four weeks, up 33% on the four weeks before. 0.0% of all new papers.

  1. Domain-Routed Delta Optimization for Multi-Teacher On-Policy Distillation

    Nov 9, 2026Lin YaoMulti-Teacher Knowledge DistillationOn-Policy Distillation

  2. Separating Common and Unique Directions for Model Merging

    Nov 3, 2026Shiting Wang, Yingjie Zou, Ke LiData-Free Model MergingModel Merging

  3. Task Vectors Can Be Linearly Decomposed into Interpretable and Controllable Components

    Oct 30, 2026Yukiya Horiba, Hakaze Cho, Takahiro YakohTask VectorsLLM Interpretability

  4. From Parameter Rebasin to Functional Transfer Across Heterogeneous Transformers

    Oct 23, 2026Filippo Rinaldi, Aniello Panariello, Giacomo Salici +4Transfer LearningTask Vectors

  5. Two Vectors Replace In-Context Demos: Structured Task Adaptation via Embeddings

    Oct 6, 2026Xi Ding, Naichen Shi, Jiawei ZhangMultimodal ICLTask Vectors

  6. Predicting and Repairing Merge Collapse in Large Language Models

    Oct 2, 2026Jungseob Lee, Seungyoon Lee, Sugyeong Eo +3Language Model MergingTask Vector Merging

  7. No Task Vector Is an Island: A Comprehensive Study on the Composability of Task Vectors from On-Policy Distillation

    Sep 30, 2026Jingang Zhou, Feiyu Han, Han Zhu +6Task VectorsTask Vector Merging

  8. Training-Free Task Vectors for LLM Behavioral Control

    Sep 8, 2026Gabriel J. Perin, Lucas Boscaini, André Araujo +1Task VectorsAI Control

  9. Revisiting the Systematicity in Negation in the Era of In-Context Learning

    Jun 15, 2026Hitomi Yanaka, Taisei YamamotoTask VectorsIn-Context Learning

  10. Fast & Faithful Function Vectors

    Jun 3, 2026Minh An Pham, Anton Segeler, Thomas Wiegand +4Task VectorsLLM Interpretability

  11. Interpretability-Guided Layer Selection over Subspace Projection: SAEs as Stethoscopes, Not Scalpels, for Raw Task Vector Model Editing

    May 27, 2026Li Lei, Madalina Ciobanu, Qingqing Mao +1Task VectorsLLM Interpretability

  12. Bilinear Coordinate Alignment for Training-Free Task-Vector Transfer

    May 27, 2026Jungyong Son, Jinwook Jung, Minhee Park +1Orthogonal Procrustes AlignmentTask Vectors

  13. Distributional Alignment as a Criterion for Designing Task Vectors in In-Context Learning

    May 20, 2026Jihoon Kwon, Jiwon Choi, Jy-yong SohnTask VectorsDistribution Matching

  14. How Few-Shot Examples Add Up: A Causal Decomposition of Function Vectors in In-Context Learning

    May 15, 2026Entang Wang, Yiwei Wang, Aleksandra Bakalova +1Transformer InterpretabilityTask Vectors

  15. CapVector: Learning Transferable Capability Vectors in Parametric Space for Vision-Language-Action Models

    May 11, 2026Wenxuan Song, Han Zhao, Fuhao Li +7Task VectorsVLM Adaptation

  16. Task Vector Geometry Underlies Dual Modes of Task Inference in Transformers

    May 5, 2026Hao Yan, Haolin Yang, Yiqiao ZhongTransformer InterpretabilityNeural Representation Geometry

  17. Auto-FlexSwitch: Efficient Dynamic Model Merging via Learnable Task Vector Compression

    Apr 30, 2026Junqi Gao, Dazhi Zhang, Zhichang Guo +3Task VectorsTask Vector Merging

  18. Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation

    Apr 21, 2026Nurkhan Laiyk, Gerard I. Gállego, Javier Ferrando +1Multilingual Language ModelsTask Vectors

  19. Understanding and Enforcing Weight Disentanglement in Task Arithmetic

    Apr 18, 2026Shangge Liu, Yuehan Yin, Lei Wang +5Task VectorsWeight-Space Analysis

  20. DeCoVec: Building Decoding Space based Task Vector for Large Language Models via In-Context Learning

    Apr 13, 2026Feiyang Li, Yile WangLanguage Model SteeringTask Vectors

  21. Fast-dVLA: Accelerating Discrete Diffusion VLA to Real-Time Performance

    Mar 26, 2026Wenxuan Song, Jiayi Chen, Shuai Chen +8VLA TrainingVision-Language-Action Models

  22. Task Vector Bases: A Unified and Scalable Framework for Compressed Task Arithmetic

    Feb 3, 2025Siqi Zeng, Yifei He, Meitong Liu +5Model CompressionTask Vectors