Instruction Tuning

Follow this topic Ask about this topic

Momentum

5 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

Jul 13Week of Sep 28

Latest papers 48

All topics

A helps B while B hurts A: directed transfer in instruction-tuning mixture
Sep 30, 2026Nima H. Siboni, Vahid RostamiInstruction Tuning Mixtures
Juna.ai, Kastanienallee 32, 10435 Berlin, Germany · Computational Systems Neuroscience, Institute of Zoology, University of Cologne, Germany
The Safety Operator: Modulating the Expression of Safety Instructions via Spectral Optimization
Sep 29, 2026Benoit Dherin, Michael Munn, Xavier Gonzalvo +14Safety Claims Instruction Tuning
Google Research · Cambridge University · Google DeepMind +2
Render Before Reading: Visual Rendering as a Prompt Injection Defense
Sep 28, 2026Jie Zhang, Andrei Baroian, Jan N. van Rijn +2Indirect Prompt Injection Multimodal Large Language Models
ETH Zurich · Leiden University
Rewired or Gated? How Instruction Tuning Shapes Knowledge-Conflict Circuits in LLMs
Sep 22, 2026Shubham Santosh Pandere, Gautam Ranka, Ritika Varshney +3Instruction Tuning Instruction-Tuned Models
IvLabs, VNIT
Stabilizing Instruction Supervision for Instruct-TTS via Controllable Diversification and Drift Filtering
Sep 8, 2026Yizhong Geng, Kecan Mao, Qifei Li +6Instruction Tuning Instruction
Beijing University of Posts and Telecommunications, China · Li Auto, China
LLMs Learn Better In-Context from Rules than from Examples
Sep 2, 2026Xiang Fu, Seungmin Cho, Yukyung Lee +1In-Context Learning In-Context
Boston University
How Output Format Confounds Data Quality and Capability in Instruction Tuning
Sep 2, 2026Chengguang Gan, Hanjun Wei, Yunhao Liang +3Instruction Tuning Data Quality
1Independent Researcher · University of Chinese Academy of Sciences · 3Pusan National University +2
Are You Sure You're Sure? On the Impact of Instruction Tuning on Confidence and Lexical Diversity
Aug 13, 2026Irina Proskurina, Mayank Kumar, Oyindolapo O. KomolafeInstruction Tuning Instruction-Tuned Models
Cohere Labs Community · Laboratoire Hubert Curien, UMR CNRS 5516, Saint-Étienne, France · School of Computer Science Engineering and Technology (SCSET), Bennett University, Greater Noida, India +1
Is Convergence Inevitable? Tracing Output Homogeneity Back to Base Models
Aug 11, 2026Alexandrine Fortier, Hazel Chen, Peter WestInstruction Tuning Convergence
University of British Columbia
DistMoE: Private-data Rehearsal-free Routing in Mixture-of-Experts for Distributed Instruction Tuning
Aug 10, 2026Mainak Singha, Niccolò Biondi, Elisa Ricci +1Multimodal Continual Instruction Tuning Mixture-Of-Experts
University of Trento, Italy · Fondazione Bruno Kessler, Italy · University of Bergamo, Italy
A Unified Moral-Value Dataset for Instruction Tuning
Jul 23, 2026Zhaohui Zeng, Florian MaiInstruction Tuning Moral Reasoning
1RWTH Aachen · University of Bonn · 3Lamarr Institute for Machine Learning and Artificial Intelligence
Learning from Synthetic Data without Model Collapse in Iterative Instruction Tuning
Jul 19, 2026Xiaonan Luo, Yue Huang, Kehan Guo +4Instruction Tuning Synthetic Training Data
University of Notre Dame · Vanderbilt University · University of Pennsylvania
Leveraging Instruction Tuning and Merging for Reasoning Model Adaptation
Jul 16, 2026Yu-Du Feng, Niels Mündler-Sasahara, Mark Vero +1LLM Reasoning Strategies Instruction Tuning
Department of Computer Science, ETH Zurich, Zurich, Switzerland.
Unlocking Speech-Text Compositional Powers: Instruction-Following Speech Language Models without Instruction Tuning
Jul 2, 2026Congrui Du, Yang Zhang, Kaizhi Qian +1Speech Language Models Instruction Tuning
University of California, Santa Barbara, USA · MIT-IBM Computing Research Lab, IBM Research, USA
Online Data Selection for Instruction Tuning via Gaussian Processes
Jun 29, 2026Jun Wang, Quoc Phong Nguyen, Julien Monteil +1Instruction Tuning Gaussian Process
Amazon
TraMP-LLaMA: Generative Interpretability with Decoupled Instruction Tuning for Facial Expression Quality Assessment
Jun 25, 2026Shuchao Duan, Alan Whone, Hossein Rahmani +2Facial Expression Recognition Multimodal Evaluation
School of Computer Science University of Bristol Bristol, UK · Translational Health Sciences University of Bristol Bristol, UK · School of Computing and Communications Lancaster University Lancaster, UK
Aligning MusicLLM with Emotion using Instruction Tuning and Feedback-Driven Alignment
Jun 23, 2026Takuya Hasumi, Welly NaptaliLarge Language Model Alignment Instruction Tuning
LY Corporation
LoopCoder-v2: Only Loop Once for Efficient Test-Time Computation Scaling
Jun 16, 2026Jian Yang, Shawn Guo, Wei Zhang +16Code Optimization Loop
1Beihang University · 2IQuest Research · 4Renming University of China +1
VoidPadding: Let [VOID] Handle Padding in Masked Diffusion Language Models so that [EOS] Can Focus on Semantic Termination
Jun 16, 2026Chunyu Liu, Zhengyang Fan, Kaisen Yang +1Masked Diffusion Language Models Decoding
Tsinghua University
From Tokens to Regions: CUDA-Sensitive Instruction Tuning for GPU Kernel Generation
Jun 15, 2026Wentao Chen, Jiace Zhu, Xing Zhe Chai +4Cuda Graphics Processing Unit Kernels
1Shanghai Jiao Tong University · 2Biren Technology
Steering Where to Listen: Instruction-Based Activation Steering Redirects Temporal Attention in Large Audio-Language Models
Jun 9, 2026Tsung-En Lin, Hung-Yi LeeLarge Audio Language Models Linear Activation Steering
National Taiwan University, Taipei, Taiwan · NTU Artificial Intelligence Center of Research Excellence (NTU AI-CoRE)
Decomposing Factual Sycophancy in Language Models: How Size and Instruction Tuning Shape Robustness
Jun 4, 2026Victor De Marez, Luna De Bruyne, Walter DaelemansSycophancy Instruction-Tuned Models
Centre for Computational Linguistics, Psycholinguistics and Sociolinguistics University of Antwerp Antwerp, Belgium
Visual Instruction Tuning Aligns Modalities through Abstraction
Jun 2, 2026Luis Palacios, Lorenzo Basile, Diego Doimo +1Multimodal Continual Instruction Tuning Instruction Tuning
Area Science Park, Trieste, Italy
Large Language Models Are Overconfident in Their Own Responses
Jun 2, 2026Mario Sanz-Guerrero, Manuel Mager, Katharina von der WenseInstruction-Tuned Models Overconfidence
∇Johannes Gutenberg University Mainz, Germany · ♠University of Colorado Boulder, USA
A unified multi-task framework enables interpretable chest radiograph analysis
Jun 2, 2026Lijian Xu, Ziyu Ni, Xinglong Liu +3Radiology Report Generation Chest X-Ray
Shenzhen University of Advanced Technology · the Chinese University of Hong Kong, Hong Kong · Lead contact +4
ProtoAda: Prototype-Guided Adaptive Adapter Expansion and Geometric Consolidation for Multimodal Continual Instruction Tuning
Jun 1, 2026Yu-Cheng Shi, Zhen-Hao Xie, Jun-Tao Tang +1Multimodal Continual Instruction Tuning Vision-Language Model Adaptation
School of Artificial Intelligence, Nanjing University, China · State Key Laboratory of Novel Software Technology, Nanjing University, China
CRAM: Centroid-Routing and Adaptive MoE for Multimodal Continual Instruction Tuning
Jun 1, 2026Jun-Tao Tang, Zhen-Hao Xie, Yu-Cheng Shi +1Multimodal Continual Instruction Tuning Instruction Tuning
School of Artificial Intelligence, Nanjing University, China · State Key Laboratory of Novel Software Technology, Nanjing University, China
Decentralized Instruction Tuning: Conflict-Aware Splitting and Weight Merging
Jun 1, 2026Minsik Choi, Geewook KimInstruction Tuning
NAVER Cloud AI · Korea University · KAIST AI
MADS: Model-Aware Diverse Core Set Selection for Instruction Tuning
May 29, 2026Yi Bai, Wenhao Zhang, Yao Chen +3Instruction Tuning Model Fine-Tuning
Shandong University, Qingdao, China · Inspurcloud, Jinan, China
Beyond Bilingual Transfer: Multilingual Code-Switching in Instruction Tuning
May 28, 2026Shunta Asano, Jeonghun Baek, Toshihiko YamasakiCross-Lingual Transfer Multilingual
The University of Tokyo
Prism: A Plug-in Reproducible Infrastructure for Scalable Multimodal Continual Instruction Tuning
May 25, 2026Jun-Tao Tang, Yu-Cheng Shi, Zhen-Hao Xie +1Multimodal Continual Instruction Tuning Instruction Tuning
National Key Laboratory for Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China
BeLink: Biomedical Entity Linking Meets Generative Re-Ranking
May 21, 2026Darya Shlyk, Stefano Montanelli, Lawrence HunterMedical Ontology Cross-Encoder Reranking
University of Milan Milan, Italy · University of Chicago Chicago, IL, USA
DarkLLM: Learning Language-Driven Adversarial Attacks with Large Language Models
May 15, 2026Ye Sun, Xin Wang, Jiaming Zhang +7Black-Box Adversarial Attacks White-Box Spectral-Subspace-Guided Attack
Fudan University · Nanyang Technological University · Tongji University
SLAP: Stratified Loss-based Pruning for On-Policy Data-Efficient Instruction Tuning
May 13, 2026Run Zou, Jianhang Ding, Yifan Ding +3Instruction Tuning Alignment-Aware Selection
Alibaba International Digital Commerce Group, HangZhou, China · Hangzhou Lingju Intelligence AI Lab, Hangzhou, China · Hangzhou Dianzi University, HangZhou, China
A Unified Graph Language Model for Multi-Domain Multi-Task Graph Alignment Instruction Tuning
May 12, 2026Haibo Chen, Xin Wang, Jiaheng Chao +2Gnn-Based Detectors Instruction Tuning
Tsinghua University
Dynamic Cross-Modal Prompt Generation for Multimodal Continual Instruction Tuning
May 11, 2026Tao Hu, Da-Wei ZhouMultimodal Continual Instruction Tuning Instruction Tuning
School of Artificial Intelligence, Nanjing University · State Key Laboratory for Novel Software Technology, Nanjing University
Let the Target Select for Itself: Data Selection via Target-Aligned Paths
May 10, 2026Huitao Yang, Hengzhi He, Tung Sum Thomas Kwok +1Alignment-Aware Selection Adaptive Sampling
University of California, Los Angeles
Instruction Tuning Changes How Upstream State Conditions Late Readout: A Cross-Patching Diagnostic
May 8, 2026Yifan ZhouInstruction Tuning Instruction-Tuned Models
University of California, Los Angeles
Decomposing the Basic Abilities of Large Language Models: Mitigating Cross-Task Interference in Multi-Task Instruct-Tuning
May 7, 2026Bing Wang, Ximing Li, Changchun Li +3Instruction Tuning Cross-Task Interference
College of Computer Science and Technology, Jilin University · Key Laboratory of Symbolic Computation and Knowledge Engineering, Ministry of Education, Jilin University · RIKEN Center for Advanced Intelligence Project +1
Open-Source Image Editing Models Are Zero-Shot Vision Learners
May 6, 2026Wei Liu, Jiaxin Lin, Rui ChenInstruction Tuning
Tencent Inc.
Attention Is Where You Attack
Apr 30, 2026Aviral Srivastava, Sourav PandaAdversarial Examples Jailbreak Attacks
Amazon · The Pennsylvania State University
What Makes Good Instruction-Tuning Data? An In-Context Learning Perspective
Apr 28, 2026Guangzeng Han, Xiaolei HuangInstruction Tuning Multimodal Continual Instruction Tuning
University of Memphis
VisNec: Measuring and Leveraging Visual Necessity for Multimodal Instruction Tuning
Mar 1, 2026Mingkang Dong, Hongyi Cai, Jie Li +4Multimodal Continual Instruction Tuning Instruction Tuning
SJTU · Universiti Malaya · CMU +3
Multi-Task Instruction Tuning via Data Scheduling for Low-Resource Arabic SpeechLLMs
Jan 18, 2026Hunzalah Hassan Bhatti, Firoj Alam, Shammur Absar ChowdhuryArabic Natural Language Processing Arabic
Qatar Computing Research Institute, Qatar
SecureCode: A Production-Grade Multi-Turn Dataset for Training Security-Aware Code Generation Models
Dec 20, 2025Scott ThorntonVulnerable Code Security Evaluation
DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collections
Aug 16, 2025Haebin Shin, Lei Ji, Xiao Liu +4Multimodal Continual Instruction Tuning Instruction Tuning
University of Michigan · Microsoft Research · KAIST AI +1
In-context Learning vs. Instruction Tuning: The Case of Small and Multilingual Language Models
Mar 3, 2025David Ponce, Thierry EtchegoyhenInstruction Tuning In-Context Learning
Vicomtech Foundation, Basque Research and Technology Alliance (BRTA) · University of the Basque Country - EHU
How LLMs Follow Instructions: Skillful Coordination, Not a Universal Mechanism
Date pendingElisabetta Rocchetti, Alfio FerraraInstruction Tuning Instruction-Tuned Models
Department of Computer Science, Università degli Studi di Milano

Sep 30, 2026, cs.AIJ/K move · Enter open · S save

A helps B while B hurts A: directed transfer in instruction-tuning mixture

Nima H. Siboni, Vahid Rostami

Juna.ai, Kastanienallee 32, 10435 Berlin, Germany · Computational Systems Neuroscience, Institute of Zoology, University of Cologne, Germany

Instruction Tuning Mixtures

Open paper AskPDF arXiv

Adapting a language model to a specialized corpus means choosing which instruction-tuning tasks to train on under a fixed budget, and testing one choice costs a fine-tuning run. Common heuristics add more source tasks or pick sources similar to the target. The first assumes transfer is never negative; the second, that it is symmetric. We show that both assumptions fail: task

A

can help task

B

while

B

hurts

A

, so helpfulness is a signed property of ordered source--target pairs. We introduce the transfer map, a signed estimate of how much each source helps or hurts each held-out target. We fit the map in hundreds of fine-tuning runs on Qwen3 and Mistral models from 0.6B to 32B parameters, with all sources drawn from one corpus and no training examples from the target. The map predicts a held-out target's accuracy on unseen mixtures: recorded before those runs, its predictions have less than half the error of a mixture-agnostic baseline. The map is specific to its target and corpus but transfers across model scale: a mixture selected in advance at one size beats training on all source tasks at every other size we tested. Transfer is thus a property of the data. The map selects the tasks that help and drops the one that interferes: accuracy on the reasoning targets (causal explanation, multi-hop questions and methodological critique) rises by up to 14 percentage points over training on all source tasks.

Momentum

Related topics

Latest papers 48