cs.LGNov 7, 2026

From Task Affinities to Optimal Allocations: Probabilistic Graphical Modelling for Supervised Finetuning

Authors: Prateek Chanda, Saral Sureka, Krishnateja Killamsetty, Ganesh Ramakrishnan

Organizations: Indian Institute of Technology Bombay (cse.iitb.ac.in) · Amazon (amazon.com)

Abstract

Supervised fine-tuning over a large pool of heterogeneous tasks is often computationally expensive, leading practitioners to train language models on a fixed training budget of carefully chosen SFT tasks. Thus, achieving strong downstream performance hinges on selecting an effective subset of tasks, or equivalently, identifying an optimal task mixture under a fixed budget constraint. In practice, supervised finetuning mixtures are often determined using simple heuristics (e.g., uniform or size-proportional sampling) that ignore task interactions, which can hurt transfer and waste budget on redundant task sources. We introduce TASKMIXPGM, a framework for learning continuous task mixtures through energy-based modelling over SFT tasks. Each task forms a node in a Markov random field, with unary potentials capturing task utility and pairwise potentials modelling inter-task relationships via behavioral divergences between predictive distributions of single-task fine-tuned models. Optimizing this objective yields task mixtures that balance utility, coverage, and redundancy. We further show that the induced set function defined over the set of such tasks is weakly submodular under budget constraints, enabling approximation guarantees for discrete task selection. Empirically, we provide a fixed, benchmark-independent mixture that improves over heuristic mixing across multiple model families and different training budgets. TASKMIXPGM also uncovers interpretable structure in task interactions, revealing complementarities and redundancies that inform effective SFT mixtures.

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Jul 16, 2025

Learning Task Mixtures from Task Affinities: A Probabilistic Graphical Model for Supervised Fine-Tuning

Supervised fine-tuning performance for large language models depends strongly on how training budget is distributed across a heterogeneous set of tasks. In practice, mixtures are often fixed using simple heuristics (e.g., uniform or size-proportional sampling) that ignore task interactions, which can hurt transfer and waste budget on redundant sources. We introduce TaskPGM, a framework for learning continuous task mixtures via an energy-based model over tasks. Tasks form the nodes of a Markov random field: unary potentials capture per-task utility, and pairwise potentials encode inter-task relationships using behavioral divergences computed from predictive distributions of single-task fine-tuned models (e.g., Jensen--Shannon divergence and pointwise mutual information). Optimizing this objective yields mixtures that balance coverage against redundancy. We show that the resulting set function is weakly submodular under budget constraints, enabling approximation guarantees for discrete selection variants. Across multiple model families (LLaMA-7B, Qwen2-7B) and evaluation suites (BIG-Bench Hard), TaskPGM improves over standard mixing strategies and provides interpretable structure over task interactions.
Mar 23, 2026

mSFT: Addressing Dataset Mixtures Overfitting Heterogeneously in Multi-task SFT

Current language model training commonly applies multi-task Supervised Fine-Tuning (SFT) using a homogeneous compute budget across all sub-datasets. This approach is fundamentally sub-optimal: heterogeneous learning dynamics cause faster-learning tasks to overfit early while slower ones remain under-fitted. To address this, we introduce mSFT, an iterative, overfitting-aware search algorithm for multi-task data mixtures. mSFT trains the model on an active mixture, identifies and excludes the earliest overfitting sub-dataset, and reverts to that specific optimal checkpoint before continuing. Extensive evaluations demonstrate that mSFT consistently outperforms 4 baselines across 10 benchmarks and 6 base models. Further analysis confirms mSFT maintains robust gains across diverse dataset sizes, task granularities, and is insensitive to its single new hyperparameter (compute budget). Notably, at low compute budget, mSFT can improve performance while lowering training FLOPs. Ultimately, mSFT establishes a practical overfitting-aware algorithm for multi-task SFT that maximizes the potential of models across diverse data mixtures.
Sep 30, 2026

A helps B while B hurts A: directed transfer in instruction-tuning mixture

Adapting a language model to a specialized corpus means choosing which instruction-tuning tasks to train on under a fixed budget, and testing one choice costs a fine-tuning run. Common heuristics add more source tasks or pick sources similar to the target. The first assumes transfer is never negative; the second, that it is symmetric. We show that both assumptions fail: task AA can help task BB while BB hurts AA, so helpfulness is a signed property of ordered source--target pairs. We introduce the transfer map, a signed estimate of how much each source helps or hurts each held-out target. We fit the map in hundreds of fine-tuning runs on Qwen3 and Mistral models from 0.6B to 32B parameters, with all sources drawn from one corpus and no training examples from the target. The map predicts a held-out target's accuracy on unseen mixtures: recorded before those runs, its predictions have less than half the error of a mixture-agnostic baseline. The map is specific to its target and corpus but transfers across model scale: a mixture selected in advance at one size beats training on all source tasks at every other size we tested. Transfer is thus a property of the data. The map selects the tasks that help and drops the one that interferes: accuracy on the reasoning targets (causal explanation, multi-hop questions and methodological critique) rises by up to 14 percentage points over training on all source tasks.