The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment

Rishab Balasubramanian; Pin-Jie Lin; Rituraj Sharma; Anjie Fang; Fardin Abdi; Viktor Rozgic; Zheng Du; Mohit Bansal

cs.LGApr 7, 2026

The Master Key Hypothesis: Unlocking Cross-Model Capability Transfer via Linear Subspace Alignment

Authors: Rishab Balasubramanian, Pin-Jie Lin, Rituraj Sharma, Anjie Fang, Fardin Abdi, Viktor Rozgic, Zheng Du, Mohit Bansal,

Organizations: Virginia Tech · Amazon · UNC Chapel Hill

Abstract

We investigate whether post-trained capabilities can be transferred across models without retraining, with a focus on transfer across different model scales. We propose the Master Key Hypothesis, which states that model capabilities correspond to directions in a low-dimensional latent subspace that induce specific behaviors and are transferable across models through linear alignment. Based on this hypothesis, we introduce UNLOCK, a training-free and label-free framework that extracts a capability direction by contrasting activations between capability-present and capability-absent Source variants, aligns it with a Target model through a low-rank linear transformation, and applies it at inference time to elicit the behavior. Experiments on reasoning behaviors, including Chain-of-Thought (CoT) and mathematical reasoning, demonstrate substantial improvements across model scales without training. For example, transferring CoT reasoning from Qwen1.5-14B to Qwen1.5-7B yields an accuracy gain of 12.1% on MATH, and transferring a mathematical reasoning direction from Qwen3-4B-Base to Qwen3-14B-Base improves AGIEval Math accuracy from 61.1% to 71.3%, surpassing the 67.8% achieved by the 14B post-trained model. Our analysis shows that the success of transfer depends on the capabilities learned during pre-training, and that our intervention amplifies latent capabilities by sharpening the output distribution toward successful reasoning trajectories.

Figures & tables

Panel of Figure 1 — Figure 1: Performance improvements from Unlock when transferring (a) Chain-of-Thought capabilities from Qwen1.5-14B onto Qwen1.5-7B ; and (b) Math reasoning capabilities from Qwen3-4B-Base onto Qwen3-14B-Base . Capability transfer substantially improves the base model without additional training , approaching the performance of the post-trained model.

Figure 2: Illustration of Unlock : Our method consists of three stages: (1) Calculating the difference in hidden states in the Source space; (2) Learning a linear transformation between the Source Locked and Target Locked models; and (3) Projecting the MasterKey from Source to Target space and applying as a test-time intervention to the residual stream at every layer.

Model	Prompt	$\,\mathcal{T}^{\vphantom{*}}_{\mathrm{L}}\,$	$\,\mathcal{S}^{\vphantom{}}_{\mathrm{U}}\,\equiv\,\mathcal{S}^{\vphantom{}}_{\mathrm{L}}\,$	GSM8K	MATH	SVAMP
Qwen1.5	Direct	7B	–	9.2	8.0	44.0
		14B	–	16.0	16.0	58.3
	CoT	7B	–	64.4	17.9	73.0
		14B	–	77.3	26.8	79.0
	Direct	7B	+ Unlock ${}_{\text{from 14B}}$	56.0	20.1	70.3
		14B	+ Unlock ${}_{\text{from 7B}}$	74.4	31.2	78.3

Table 1: Chain-of-Thought Capability Transfer. Transfer performance across model families. Accuracies of base model with Direct prompting (

\,\mathcal{T}^{\vphantom{*}}_{\mathrm{L}}\,

) and base model with explicit CoT prompting shown in gray.

Figure 3: Comparison in Generation Length between

\,\mathcal{T}^{\vphantom{*}}_{\mathrm{L}}\,

and

\,\mathcal{T}^{\vphantom{*}}_{\mathrm{U}}\,

: A consistent increase in generation length is observed post-transfer, representative of reasoning behavior.

Model	$\,\mathcal{T}^{\vphantom{*}}_{\mathrm{L}}\,$	$\,\mathcal{S}^{\vphantom{}}_{\mathrm{U}}\,\equiv\mathcal{I}(\,\mathcal{S}^{\vphantom{}}_{\mathrm{L}}\,)$	AGIEval Math	Deepmind Math	Minerva Math	Olympiad Bench
Qwen3	4B	–	52.3	71.3	27.5	19.7
	14B	–	61.1	78.8	34.7	29.0
	$\mathcal{I}$ (4B)	–	75.6	88.4	31.5	39.8
	$\mathcal{I}$ (14B)	–	67.8	80.1	27.9	37.8
	4B	+ Unlock ${}_{\text{from 14B}}$	58.9	75.8	27.0	26.4
	14B	+ Unlock ${}_{\text{from 4B}}$	64.1	79.9	31.5	35.4

Table 2: Math Reasoning Transfer Results: Performance of Unlock across model families. For simplicity, we use

\mathcal{I}(x)

to denote the instruction-tuned version of the corresponding model.

Panel of Figure 4 — Figure 4: Statistics of First Generated Word: The output distribution is significantly skewed to a minimal set of starting traces post-steering.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Transfer Space	No Labeled Data	Fixed Compute	Transferrable Across Sizes	Extrinsic Evaluations
Task Vectors [ Ilharco et al., 2023 ]	Weight	✓	✓	✗	✓
Knowledge Distillation [ Gu et al., 2025 ]	Weight	✗	✓	✓	✓
Proxy Tuning [ Liu et al., 2024a ]	Logit	✓	✗	✓	✓
Steering Vectors [ Panickssery et al., 2024 ]	Latent	✗	✓	✗	✗
Patchscopes [ Ghandeharioun et al., 2024 ]	Latent	✗	✓	✓	✗
Activation Intervention [ Oozeer et al., 2025 ]	Latent	✗	✓	✓	✗

Appendix

Table 3: Comparison of our method to other approaches: Unlock is completely training-free and label-free, and shows improvements across model scales and architectures for extrinsic evaluation tasks.

Panel of Figure 5 — Figure 5: Example Prompts: Example CoT (green) and Direct (red) prompts which are used for all evaluations.

Panel of Figure 6 — Figure 6: Increased Generation Length of $\,\mathcal{T}^{\vphantom{*}}_{\mathrm{U}}\,$ : Unlock leads to a clear increase in generation length over the base model with Direct prompting, matching the length of the instruction-tuned model with explicit CoT prompts.

Panel of Figure 7 — Figure 7: Evidence for Improved Reasoning: Generation length of the Unlocked model significantly increases over the Locked model, with a corresponding improvement in downstream performance.

Model	Prompt	$\,\mathcal{T}^{\vphantom{*}}_{\mathrm{L}}\,$	$\,\mathcal{S}^{\vphantom{}}_{\mathrm{U}}\,\equiv\,\mathcal{S}^{\vphantom{}}_{\mathrm{L}}\,$	GSM8K	MATH	SVAMP
Qwen1.5	Direct	7B	–	9.2	8.0	44.0
		14B	–	16.0	16.0	58.3
	CoT	7B	–	64.4	17.9	73.0
		14B	–	77.3	26.8	79.0
	CoT	7B-Chat	–	58.1	18.2	69.7
		14B-Chat	–	74.8	30.2	82.0

Appendix

Table 4: Chain-of-thought Capability Transfer Results. Performance of Unlock across model families. Accuracies of the base model with Direct prompting (Locked model), base model with CoT prompting, and instruction-tuned model with CoT prompting shown in gray .

Panel of Figure 8 — Figure 8: Spectral Entropy of the Covariance Matrix: Increasing the number of examples leads to a corresponding increase in entropy — providing evidence that the MasterKey captures more information with additional examples.

Panel of Figure 9 — Figure 9: Convergence in performance of the linear transformation at low ranks: The normalized $\ell^{\vphantom{*}}_{2}$ error of the linear mapping as a function of number of samples $n$ with rank $k=4$ shows the diminishing impact of additional examples in rank-constrained settings.

Panel of Figure 10 — Figure 10: Overfitting of the linear transformation at high ranks: The normalized $\ell^{\vphantom{*}}_{2}$ error of the linear mapping as a function of rank $k$ with number of samples $n=512$ shows the overfitting of the transformation at high ranks.

Panel of Figure 11 — Figure 11: Representation Space with $\mathbf{\Phi}$ =Avg: Performance of $\texttt{OLMo-2-7B}\,+\,\text{{{Unlock}} }_{\mathrm{from\;}1B}$ (top) and $\texttt{OLMo-2-7B}\,+\,\text{{{Unlock}} }_{\mathrm{from\;}13B}$ (bottom) with the mean aggregator.

Panel of Figure 12 — Figure 12: Representation Space with $\mathbf{\Phi}$ =PCA: Performance of $\texttt{OLMo-2-7B}\,+\,\text{{{Unlock}} }_{\mathrm{from\;}1B}$ (top) and $\texttt{OLMo-2-7B}\,+\,\text{{{Unlock}} }_{\mathrm{from\;}13B}$ (bottom) with the principal component aggregator.

$\,\mathcal{T}^{\vphantom{*}}_{\mathrm{U}}\,$	Dataset	Agg. Method	n	k	$\alpha$
$\texttt{Qwen1.5-7B}\,+\,\text{{{Unlock}} }_{\mathrm{from\;}\texttt{14B}}$	GSM8K MATH SVAMP	Avg Avg PCA	512 64 64	64 64 16	0.1 0.05 0.1
$\texttt{Qwen1.5-14B}\,+\,\text{{{Unlock}} }_{\mathrm{from\;}\texttt{7B}}$	GSM8K MATH SVAMP	PCA PCA PCA	128 512 128	4 512 128	0.1 0.1 0.1
$\texttt{OLMo-2-1B}\,+\,\text{{{Unlock}} }_{\mathrm{from\;}\texttt{7B}}$	GSM8K MATH SVAMP	Avg Avg Avg	512 512 64	128 256 64	0.2 0.05 0.2
$\texttt{OLMo-2-7B}\,+\,\text{{{Unlock}} }_{\mathrm{from\;}\texttt{1B}}$	GSM8K MATH SVAMP	Avg Avg Avg	1024 128 256	1024 4 128	0.5 0.2 0.2
$\texttt{OLMo-2-7B}\,+\,\text{{{Unlock}} }_{\mathrm{from\;}\texttt{13B}}$	GSM8K MATH SVAMP	PCA Avg PCA	512 512 16	1 128 16	0.1 0.1 0.1
$\texttt{OLMo-2-13B}\,+\,\text{{{Unlock}} }_{\mathrm{from\;}\texttt{7B}}$	GSM8K MATH SVAMP	Avg PCA Avg	64 64 256	64 4 16	0.1 0.1 0.05

Appendix

Table 5: Optimal Hyperparameters for CoT Transfer: The absence of a universal optimal hyperparameter combination across models suggests misalignment in the underlying representation spaces.

Model	$\,\mathcal{T}^{\vphantom{*}}_{\mathrm{L}}\,$	$\,\mathcal{S}^{\vphantom{*}}_{\mathrm{L}}\,$	$\,\mathcal{S}^{\vphantom{*}}_{\mathrm{U}}\,$	AGI-M	D-M	M-M	OB
Qwen2.5	1.5B	–	–	35.9	45.3	10.8	9.9
	7B	–	–	48.2	67.7	22.5	20.8
	14B	–	–	52.2	70.7	18.5	20.0
	1.5B-Instruct	–	–	37.8	46.6	12.6	13.3
	7B-Instruct	–	–	54.7	71.9	27.5	26.1
	14B-Instruct	–	–	65.7	78.5	29.7	33.8

Appendix

Table 6: Mathematical Reasoning Transfer From Instruction-Tuned Unlocked Models

\,\mathcal{S}^{\vphantom{*}}_{\mathrm{U}}\,

: Performance of the baselines, task-conditioned , and task-agnostic transfer.

Model	$\,\mathcal{T}^{\vphantom{*}}_{\mathrm{L}}\,$	$\,\mathcal{S}^{\vphantom{*}}_{\mathrm{L}}\,$	$\,\mathcal{S}^{\vphantom{*}}_{\mathrm{U}}\,$	AGI-M	D-M	M-M	OB
Qwen2.5	7B	–	–	48.2	67.7	22.5	20.8
	14B	–	–	52.2	70.7	18.5	20.0
	7B-Instruct	–	–	54.7	71.9	27.5	26.1
	14B-Instruct	–	–	65.7	78.5	29.7	33.8
	Nemotron-14B	–	–	58.1	82.2	10.8	9.1
	DLER-R1-7B	–	–	80.7	88.6	40.5	50.2

Appendix

Table 7: Mathematical Reasoning Transfer From Math-specific Unlocked Models

\,\mathcal{S}^{\vphantom{*}}_{\mathrm{U}}\,

: Performance of the baselines, math post-trained models , task-conditioned , and task-agnostic transfer.

Panel of Figure 13 — Figure 13: Additional Statistics of the First-generated Word: A clear shift in the distribution of first-generated word is observed after applying Unlock

Panel of Figure 14 — Figure 14: Length to answer (left); and Number of repeating substrings (middle and right) for Qwen3 and Ministral-3 families.

Model	Prompt	$\,\mathcal{T}^{\vphantom{*}}_{\mathrm{L}}\,$	$\,\mathcal{S}^{\vphantom{}}_{\mathrm{U}}\,\equiv\mathcal{I}(\,\mathcal{S}^{\vphantom{}}_{\mathrm{L}}\,)$	GSM8K	MATH	SVAMP
gemma-2	Direct	9B	–	3.0	3.5	21.0
	CoT	9B	–	66.6	26.4	79.3
	CoT	9B-Instruct	–	87.6	43.5	85.3
	Direct	9B	14B	60.5	24.2	76.7
	Direct	9B	7B	43.7	26.3	72.3
OLMo-2	Direct	7B	–	10.0	9.7	43.7

Appendix

Table 8: Cross-Family CoT Transfer: Results when transferring Chain-of-Thought capabilities from the Qwen1.5 family onto gemma-2 and OLMo-2 families. For simplicity, we use

\mathcal{I}(x)

to denote the instruction-tuned version of the corresponding model. Baseline comparisons shown in gray .

Transfer Learning

Transfer learning studies how models reuse knowledge across tasks, domains, or modalities when target data or computation are limited.

See all 416 papers

Post-Training Leaves Behavioral Shadows on Unrelated Decisions
Sep 24, 2026Ziyang Zhang, Yubin Jing, Yuanhao Zeng +3Subliminal Learning Language Model Distillation
Peking University · Georgia Institute of Technology · ShanghaiTech University +2
Reasoning that Travels: Dissecting How Chain-of-Thought Transfers Across Models
May 27, 2026Xinyuan Cheng, Beiduo Chen, Philipp Mondorf +1LLM Prompting LLM Reasoning
UMaiNLP, Center for Information and Language Processing, LMU Munich, Germany · ÆMunich Center for Machine Learning, Germany
A Negative Result on Cross-Model Activation Transfer in a Pythia Multi-Hop Setting
Jun 2, 2026Peiyan Zhang, Jason XinCausal Interventions in Language Models Representation Alignment
Independent Researcher

Sep 24, 2026J/K move · Enter open · S save

Post-Training Leaves Behavioral Shadows on Unrelated Decisions

Ziyang Zhang, Yubin Jing, Yuanhao Zeng, Yuyao Li+2

Peking University · Georgia Institute of Technology · ShanghaiTech University+2

Subliminal Learning Language Model Distillation Language Model Post-Training

Open paper AskPDF arXiv

We find that language models can transfer capabilities through task-unrelated text. Post-training typically improves language models using task-specific data. Prior work on subliminal learning shows that information about these updates can pass through unrelated generations, but has largely focused on traits or preferences using extensive teacher outputs. We introduce Active Taskless Distillation (ATD), which achieves capability transfer using only a single word from the teacher per prompt. ATD probes the behavioral shadow of post-training by selecting prompts where the teacher and student's shared public ancestor is nearly indifferent between two ordinary words. A student initialized from this ancestor learns solely from the resulting prompt-word pairs, without target-task examples, teacher logits, or teacher parameters. In the primary coding experiment with Qwen2.5-1.5B, 5,664nses yield a 5.34 pp gain on HumanEval+ over an exact nuisance-matched control thadisrupts prompt-resperiments showtransfer in scientific knowledge, commonsense reasoning, and reading comprehensins across additional model generations, sizes, and families. Functional analyses show that the learned sid composable, andthat its strength tracks the teacher's update strength.

Abstract

Transfer Learning

Explore similar work

Post-Training Leaves Behavioral Shadows on Unrelated Decisions

Reasoning that Travels: Dissecting How Chain-of-Thought Transfers Across Models

A Negative Result on Cross-Model Activation Transfer in a Pythia Multi-Hop Setting