Adapting pretrained Transformers to downstream tasks produces task-specific updates that are expensive to reproduce for every model variant. Reusing these updates across different depths and widths is challenging because their parameters have incompatible shapes and do not share a canonical coordinate system. We introduce Ariadne, a training-free method that transfers task-specific behavior across heterogeneous Transformers without resizing the source model or transporting its parameters. Rather than treating the source task vector, i.e. the difference in parameter space between the fine-tuned and the base models, as the object to move, Ariadne characterizes fine-tuning by the activation changes it induces at matched blocks. First, it aligns these effects to a target base through local Procrustes maps. Then, it directly synthesizes a native target update by closed-form ridge regression. Across Transformer expansion, reduction and language-model transfer, Ariadne improves target-model performance while substantially reducing the time and memory required to construct an update relative to previous cross-architecture rebasin approaches. These results show that local functional effects provide an efficient alternative to parameter transport.
Figures & tables
Figure 1: Ariadne transfers the functional effect of source fine-tuning to the target model in activation space. Source ⚫ and target ⚫ base activations are aligned using a component-wise Procrustes transformation Qj . The source fine-tuning effect ⚫ is mapped into target coordinates to obtain the desired activation shift Dj , giving the aligned fine-tuning displacement. Ridge regression converts this desired shift into target-native parameter updates ◼ .
Table 1: Task-vector transfer across transformer depth changes. We report ViT-B/16 → ViT-L/14 and ViT-L/14 → ViT-B/16 for B∈{1,10} calibration batches. Time is the median phase-total runtime per task, summing activation collection, transformation, and transport.