cs.CLJun 20, 2026

Are Multilingual Models Actually Improving? Isolating True Cross-Lingual Transfer

Authors: Prasoon Bajpai, Eleftheria Briakou, Colin Cherry, Preethi Jyothi, Vihari Piratla

Abstract

Cross-lingual transfer is a model's ability to generalize capabilities from well-represented source languages to under-represented target languages. Existing measures of a model's transfer strength conflate improvements in transfer with general improvements to accuracy in the source language. We advocate for an alternate metric that reliably captures transfer strength called Hardness Adjusted Transfer (HAT) Score, and use it to derive multiple insights on factors influencing transfer strength. Our analysis across twenty diverse language models and three popular mainstream multilingual benchmarks argues that 1) transfer in small models is not broken, 2) we are making slower than expected progress in cross-lingual transfer with model size, and 3) we have made clear progress over time.

Explore similar work

CardsList
  1. Apples to Apples? Towards Comparable Crosslingual Language Model Evaluation

    Aug 25, 2026Xiulin Yang, Ethan Gotlieb Wilcox, Catherine ArnettLinguisticsLikelihood