Why Transformers Struggle with Distribution-Independent In-Context Learning
Organizations: Institut de Recherche en Informatique de Toulouse; University of Toulouse; Toulouse, France · Toulouse School of Economics; University of Toulouse Capitole; Toulouse, France · Institut de Recherche en Informatique de Toulouse; Centre Nationale de la Recherche Scientifique; Toulouse, France
Abstract
With in-context learning (ICL), transformer models adapt to new tasks from illustrative examples at inference time without updating their parameters. A common interpretation is that transformers implement implicit learning algorithms in their forward pass. We test this interpretation and ICL capability under distribution shift in noiseless polynomial regression, where the context uniquely determines the target function. Models that are near-perfect in distribution exhibit sharp performance degradation under shifts in the input or target-function distribution. This behavior persists across polynomial degrees and multiple training interventions, and in higher-dimensional nonlinear regression. We identify two architectural mechanisms that explain this breakdown: output saturation at finite boundary values linked to final normalization, and attention concentration on a small subset of the context, which can limit context-dependent adaptation required for ICL. Controlled experiments, architectural ablations, targeted interventions, and theoretical analysis provide strong evidence for both mechanisms and their consequence: strong in-distribution ICL does not imply distribution-independent ICL, which calls for caution when using ICL instead of explicit retraining or adaptation under distribution shift.
Figures & tables
| Transformer | ||||||
|---|---|---|---|---|---|---|
| NW (RBF) | ||||||
| KRR (RBF) | ||||||
| KRR (Poly) |
| Final norm | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Norm removed at test time | ||||||||||
| No final norm (retrained) |
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
| Deg | ||||||||||
| 1 | ||||||||||
| 2 | ||||||||||
| 3 | ||||||||||
| 4 | ||||||||||
| 5 | ||||||||||
| 6 |
| Deg | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 2.05 | 0.02 | 0.15 | 0.27 | 0.37 | 0.44 | 0.52 | 0.56 | 0.61 | 0.65 |
| 2 | 7.62 | 0.02 | 0.13 | 0.27 | 0.37 | 0.46 | 0.54 | 0.59 | 0.64 | 0.68 |
| 3 | 2.53 | 0.01 | 0.07 | 0.16 | 0.24 | 0.31 | 0.39 | 0.43 | 0.49 | 0.53 |
| 4 | 1.35 | 0.01 | 0.08 | 0.16 | 0.24 | 0.33 | 0.39 | 0.43 | 0.48 | 0.53 |
| 5 | 2.32 | 0.01 | 0.06 | 0.15 | 0.24 | 0.32 | 0.39 | 0.43 | 0.50 | 0.56 |
| 6 | 2.22 | 0.01 | 0.07 | 0.17 | 0.28 | 0.37 | 0.43 | 0.49 | 0.55 | 0.60 |
| Deg | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | 0.00 | 0.03 | 0.83 | 2.12 | 6.26 | 7.60 | 14.15 | 18.07 | 27.66 | 39.17 |
| 2 | 0.00 | 0.04 | 1.07 | 2.76 | 7.23 | 10.69 | 15.70 | 20.25 | 28.85 | 44.72 |
| 3 | 0.00 | 0.05 | 0.86 | 2.25 | 6.37 | 11.44 | 15.05 | 19.77 | 29.68 | 46.47 |
| models | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| 0.04 | 0.62 | 1.95 | 4.29 | 7.14 | 10.36 | 15.79 | 24.48 | 27.06 | ||
| 0.11 | 1.16 | 3.33 | 5.08 | 10.19 | 12.56 | 18.67 | 23.38 | 32.98 | 44.91 | |
| 0.01 | 0.75 | 2.71 | 4.48 | 9.84 | 11.59 | 17.97 | 22.22 | 31.80 | 43.53 |
| models | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| 0.04 | 0.62 | 1.95 | 4.29 | 7.14 | 10.36 | 15.79 | 24.48 | 27.06 | ||
| 0.01 | 0.01 | 0.02 | 0.03 | 0.03 | 0.05 | 0.12 | 0.27 | 0.75 | 1.61 | |
| 0.13 | 0.15 | 0.17 | 0.2 | 0.26 | 0.26 | 0.32 | 0.35 | 0.41 | 0.49 | |
| 2217.84 | 2373.82 | 2494.31 | 2526.93 | 2472.45 | 2467.52 | 2317.92 | 2232.03 | 2129.0 | 2092.81 |
| Degree | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | ||||||||||
| 2 | ||||||||||
| 3 | ||||||||||
| 4 | ||||||||||
| 5 | ||||||||||
| 6 |
| Degree | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| 1 | ||||||||||
| 2 | ||||||||||
| 3 | ||||||||||
| 4 | ||||||||||
| 5 | ||||||||||
| 6 |
| Degree | ||
|---|---|---|
| 1 | ||
| 2 | ||
| 3 | ||
| 4 | ||
| 5 | ||
| 6 |
| Baseline | 0.04 | 0.62 | 1.95 | 4.29 | 7.14 | 10.36 | 15.79 | 24.48 | 27.06 | |
|---|---|---|---|---|---|---|---|---|---|---|
| Label-normalized | 0.10 | 0.45 | 0.99 | 1.76 | 2.72 | 3.60 | 5.49 | 7.09 | 8.76 | 11.32 |
| Baseline | ||||||
|---|---|---|---|---|---|---|
| Post-hoc | ||||||
| Post-hoc | ||||||
| Post-hoc | ||||||
| Frozen (retrained) | ||||||
| Large init (learned) |
| Degree | Temperature | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | |||||||||||
| 2 | |||||||||||
| models | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
|---|---|---|---|---|---|---|---|---|---|---|
| Softmax | 0.04 | 0.62 | 1.95 | 4.29 | 7.14 | 10.36 | 15.79 | 24.48 | 27.06 | |
| Linear Attention | 0.03 | 0.53 | 1.66 | 3.47 | 6.26 | 11.46 | 15.30 | 23.13 | 29.71 | |
| Sparsemax | 0.06 | 0.68 | 2.24 | 4.57 | 7.62 | 13.43 | 18.12 | 24.76 | 32.93 | |
| 1.5-Entmax | 0.07 | 0.76 | 2.17 | 4.29 | 6.78 | 11.66 | 15.61 | 21.22 | 28.06 | |
| Temperature Softmax | 0.07 | 0.83 | 2.45 | 4.81 | 7.54 | 12.89 | 17.05 | 22.91 | 30.41 | |
| CosFormer | 0.05 | 0.77 | 2.58 | 5.56 | 9.21 | 16.14 | 21.19 | 28.53 | 37.50 |
| Model | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Set regressor (CNP) | ||||||||||
| Transformer–ridge hybrid |
In-Context Learning
In-context learning studies how models adapt to new tasks from examples or instructions supplied at inference time, without changing their weights.
See all 429 papers