What Travels Across Languages? Reading, Transfer and Erasure of Cross-Lingual Concept Subspaces in Bangla Idioms
Organizations: Systems and Software Lab (SSL) Department of Computer Science and Engineering Islamic University of Technology, Gazipur, Bangladesh
Abstract
Cross-lingual concept analysis requires distinguishing the information that a subspace retains from the classifiers and erasers that transfer through it. We study these operations in six frozen encoders using 10,359 Bangla (Bengali) idiom senses, eight input conditions, and figurative-class and sentiment labels. Class-mean directions provide the most consistent evidence of shared coarse meaning: in post hoc analyses, they outperform random-subspace, permuted-label and covariance-matched nulls in all 12 model-label tests under balanced accuracy. A prospective internal re-split supports idiom-alone sharing in 10 of 12 tests, spanning five encoders. The original confirmatory logistic-regression family rejects both planned nulls in 4 of 12 tests, where English directions gain only a few accuracy points above the majority-class floor (3.0-7.3). Classifier transfer depends on the input map and regularization: English-trained class-mean readouts beat both zero-shot nulls in 8 of 12 discovery tests, and stronger source-selected penalties improve transfer through logistic and LEACE directions. These readouts use Bangla-validation-selected layers and unlabeled target preprocessing. Input controls show greater coarse-class sharing for explicit glosses than for idiom counterparts. Reading, however, does not imply effective erasure: English-fitted LEACE removes less than one percentage point of Bangla figurative-class accuracy, compared with 4-11 points for Bangla-fitted LEACE; INLP shows a similar asymmetry. Pattern-filter analysis, covariance-weighted input maps and a binary Gaussian overlap bound explain how these operations can diverge. The analogous multiclass ordering is an empirical diagnostic, observed in 11 of 12 cells. The results support an estimator-sensitive account of cross-lingual concept sharing and motivate evaluating reading, classifier transfer, erasure and overlap separately.
Figures & tables
| Estimator | Kind | Image | Input map | Role here |
|---|---|---|---|---|
| RAND | null | random orthonormal directions | orthogonal | RAND null; chance overlap ( Proposition 4 ) |
| PCA | unsupervised | leading principal components | orthogonal | label-free baseline |
| MLR | filter | logistic-regression weight rows | orthogonal | planned estimator; tends to COV as ( Proposition 1 ) |
| LDA | filter | discriminant directions | orthogonal | a second filter |
| CPCA | pattern | principal components of class centroids | orthogonal | check on COV; same image at |
| COV | pattern | class-mean directions, from | orthogonal | pattern estimator; – ; portable ( Proposition 2 ) |
| Condition | Input | Pool |
|---|---|---|
| bn-fig | ‘He is already dreaming of a new car… .’ | idiom |
| bn-context | same sentence, idiom [MASK] | context |
| bn-sentence | same sentence | sentence |
| bn-idiom | ‘Here the expression “ ” is used.’ | idiom |
| bn-para | ‘Arranging enjoyment before attainment…’ | gloss |
| en-idiom | ‘Here the expression “Counting one’s chickens before they hatch” is used.’ | span |
| Selected layer | |||||||
|---|---|---|---|---|---|---|---|
| Backbone | Cross-lingual signal | Params (M) | Width | Blocks | Pinned revision | Discovery | Seed 1 |
| BanglaBERT | Bangla only | 110.0 | 768 | 12 | 9ce791f33057 | 9 | 6 |
| BanglishBERT | Bangla and English | 110.0 | 768 | 12 | 88f2777acd65 | 12 | 9 |
| mBERT | monolingual corpora | 177.9 | 768 | 12 | 3f076fdb1ab6 | 12 | 9 |
| XLM-R | monolingual corpora | 559.9 | 1024 | 24 | c23d21b0620b | 24 | 15 |
| MuRIL | translation pairs | 505.9 | 1024 | 24 | ace319f0d175 | 12 | 15 |
| Planned test | Balanced acc. | Patterns | Seed 1 | Zero-shot | Idiom alone | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Model | F [95% CI] | S [95% CI] | F | S | F | S | F | S | F | S | F | S |
| BanglaBERT | .16 [.03, .28] | .15 [.00, .29] | .33 | .15 | .48 | .62 | .66 | .56 | .32 | .53 | .76 | .54 |
| BanglishBERT | .20 [ .06, .43] | .38 [.15, .58] | .47 | .37 | .33 | .88 | .73 | .84 | .04 | .68 | 1.00 | 1.09 |
| mBERT | .20 [ .14, .47] | .40 [.06, .77] | .50 | .36 | .32 | .28 | .61 | .58 | .09 | .24 | .85 | .59 |
| XLM-R | .29 [.10, .46] | .47 [.29, .65] | .47 | .43 | .31 | .38 | .95 | 1.22 | .13 | .31 | 1.16 | 1.75 |
| MuRIL | .48 [.34, .62] | .32 [.11, .52] | .59 | .42 | .82 | .92 | .89 | .91 | .76 | .93 | .96 | 1.12 |
Cross-Lingual Transfer
Cross-lingual transfer enables models to apply knowledge or capabilities learned in one language to tasks in another.
See all 268 papers