cs.LGOct 12, 2026

The Bell Boundary of Shared Representations

Authors: Yang Wu, Hanyu Zhao, Yazhou Zhang, Yuexian Hou

Organizations: Tianjin University, China

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

When does a shared representation become insufficient for jointly supporting multiple tasks? This question lies at the heart of multi-task learning models. Although task interference in shared representations has been widely observed, its onset, measurable form, and capacity boundary remain poorly understood. In this paper, we revisit this problem through the lens of Bell's theorem. We show that if four binary task outcomes can be jointly represented by a common shared representation, their correlations must satisfy the classical CHSH bound ∣S∣≤2|S|\leq2. We introduce NCnet, a controlled multi-task model for studying how representational capacity affects task compatibility. Experiments reveal a clear capacity-dependent transition. Once nontrivial task fitting emerges, constrained capacity can yield ∣S∣>2|S|>2. As capacity further increases, these exceedances progressively disappear and SS returns to the bounded region. This systematic pattern provides evidence that limited representational capacity is an important source of the observed incompatibility. We observe similar transitions across CNNs, PLMs, and LLMs, suggesting that the phenomenon extends beyond the controlled NCnet setting. We refer to this capacity-dependent transition as the Bell Boundary, providing a measurable diagnostic for representation bottlenecks, task incompatibility, and capacity allocation in multi-task models.

Figures & tables

Figure 1: Overview of the Bell Boundary framework. (1) NCnet establishes a correspondence between shared-representation learning and the Bell/CHSH setting. (2) Four independently trained task-pair models provide the correlations used to compute SNNS_{\mathrm{NN}} . (3) The analysis extends to CNNs, PLMs, and LLMs under four experimental settings. The bottom-right schematic illustrates CHSH exceedances in a critical capacity regime and their disappearance as capacity increases.
CHSH settingMulti-task learning
Hidden variable λ∼ρ(λ)\lambda\sim\rho(\lambda)Shared representation h=fθ(x)h=f_{\theta}(x) , h∼ρθ(h)h\sim\rho_{\theta}(h)
Measurement branches AA and BBTask families AA and BB
Measurement settings a1a_{1} , a2a_{2} and b1b_{1} , b2b_{2}Task settings a1a_{1} , a2a_{2} and b1b_{1} , b2b_{2}
Binary outcomes Ai(λ)A_{i}(\lambda) , Bj(λ)B_{j}(\lambda)Binary task outcomes gϕAi(h)g_{\phi_{A_{i}}}\left(h\right) , gϕBj(h)g_{\phi_{B_{j}}}\left(h\right)
Correlation EijE_{ij}Task correlation EijE_{ij}
CHSH statistic SSNeural CHSH statistic SNNS_{\mathrm{NN}}
Table 1: Correspondence between the CHSH setting and multi-task learning.
ddA1B1A_{1}B_{1}A1B2A_{1}B_{2}A2B1A_{2}B_{1}A2B2A_{2}B_{2}SNNS_{\mathrm{NN}}P0P_{0} (%)
EEAcc(Ai)\mathrm{Acc}(A_{i})Acc(Bj)\mathrm{Acc}(B_{j})EEAcc(Ai)\mathrm{Acc}(A_{i})Acc(Bj)\mathrm{Acc}(B_{j})EEAcc(Ai)\mathrm{Acc}(A_{i})Acc(Bj)\mathrm{Acc}(B_{j})EEAcc(Ai)\mathrm{Acc}(A_{i})Acc(Bj)\mathrm{Acc}(B_{j})
1−0.00\phantom{-}0.00 ±0.00\pm 0.000.720.72 ±0.25\pm 0.250.780.78 ±0.25\pm 0.25−0.02\phantom{-}0.02 ±0.07\pm 0.070.970.97 ±0.10\pm 0.100.520.52 ±0.07\pm 0.07−0.01\phantom{-}0.01 ±0.06\pm 0.060.520.52 ±0.06\pm 0.060.980.98 ±0.09\pm 0.09−0.06\phantom{-}0.06 ±0.06\pm 0.060.620.62 ±0.11\pm 0.110.640.64 ±0.11\pm 0.11−0.02-0.02 ±0.12\pm 0.1200
2−1.00\phantom{-}1.00 ±0.00\pm 0.001.001.00 ±0.00\pm 0.001.001.00 ±0.00\pm 0.00−0.48\phantom{-}0.48 ±0.11\pm 0.111.001.00 ±0.00\pm 0.000.740.74 ±0.05\pm 0.05−0.47\phantom{-}0.47 ±0.10\pm 0.100.740.74 ±0.05\pm 0.051.001.00 ±0.00\pm 0.00−0.22\phantom{-}0.22 ±0.20\pm 0.200.760.76 ±0.17\pm 0.170.760.76 ±0.17\pm 0.17−1.74\phantom{-}1.74 ±0.24\pm 0.2400
3−1.00\phantom{-}1.00 ±0.00\pm 0.001.001.00 ±0.00\pm 0.001.001.00 ±0.00\pm 0.00−0.91\phantom{-}0.91 ±0.21\pm 0.211.001.00 ±0.00\pm 0.000.950.95 ±0.10\pm 0.10−0.93\phantom{-}0.93 ±0.17\pm 0.170.960.96 ±0.09\pm 0.091.001.00 ±0.00\pm 0.00−0.53\phantom{-}0.53 ±0.24\pm 0.240.880.88 ±0.14\pm 0.140.860.86 ±0.15\pm 0.15−2.31\mathbf{\phantom{-}2.31} ±0.35\pm 0.3500
4−1.00\phantom{-}1.00 ±0.00\pm 0.001.001.00 ±0.00\pm 0.001.001.00 ±0.00\pm 0.00−0.99\phantom{-}0.99 ±0.07\pm 0.071.001.00 ±0.00\pm 0.000.990.99 ±0.04\pm 0.04−0.98\phantom{-}0.98 ±0.10\pm 0.100.990.99 ±0.05\pm 0.051.001.00 ±0.00\pm 0.00−0.93\phantom{-}0.93 ±0.11\pm 0.110.980.98 ±0.04\pm 0.040.980.98 ±0.04\pm 0.04−2.04\mathbf{\phantom{-}2.04} ±0.16\pm 0.165353
5−1.00\phantom{-}1.00 ±0.00\pm 0.001.001.00 ±0.00\pm 0.001.001.00 ±0.00\pm 0.00−1.00\phantom{-}1.00 ±0.00\pm 0.001.001.00 ±0.00\pm 0.001.001.00 ±0.00\pm 0.00−1.00\phantom{-}1.00 ±0.00\pm 0.001.001.00 ±0.00\pm 0.001.001.00 ±0.00\pm 0.00−0.99\phantom{-}0.99 ±0.05\pm 0.051.001.00 ±0.02\pm 0.021.001.00 ±0.02\pm 0.02−2.01\mathbf{\phantom{-}2.01} ±0.05\pm 0.059595
Table 2: NCnet results across hidden dimensions d=1d=1 – 2020 (mean ±\pm std over 100 random seeds). EijE_{ij} denotes task-pair correlation, Acc\mathrm{Acc} denotes per-head training accuracy, and P0P_{0} is the fraction of runs with near-zero mean best-loss. Bold SNNS_{\mathrm{NN}} values significantly exceed 22 .
Settinga1a_{1}a2a_{2}b1b_{1}b2b_{2}
Setting 1Left digit, predict 0Left digit, 10-classRight digit, predict 1Right digit, 10-class
Setting 2PAWS-X-enPAWS-X-frPAWS-X-jaPAWS-X-ko
Setting 3SST-2CommonsenseQAMRPCMathQA
Setting 4MathQAMATHRACE-MiddleLogiQA2.0
Table 3: Task assignments for each experimental setting.
Figure 2: Capacity boundary and gradient conflict in NCnet. Top: SNNS_{\mathrm{NN}} (left axis) and average training loss (right axis) versus hidden width nn . Points and bands show individual seeds and mean ±\pm std; the dashed line marks SNN=2S_{\mathrm{NN}}=2 , and the inset shows its distribution at n=3n=3 . Bottom: gradient cosine similarity averaged over the four MijM_{ij} contexts and all training phases (green: alignment; red: conflict).
DepthChannel width
112244881616323264649696
1−1.02-1.02 ±0.57\pm 0.57−1.82-1.82 ±0.70\pm 0.70−1.30-1.30 ±0.04\pm 0.04−1.23-1.23 ±0.04\pm 0.04−1.13-1.13 ±0.03\pm 0.03−0.95-0.95 ±0.03\pm 0.03−0.80-0.80 ±0.02\pm 0.02−0.76-0.76 ±0.01\pm 0.01
2−1.56-1.56 ±0.50\pm 0.50−1.22-1.22 ±0.03\pm 0.03−1.08-1.08 ±0.04\pm 0.04−0.85-0.85 ±0.07\pm 0.07−0.56-0.56 ±0.01\pm 0.01−0.32-0.32 ±0.02\pm 0.02−0.07-0.07 ±0.05\pm 0.05−0.07\phantom{-}0.07 ±0.04\pm 0.04
3−1.18-1.18 ±0.03\pm 0.03−1.01-1.01 ±0.09\pm 0.09−0.68-0.68 ±0.05\pm 0.05−0.13-0.13 ±0.08\pm 0.08−0.62\phantom{-}0.62 ±0.06\pm 0.06−1.41\phantom{-}1.41 ±0.04\pm 0.04−1.93\phantom{-}1.93 ±0.04\pm 0.04−2.26\mathbf{\phantom{-}2.26} ±0.07\pm 0.07
4−0.89-0.89 ±0.05\pm 0.05−0.26-0.26 ±0.11\pm 0.11−0.68\phantom{-}0.68 ±0.09\pm 0.09−1.61\phantom{-}1.61 ±0.04\pm 0.04−2.46\mathbf{\phantom{-}2.46} ±0.07\pm 0.07−2.96\mathbf{\phantom{-}2.96} ±0.04\pm 0.04−2.06\mathbf{\phantom{-}2.06} ±0.02\pm 0.02−2.00\phantom{-}2.00 ±0.00\pm 0.00
5−0.08-0.08 ±0.09\pm 0.09−1.41\phantom{-}1.41 ±0.08\pm 0.08−2.64\mathbf{\phantom{-}2.64} ±0.06\pm 0.06−2.17\mathbf{\phantom{-}2.17} ±0.02\pm 0.02−2.00\phantom{-}2.00 ±0.00\pm 0.00−2.00\phantom{-}2.00 ±0.00\pm 0.00−2.00\phantom{-}2.00 ±0.00\pm 0.00−2.00\phantom{-}2.00 ±0.00\pm 0.00
Table 4: Train-phase SNNS_{\mathrm{NN}} (mean ±\pm std over 5 random seeds) for the multi-task CNN across shared-encoder depth (rows) and channel width (columns). Bold cells are significantly greater than the classical Bell bound S=2S=2
Figure 3: SNNS_{\mathrm{NN}} versus Trainable Parameter Count Across Settings. (a) Mean SNNS_{\mathrm{NN}} across five seeds for CNNs of varying depth and base channel width; values are shown in each heatmap cell. (b) Mean SNNS_{\mathrm{NN}} versus LoRA rank rr for Settings 2, 3, and 4. Shaded bands show one standard deviation across three seeds, the dashed line marks SNN=2S_{\mathrm{NN}}=2 .
Figure 4: Gradient conflict rates for CNN task pair A2B2A_{2}B_{2} . (a) Rates grouped by mean SNNS_{\mathrm{NN}} : <1.97<1.97 , 1.971.97 – 2.032.03 , and >2.03>2.03 . Points denote architectures, boxes show medians and interquartile ranges, and diamonds mark group means. (b) Rates by shared-encoder depth and base channel width; cell labels show final-epoch mean SNNS_{\mathrm{NN}} .
rrE11E_{11}E12E_{12}E21E_{21}E22E_{22}SNNS_{\mathrm{NN}}
(a) Setting 2
10.743 ±\pm 0.0210.695 ±\pm 0.0250.689 ±\pm 0.0270.661 ±\pm 0.0251.466 ±\pm 0.037
20.839 ±\pm 0.0170.836 ±\pm 0.0080.838 ±\pm 0.0180.822 ±\pm 0.0181.691 ±\pm 0.028
40.936 ±\pm 0.0020.915 ±\pm 0.0050.918 ±\pm 0.0130.896 ±\pm 0.0161.872 ±\pm 0.003
80.968 ±\pm 0.0040.956 ±\pm 0.0070.955 ±\pm 0.0080.946 ±\pm 0.0021.934 ±\pm 0.004
160.969 ±\pm 0.0040.961 ±\pm 0.0030.957 ±\pm 0.0010.951 ±\pm 0.0021.936 ±\pm 0.003
Table 5: Pairwise task correlations EijE_{ij} and CHSH statistic SNNS_{\mathrm{NN}} across LoRA ranks rr for (a) Setting 2, (b) Setting 3, and (c) Setting 4. Values are mean ±\pm std over three random seeds for Settings 2, 3, and 4. Bold SNNS_{\mathrm{NN}} values significantly exceed 22 .
Figure 5: Gradient conflict across LoRA ranks rr . (a, c) Training-averaged conflict rates for the four task pairs in BERT and Qwen, respectively; error bars show standard deviations across seeds. (b) Gradient cosine similarity distributions for BERT’s A2B2A_{2}B_{2} pair: violins show density, boxes show medians and interquartile ranges, and black circles mark means. The red dashed line marks the zero-cosine conflict boundary.
Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

ParameterCNN Setting 1BERT Setting 2BERT Setting 3Qwen3-8B Setting 4
OptimizerAdamW—AdamWAdamW
Initial learning rate10−310^{-3}3×10−43\times 10^{-4}10−310^{-3} (LoRA)2×10−42\times 10^{-4}
Classification-head learning rate10−310^{-3}—2.5×10−32.5\times 10^{-3}—
Learning-rate decayCosinePlateauCosineCosine
Linear warmup5 epochs5%5\%3%3\%5%5\%
Learning-rate floor0010−610^{-6}10−610^{-6}10−610^{-6} or 10−510^{-5}
Appendix
Table 6: Training hyperparameters for Settings 1–4. CNN uses its full training set; language sample caps are per task. A dash indicates an inapplicable parameter or one not recorded in the available training logs. The Qwen learning-rate floors are specified below.
Channel widthA1B1A_{1}B_{1}A1B2A_{1}B_{2}A2B1A_{2}B_{1}A2B2A_{2}B_{2}
A1A_{1}B1B_{1}A1A_{1}B2B_{2}A2A_{2}B1B_{1}A2A_{2}B2B_{2}
190.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0074.00±35.7874.00\pm 35.7810.02±0.0310.02\pm 0.039.95±0.139.95\pm 0.1390.00±0.0090.00\pm 0.0010.02±0.0310.02\pm 0.0310.07±0.1610.07\pm 0.16
274.00±35.7874.00\pm 35.7872.48±35.0872.48\pm 35.0888.35±3.6988.35\pm 3.699.52±1.059.52\pm 1.059.30±0.889.30\pm 0.8890.00±0.0090.00\pm 0.0010.07±0.4510.07\pm 0.459.62±0.719.62\pm 0.71
485.69±4.9585.69\pm 4.9590.00±0.0090.00\pm 0.0089.99±0.0289.99\pm 0.0210.40±0.6610.40\pm 0.6610.57±1.3810.57\pm 1.3889.27±1.6389.27\pm 1.6310.25±0.7610.25\pm 0.7610.06±0.3310.06\pm 0.33
890.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0089.98±0.0489.98\pm 0.0410.70±1.1310.70\pm 1.1310.58±1.3810.58\pm 1.3890.00±0.0090.00\pm 0.0010.93±0.8910.93\pm 0.8910.49±1.1810.49\pm 1.18
1690.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0012.80±1.1512.80\pm 1.1511.73±0.7411.73\pm 0.7490.00±0.0090.00\pm 0.0011.97±0.7811.97\pm 0.7813.28±1.1013.28\pm 1.10
Appendix
Table 7: Task accuracy (%) for CNN in Setting 2 at shared-encoder depth 11 . Each entry is mean ±\pm sample standard deviation over five random seeds.
Channel widthA1B1A_{1}B_{1}A1B2A_{1}B_{2}A2B1A_{2}B_{1}A2B2A_{2}B_{2}
A1A_{1}B1B_{1}A1A_{1}B2B_{2}A2A_{2}B1B_{1}A2A_{2}B2B_{2}
172.50±35.1072.50\pm 35.1072.36±35.0472.36\pm 35.0486.07±5.1886.07\pm 5.1810.09±0.1310.09\pm 0.1310.08±0.1210.08\pm 0.1278.11±26.5978.11\pm 26.5910.13±0.2210.13\pm 0.229.61±0.679.61\pm 0.67
290.00±0.0090.00\pm 0.0090.01±0.0190.01\pm 0.0189.84±0.3789.84\pm 0.3710.13±0.1610.13\pm 0.1611.14±1.9111.14\pm 1.9189.89±0.2589.89\pm 0.2511.51±1.9211.51\pm 1.9210.78±1.1210.78\pm 1.12
490.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0011.72±0.9511.72\pm 0.9511.96±1.1511.96\pm 1.1590.00±0.0090.00\pm 0.0015.58±1.2815.58\pm 1.2813.91±1.3013.91\pm 1.30
890.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0016.12±1.6916.12\pm 1.6915.04±1.7315.04\pm 1.7390.00±0.0090.00\pm 0.0018.20±0.9718.20\pm 0.9718.60±1.8318.60\pm 1.83
1690.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0021.29±0.7621.29\pm 0.7619.79±0.7319.79\pm 0.7390.00±0.0090.00\pm 0.0021.86±0.2121.86\pm 0.2122.71±0.8022.71\pm 0.80
Appendix
Table 8: Task accuracy (%) for CNN in Setting 2 at shared-encoder depth 22 . Each entry is mean ±\pm sample standard deviation over five random seeds.
Channel widthA1B1A_{1}B_{1}A1B2A_{1}B_{2}A2B1A_{2}B_{1}A2B2A_{2}B_{2}
A1A_{1}B1B_{1}A1A_{1}B2B_{2}A2A_{2}B1B_{1}A2A_{2}B2B_{2}
190.00±0.0090.00\pm 0.0089.99±0.0289.99\pm 0.0289.83±0.3889.83\pm 0.3810.79±1.0110.79\pm 1.0111.69±2.3511.69\pm 2.3590.00±0.0090.00\pm 0.0012.58±0.7612.58\pm 0.7611.19±1.6511.19\pm 1.65
290.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0089.97±0.0889.97\pm 0.0813.74±2.2013.74\pm 2.2014.16±1.9014.16\pm 1.9089.99±0.0389.99\pm 0.0315.01±1.3715.01\pm 1.3715.09±1.8215.09\pm 1.82
490.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0018.30±2.4718.30\pm 2.4718.75±1.7418.75\pm 1.7490.00±0.0090.00\pm 0.0020.82±1.5820.82\pm 1.5821.16±1.8621.16\pm 1.86
890.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0027.00±2.3727.00\pm 2.3726.42±1.7226.42\pm 1.7290.00±0.0090.00\pm 0.0030.51±1.8730.51\pm 1.8731.81±2.5531.81\pm 2.55
1690.01±0.0290.01\pm 0.0290.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0041.44±3.0141.44\pm 3.0136.53±1.5836.53\pm 1.5890.00±0.0090.00\pm 0.0039.90±1.0039.90\pm 1.0042.64±0.7442.64\pm 0.74
Appendix
Table 9: Task accuracy (%) for CNN in Setting 2 at shared-encoder depth 33 . Each entry is mean ±\pm sample standard deviation over five random seeds.
Channel widthA1B1A_{1}B_{1}A1B2A_{1}B_{2}A2B1A_{2}B_{1}A2B2A_{2}B_{2}
A1A_{1}B1B_{1}A1A_{1}B2B_{2}A2A_{2}B1B_{1}A2A_{2}B2B_{2}
190.00±0.0090.00\pm 0.0089.99±0.0389.99\pm 0.0389.99±0.0289.99\pm 0.0215.66±2.3315.66\pm 2.3315.94±2.1515.94\pm 2.1590.00±0.0090.00\pm 0.0016.49±1.5316.49\pm 1.5317.09±1.7617.09\pm 1.76
290.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0090.00±0.0090.00\pm 0.0024.17±3.2624.17\pm 3.2621.59±2.2021.59\pm 2.2090.00±0.0090.00\pm 0.0028.97±2.8628.97\pm 2.8630.03±3.0330.03\pm 3.03
490.06±0.0690.06\pm 0.0690.46±0.4890.46\pm 0.4890.14±0.2790.14\pm 0.2740.38±2.7240.38\pm 2.7239.33±2.2739.33\pm 2.2790.44±0.3490.44\pm 0.3441.51±2.2741.51\pm 2.2744.13±3.2444.13\pm 3.24
892.26±0.6392.26\pm 0.6391.53±0.1691.53\pm 0.1690.95±0.1490.95\pm 0.1457.24±1.9257.24\pm 1.9255.01±1.4855.01\pm 1.4890.96±0.0890.96\pm 0.0851.32±0.9551.32\pm 0.9554.04±0.2954.04\pm 0.29
1698.34±0.5198.34\pm 0.5196.32±0.6796.32\pm 0.6790.32±0.1890.32\pm 0.1878.80±2.1378.80\pm 2.1375.95±2.0475.95\pm 2.0490.50±0.1090.50\pm 0.1054.10±0.2654.10\pm 0.2657.36±0.2657.36\pm 0.26
Appendix
Table 10: Task accuracy (%) for CNN in Setting 2 at shared-encoder depth 44 . Each entry is mean ±\pm sample standard deviation over five random seeds.
Channel widthA1B1A_{1}B_{1}A1B2A_{1}B_{2}A2B1A_{2}B_{1}A2B2A_{2}B_{2}
A1A_{1}B1B_{1}A1A_{1}B2B_{2}A2A_{2}B1B_{1}A2A_{2}B2B_{2}
190.00±0.0090.00\pm 0.0090.33±0.4590.33\pm 0.4590.01±0.0290.01\pm 0.0230.58±1.7730.58\pm 1.7728.45±2.1228.45\pm 2.1290.13±0.2590.13\pm 0.2527.40±3.2527.40\pm 3.2528.66±2.7028.66\pm 2.70
292.48±0.8092.48\pm 0.8093.91±1.0593.91\pm 1.0590.37±0.3390.37\pm 0.3358.91±2.0958.91\pm 2.0954.17±3.0554.17\pm 3.0590.56±0.3090.56\pm 0.3045.57±1.3945.57\pm 1.3948.83±1.1448.83\pm 1.14
498.00±0.5798.00\pm 0.5798.71±0.7698.71\pm 0.7690.81±0.2490.81\pm 0.2488.21±0.8088.21\pm 0.8087.47±1.4287.47\pm 1.4291.67±0.6691.67\pm 0.6659.50±1.9759.50\pm 1.9763.40±1.4863.40\pm 1.48
8100.00±0.01100.00\pm 0.0199.98±0.0299.98\pm 0.0298.28±0.3798.28\pm 0.3799.75±0.1099.75\pm 0.1099.45±0.0599.45\pm 0.0598.24±0.5198.24\pm 0.5193.53±0.8993.53\pm 0.8993.47±0.3393.47\pm 0.33
16100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.01100.00\pm 0.01100.00±0.00100.00\pm 0.00
Appendix
Table 11: Task accuracy (%) for CNN in Setting 2 at shared-encoder depth 55 . Each entry is mean ±\pm sample standard deviation over five random seeds.
Channel widthA1B1A_{1}B_{1}A1B2A_{1}B_{2}A2B1A_{2}B_{1}A2B2A_{2}B_{2}
A1A_{1}B1B_{1}A1A_{1}B2B_{2}A2A_{2}B1B_{1}A2A_{2}B2B_{2}
195.73±0.9295.73\pm 0.9298.33±0.6398.33\pm 0.6390.42±0.6390.42\pm 0.6351.35±9.8551.35\pm 9.8559.09±4.8959.09\pm 4.8994.13±1.1494.13\pm 1.1453.93±1.5153.93\pm 1.5154.85±3.2254.85\pm 3.22
299.85±0.0999.85\pm 0.0999.81±0.1099.81\pm 0.1096.75±1.0996.75\pm 1.0991.22±2.2391.22\pm 2.2390.58±1.4790.58\pm 1.4798.04±1.0798.04\pm 1.0786.73±2.3986.73\pm 2.3987.66±2.4787.66\pm 2.47
4100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.0099.95±0.0499.95\pm 0.04100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.0099.89±0.0699.89\pm 0.0699.75±0.1199.75\pm 0.1199.77±0.1899.77\pm 0.18
8100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00
16100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00
Appendix
Table 12: Task accuracy (%) for CNN in Setting 2 at shared-encoder depth 66 . Each entry is mean ±\pm sample standard deviation over five random seeds.
Channel widthA1B1A_{1}B_{1}A1B2A_{1}B_{2}A2B1A_{2}B_{1}A2B2A_{2}B_{2}
A1A_{1}B1B_{1}A1A_{1}B2B_{2}A2A_{2}B1B_{1}A2A_{2}B2B_{2}
199.17±0.3199.17\pm 0.3199.68±0.1199.68\pm 0.1193.61±0.4193.61\pm 0.4176.49±4.3176.49\pm 4.3176.18±2.5776.18\pm 2.5798.76±0.4698.76\pm 0.4669.54±4.6869.54\pm 4.6871.84±2.8771.84\pm 2.87
2100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.0099.92±0.1299.92\pm 0.1299.95±0.0799.95\pm 0.0799.92±0.0399.92\pm 0.0399.83±0.0899.83\pm 0.0899.24±0.3699.24\pm 0.3699.24±0.4099.24\pm 0.40
4100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00
8100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00
16100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00100.00±0.00100.00\pm 0.00
Appendix
Table 13: Task accuracy (%) for CNN in Setting 2 at shared-encoder depth 77 . Each entry is mean ±\pm sample standard deviation over five random seeds.
LoRA rank rrA1B1A_{1}B_{1}A1B2A_{1}B_{2}A2B1A_{2}B_{1}A2B2A_{2}B_{2}
A1A_{1}B1B_{1}A1A_{1}B2B_{2}A2A_{2}B1B_{1}A2A_{2}B2B_{2}
195.32±0.1495.32\pm 0.1490.85±0.8590.85\pm 0.8595.52±0.1895.52\pm 0.1889.15±0.9689.15\pm 0.9693.44±0.2193.44\pm 0.2191.50±0.4091.50\pm 0.4094.29±0.4394.29\pm 0.4389.98±0.9289.98\pm 0.92
297.24±0.3797.24\pm 0.3794.43±0.4594.43\pm 0.4597.42±0.1497.42\pm 0.1494.46±0.0594.46\pm 0.0596.22±0.0896.22\pm 0.0895.24±0.6095.24\pm 0.6096.25±0.0896.25\pm 0.0893.86±0.2793.86\pm 0.27
498.80±0.1198.80\pm 0.1197.88±0.2497.88\pm 0.2498.74±0.0698.74\pm 0.0697.36±0.1997.36\pm 0.1998.01±0.2298.01\pm 0.2297.52±0.2097.52\pm 0.2098.05±0.1198.05\pm 0.1196.89±0.3496.89\pm 0.34
899.51±0.0399.51\pm 0.0398.87±0.0898.87\pm 0.0899.27±0.0599.27\pm 0.0598.39±0.0198.39\pm 0.0198.92±0.0698.92\pm 0.0698.83±0.0798.83\pm 0.0798.99±0.1198.99\pm 0.1198.54±0.0398.54\pm 0.03
1699.52±0.0499.52\pm 0.0498.88±0.0898.88\pm 0.0899.38±0.0799.38\pm 0.0798.73±0.0598.73\pm 0.0598.98±0.0498.98\pm 0.0498.89±0.0598.89\pm 0.0598.93±0.0298.93\pm 0.0298.53±0.1098.53\pm 0.10
Appendix
Table 14: Task accuracy (%) for BERT in Setting 2: A1A_{1} = PAWS-X-en, A2A_{2} = PAWS-X-fr, B1B_{1} = PAWS-X-ja, and B2B_{2} = PAWS-X-ko. Each entry is mean ±\pm sample standard deviation over three random seeds.
LoRA rank rrA1B1A_{1}B_{1}A1B2A_{1}B_{2}A2B1A_{2}B_{1}A2B2A_{2}B_{2}
A1A_{1}B1B_{1}A1A_{1}B2B_{2}A2A_{2}B1B_{1}A2A_{2}B2B_{2}
199.82±0.0899.82\pm 0.0899.81±0.1599.81\pm 0.1599.41±0.1099.41\pm 0.1063.37±2.4663.37\pm 2.4690.26±0.6490.26\pm 0.6499.19±0.1499.19\pm 0.1490.79±0.6390.79\pm 0.6345.68±0.8145.68\pm 0.81
299.96±0.0299.96\pm 0.0299.96±0.0499.96\pm 0.0499.66±0.0899.66\pm 0.0882.93±2.4682.93\pm 2.4697.41±0.4997.41\pm 0.4999.80±0.0999.80\pm 0.0995.63±0.5195.63\pm 0.5170.18±2.1470.18\pm 2.14
499.99±0.0299.99\pm 0.0299.98±0.0499.98\pm 0.0499.83±0.1099.83\pm 0.1095.57±0.2995.57\pm 0.2999.22±0.1899.22\pm 0.1899.93±0.0399.93\pm 0.0397.70±0.3897.70\pm 0.3891.53±0.5991.53\pm 0.59
899.99±0.0299.99\pm 0.0299.98±0.0399.98\pm 0.0399.90±0.0099.90\pm 0.0098.76±0.1098.76\pm 0.1099.68±0.0899.68\pm 0.0899.98±0.0299.98\pm 0.0299.30±0.2299.30\pm 0.2297.89±0.2297.89\pm 0.22
1699.99±0.0299.99\pm 0.0299.99±0.0299.99\pm 0.0299.99±0.0299.99\pm 0.0299.46±0.1399.46\pm 0.1399.68±0.1499.68\pm 0.1499.98±0.0299.98\pm 0.0299.61±0.1799.61\pm 0.1799.44±0.0599.44\pm 0.05
Appendix
Table 15: Final-epoch training accuracy (%) for BERT in Setting 3: A1A_{1} = SST-2, A2A_{2} = CommonsenseQA, B1B_{1} = MRPC, and B2B_{2} = MathQA. Each entry is mean ±\pm sample standard deviation over three random seeds.
LoRA rank rrA1B1A_{1}B_{1}A1B2A_{1}B_{2}A2B1A_{2}B_{1}A2B2A_{2}B_{2}
A1A_{1}B1B_{1}A1A_{1}B2B_{2}A2A_{2}B1B_{1}A2A_{2}B2B_{2}
198.27±0.0998.27\pm 0.0998.89±0.1298.89\pm 0.1297.48±0.1797.48\pm 0.1797.58±0.0897.58\pm 0.0867.79±0.4567.79\pm 0.4599.00±0.0699.00\pm 0.0661.76±0.3361.76\pm 0.3399.07±0.0399.07\pm 0.03
299.92±0.0299.92\pm 0.0299.52±0.1299.52\pm 0.1299.78±0.0299.78\pm 0.0299.02±0.1499.02\pm 0.1483.39±0.4583.39\pm 0.4599.79±0.0599.79\pm 0.0579.22±0.2979.22\pm 0.2999.71±0.0799.71\pm 0.07
499.90±0.0399.90\pm 0.0399.63±0.1099.63\pm 0.1099.94±0.0299.94\pm 0.0299.48±0.1399.48\pm 0.1391.57±0.2891.57\pm 0.2899.92±0.0599.92\pm 0.0587.84±0.2787.84\pm 0.2799.97±0.0399.97\pm 0.03
899.98±0.0299.98\pm 0.0299.91±0.0799.91\pm 0.0799.98±0.0299.98\pm 0.0299.74±0.0799.74\pm 0.0796.52±0.2596.52\pm 0.2599.88±0.0299.88\pm 0.0295.72±0.2595.72\pm 0.2599.87±0.0099.87\pm 0.00
16100.00±0.00100.00\pm 0.0099.99±0.0299.99\pm 0.0299.99±0.0299.99\pm 0.0299.86±0.0599.86\pm 0.0599.10±0.1799.10\pm 0.1799.97±0.0099.97\pm 0.0099.22±0.0799.22\pm 0.0799.96±0.0299.96\pm 0.02
Appendix
Table 16: Task accuracy (%) for Qwen3-8B in Setting 4: A1A_{1} = MathQA, A2A_{2} = MATH, B1B_{1} = RACE-Middle, and B2B_{2} = LogiQA2.0. Each entry is mean ±\pm sample standard deviation over three random seeds.
More in this topic

Shared Representation Learning

See all 9 papers

Explore similar work

CardsList

  1. Capacity and Redundancy Trade-offs in Multi-Task Learning

    Jul 17, 2026Asif KhanMulti-Task LearningMulti-Task Optimization

    1Harvard Medical School, Boston, USA

  2. The Information-Theoretic Benefit of Shared Representations under Orthogonality Constraints

    Jun 14, 2026Thomas Dittrich, Oliver Potocki, Philipp GrohsShared Representation LearningRepresentation Learning

    Johann Radon Institute of Computational and Applied Mathematics, Austrian Academy of Sciences · Faculty of Mathematics, University of Vienna

  3. An Empirical Study and Theoretical Explanation on Task-Level Model-Merging Collapse

    Mar 10, 2026Yuan Cao, Dezhi Ran, Yuzhe Guo +5Language Model MergingModel Merging

    Key Lab of HCST (PKU), MOE; SCS, Peking University, Beijing, China · Computer Science Department at Columbia University, New York, USA · School of Computing Science, Simon Fraser University, Burnaby, BC, Canada +1