Multi-Task Learning for Metal Alloy Property Prediction: An Empirical Study of Negative Transfer and Mitigation Strategies

Sungwoo Kang

cs.LGDec 28, 2025

Multi-Task Learning for Metal Alloy Property Prediction: An Empirical Study of Negative Transfer and Mitigation Strategies

Authors: Sungwoo Kang

Organizations: Department of Electrical and Computer Engineering, Korea University, Seoul 02841, Republic of Korea

Abstract

Multi-task learning (MTL) in materials science relies on the assumption that physically related properties share learnable representations. We challenge this assumption using a 54,028-sample metal alloy dataset exhibiting extreme task-level imbalance. Our results reveal a striking dichotomy: MTL significantly degrades regression performance for resistivity and hardness but improves classification recall for amorphous-forming ability. We trace this divergence to mismatched functional forms--such as resistivity's polynomial dependence versus hardness's complex interactions--which cause severe gradient misalignment during optimization. Evaluating Deep Imbalanced Regression techniques, we find that projecting conflicting gradients (PCGrad) recovers single-task performance, while combining label distribution smoothing with gradient normalization achieves the best overall balance. Consequently, we propose a strategic framework: utilize independent models for high-precision characterization, but employ MTL for high-throughput screening where recall is paramount. These findings support a "materials property clustering" hypothesis, suggesting that distinct physical mechanisms require specialized optimization strategies to overcome negative transfer.

Figures & tables

Property	Train	Validation	Test
Resistivity	36,670	7,859	7,859
Hardness	560	120	120
Amorphous	588	126	126
Total (Union)	37,818	8,105	8,105

Table 1: Dataset statistics showing severe imbalance across properties.

Property	Metric	Independent	Standard MTL	Structured MTL
Resistivity	RMSE	0.252 $\pm$ 0.002	0.313 $\pm$ 0.019	0.310 $\pm$ 0.022**
	MAE	0.161 $\pm$ 0.001	0.217 $\pm$ 0.018	0.215 $\pm$ 0.022**
	$R^{2}$	0.897 $\pm$ 0.001	0.842 $\pm$ 0.019	0.844 $\pm$ 0.021**
Hardness	RMSE	0.407 $\pm$ 0.011	0.516 $\pm$ 0.075	0.548 $\pm$ 0.035**
	MAE	0.312 $\pm$ 0.011	0.376 $\pm$ 0.069	0.396 $\pm$ 0.041**
	$R^{2}$	0.832 $\pm$ 0.009	0.724 $\pm$ 0.078	0.694 $\pm$ 0.038**

Table 2: Performance comparison across modeling approaches. Best results for each metric are bolded . Statistical significance of Structured MTL vs. Independent is indicated: *

p<0.05

, **

p<0.01

Figure 1: Performance comparison between independent and MTL models. R² shown for regression tasks (Resistivity, Hardness); F1 score shown for classification (Amorphous). MTL degrades regression but improves classification.

Property	Metric	$p$ -value
Resistivity	$R^{2}$	0.865
Hardness	$R^{2}$	0.617
Amorphous	F1	0.273

Table 3: Statistical comparison: Structured MTL vs. Standard MTL. No significant differences observed.

Figure 2: Learned task relation graph. All inter-task weights are near zero, indicating task independence.

Relation	Strength
Resistivity $\rightarrow$ Hardness	0.006 $\pm$ 0.012
Resistivity $\rightarrow$ Amorphous	0.006 $\pm$ 0.013
Hardness $\rightarrow$ Resistivity	0.007 $\pm$ 0.013
Hardness $\rightarrow$ Amorphous	0.006 $\pm$ 0.013
Amorphous $\rightarrow$ Resistivity	0.006 $\pm$ 0.013
Amorphous $\rightarrow$ Hardness	0.006 $\pm$ 0.012

Table 4: Learned task relation strengths (mean

\pm

std across seeds).

Figure 3: Training curves showing validation loss over epochs. MTL models exhibit higher loss for regression tasks despite longer training.

Figure 4: Error distributions for each property. Independent models show tighter distributions for regression tasks.

Figure 5: Confusion matrices for amorphous classification. MTL improves recall (fewer false negatives) at slight cost to precision.

Figure 6: Cost of data imbalance. Hardness

R^{2}

generally decreases as the resistivity dataset grows beyond 5,000 samples, demonstrating that severe data imbalance causes negative transfer. Error bars show

\pm

1 std across 5 seeds.

Resistivity Samples	Hardness $R^{2}$	Imbalance Ratio
1,000	0.830 $\pm$ 0.022	1.25:1
5,000	0.866 $\pm$ 0.017	6.25:1
10,000	0.845 $\pm$ 0.030	12.5:1
25,000	0.756 $\pm$ 0.045	31.25:1
52,388	0.668 $\pm$ 0.095	65.5:1

Table 5: Hardness prediction performance vs. resistivity dataset size. Performance generally degrades with increasing imbalance, though peaks at moderate imbalance (5,000 samples).

Figure 7: Gradient cosine similarity between task pairs. Values near zero indicate orthogonal gradients (no synergy); negative values indicate conflicting gradients.

Figure 8: Transfer learning comparison. Pre-training on resistivity provides no benefit over training from scratch for hardness prediction (

p=0.127

Method	Resistivity $R^{2}$	$\Delta$	Hardness $R^{2}$	$\Delta$	Amorphous F1	$\Delta$
Baseline MSE	0.874	–	0.761	–	0.692	–
LDS Only	0.869	-0.6%	0.767	+0.8%	0.729	+5.3%
Balanced MSE	0.869	-0.6%	0.767	+0.8%	0.729	+5.3%
GradNorm Only	0.873	-0.1%	0.786	+3.3%	0.744	+7.5%
PCGrad Only	0.852	-2.5%	0.855	+12.4%	0.701	+1.3%
Balanced MSE + PCGrad	0.850	-2.8%	0.827	+8.7%	0.676	-2.3%

Table 6: DIR mitigation experiment results. Best results for each metric are bolded .

\Delta

indicates improvement over baseline MSE.

Method	Resistivity $R^{2}$	Hardness $R^{2}$	Amorphous F1
Baseline MSE	0.874	0.761	0.692
FDS Only	0.874	0.761	0.692
LDS + FDS	0.874	0.761	0.692

Table 7: Feature Distribution Smoothing results. FDS shows no improvement over baseline.

Config	Resistivity $R^{2}$	Hardness $R^{2}$	Amorphous F1	Backbone Sim.
Hard Sharing	0.875	0.735	0.714	1.000
Soft $\lambda=0.001$	0.868	0.764	0.744	0.996
Soft $\lambda=0.01$	0.852	0.692	0.750	0.999
Soft $\lambda=0.1$	0.849	0.704	0.774	0.999

Table 8: Soft parameter sharing results.

\lambda=0.001

improves hardness and classification while maintaining resistivity performance.

Task	Max Error	P95 Error	P99 Error	Worst-10% MAE
Hard Sharing
Resistivity	2.19	0.60	1.06	0.69
Hardness	3.11	1.00	1.80	1.22
Soft $\lambda=0.001$
Resistivity	2.07	0.63	1.06	0.71
Hardness	1.95	1.05	1.39	1.12

Table 9: Tail metrics for soft parameter sharing (

\lambda=0.001

). Soft sharing reduces worst-case errors for hardness.

Property	Samples	Element-wise $R^{2}$	$\delta$ -based $R^{2}$	Primary Dependence
Resistivity	52,388	0.740	0.018	Element-wise scattering
Hardness	800	0.685	0.075	Mixed (weaker signal)

Table 10: Functional form analysis. Comparison of element-wise polynomial features (Nordheim-type:

\sum_{i}k_{i}c_{i}(1-c_{i})

) versus global lattice distortion features (

\delta

-based). Resistivity shows strong dependence on element-wise scattering terms, confirming Nordheim-rule physics.

Property	Poly Features	Power Features	Poly $R^{2}$	Power $R^{2}$
Resistivity	40	7	0.790	0.684
Hardness	40	7	0.847	0.090

Table 11: Linear model comparison with polynomial vs. power-law feature sets. The same linear regression is fit using element-wise polynomial features versus global power-law features. Despite having fewer features, the polynomial representation dramatically outperforms power-law for both properties, confirming that element-wise concentration terms are essential.

Cluster	Properties	Governing Physics
Electronic	Resistivity, Thermal Conductivity, Seebeck	Electron transport/scattering
Mechanical	Hardness, Yield Strength, UTS	Dislocation mechanics
Thermodynamic	Formation Enthalpy, Phase Stability	Free energy landscapes

Table 12: Proposed materials property clusters based on underlying physics. MTL is expected to yield positive transfer within clusters but negative transfer across clusters. Caveat: As demonstrated in Section 5.4.1 , mathematically derived properties (e.g., thermal conductivity from resistivity via Wiedemann-Franz law) do not provide informational independence and should not be used as auxiliary tasks.

Scenario	Configuration	Resistivity $R^{2}$	Transfer Type
A (Cross-mech)	Res+Hard+Amor	0.897 $\pm$ 0.006	Cross-mechanism
B (Within-mech)	Res+Therm+Amor	0.874 $\pm$ 0.013	Within-mechanism
$\Delta$ (B - A)	–	$-$ 0.023	–

Table 13: Derived property experiment results. Cross-mechanism MTL outperforms within-mechanism MTL, contrary to the naive clustering hypothesis.

Multi-Task Learning

Multi-task learning trains models to solve related tasks together by sharing representations, parameters, or supervision.

See all 389 papers

Cross-Material Support Transfer for Core-Loss Prediction Under Waveform Covariate Shift
Sep 15, 2026Cong Yao, Chunye GongMaterials Property Prediction Transfer Learning
School of Computer Science and Technology, Changsha University of Science and Technology, Changsha 410076, China · College of Computing, National University of Defense Technology, Changsha 410073, China · National Supercomputer Center in Tianjin, Tianjin 300457, China +1
Information-Theoretic Requirements for Gradient-Based Task Affinity Estimation in Multi-Task Learning
Apr 9, 2026Jasper Zhang, Bryan ChengMulti-Task Learning Task Transferability Estimation
Great Neck South High School
Distributionally Robust Multi-Task Reinforcement Learning via Adaptive Task Sampling
May 14, 2026Nicholas E. Corrado, Wenyuan Huang, Josiah P. HannaDistributionally Robust RL Multi-Task RL
Computer Sciences Department · University of Wisconsin – Madison

Sep 15, 2026J/K move · Enter open · S save

Cross-Material Support Transfer for Core-Loss Prediction Under Waveform Covariate Shift

Cong Yao, Chunye Gong

School of Computer Science and Technology, Changsha University of Science and Technology, Changsha 410076, China · College of Computing, National University of Defense Technology, Changsha 410073, China · National Supercomputer Center in Tianjin, Tianjin 300457, China+1

Materials Property Prediction Transfer Learning Covariate Shift

Open paper AskPDF arXiv

Power magnetic materials are characterized on the sinusoidal and triangular waveforms that excitation hardware conveniently produces, whereas deployed converters expose cores to trapezoidal, PWM-shaped flux trajectories, so loss models must predict exactly where their training data are thinnest. The final test of the MagNet Challenge embeds a deliberately extreme instance of this characterization-deployment mismatch: for material D, trapezoids form 16.4% of the test set but only 1.4% of the training set. The 95th-percentile relative error, hereafter p95, of the best submission, built on sequential transfer learning, stalled at 15.9%, the worst among the five materials. This paper shows that the obstacle is missing information under covariate shift rather than class imbalance, and that the missing support can be borrowed from sibling materials instead of being extrapolated. Controlled experiments first refute the imbalance reading: four standard remedies fail, and raising the trapezoidal share to the test-set level degrades accuracy further. The proposed material-identity support transfer, MIST, then trains one 2784-parameter predictor jointly on all five challenge materials. Material identity enters through feature-wise linear modulation, or FiLM, the scarce material's true-label loss is reweighted, and material D receives no fine-tuning, so that the bias of its trapezoid-free training set is never re-installed. MIST lowers the five-seed material-D p95 from 20.39+/-2.03% to 12.38+/-0.92% and the trapezoidal-class p95 from 37.4+/-8.8% to 15.16+/-1.69%, surpassing the best submission with one-sixth of its parameters and no fine-tuning stage; removing material identity at matched capacity inflates the error by an order of magnitude. These results argue that scarce materials should be characterized jointly with their siblings.

Abstract

Multi-Task Learning

Explore similar work

Cross-Material Support Transfer for Core-Loss Prediction Under Waveform Covariate Shift

Information-Theoretic Requirements for Gradient-Based Task Affinity Estimation in Multi-Task Learning

Distributionally Robust Multi-Task Reinforcement Learning via Adaptive Task Sampling