Gradient Interference

Gradient interference occurs when tasks or objectives produce updates that compete in shared parameters, potentially causing negative transfer, forgetting, or unstable training. Researchers study it in settings such as multi-task and continual learning, multimodal models, federated training, reinforcement learning, and physics-informed neural networks. Common approaches measure gradient alignment and use projection or reweighting methods, while others isolate updates with adapters, parameter subspaces, routing, or separate modules. A central tension is that directional conflict does not always predict actual performance harm, so interventions must account for context, gradient magnitude and variance, and trade-offs among objectives.