Consistency Regularization

Consistency regularization trains a model to agree when inputs or conditions should preserve the same meaning, helping it learn from unlabeled data and remain stable under change. Common approaches align predictions with KL losses or pseudo-labels across augmented views, use teacher–student or cross-model agreement, and enforce cycle, temporal, or latent-dynamics consistency. These methods appear in semi-supervised segmentation, domain adaptation, robustness, continual learning, and scientific modeling, with evaluation ranging from image benchmarks to shifted environments and long-horizon predictions. A central tension is that agreement can improve stability yet also reinforce shared errors, suppress meaningful variation, or even amplify undesirable behaviors.