Machine Learning under Imperfect Data: Challenges and Methods
Organizations: Shanghai Jiao Tong University, China
Abstract
Machine-learning models are commonly developed under an assumption that training and test data are sufficiently complete, balanced, labelled, and drawn from compatible distributions. In practice, one or more of these conditions is often violated. Measurements may be missing or corrupted, rare classes may be poorly represented, supervision may be weak, and the deployment environment may differ from the training environment. These imperfections are usually treated as separate technical problems, although they alter learning through a small number of shared mechanisms: loss of information, biased empirical risk, ambiguous supervision, and unstable representations. This short survey organises representative methods around these mechanisms. It reviews reconstruction and generation, rebalancing and representation calibration, learning with limited supervision, adaptation across domains and modalities, and reliability under distribution change. The discussion highlights the limits of plausible reconstruction, benchmark-specific correction, and adaptation without trustworthy feedback. It concludes with directions for evidence-aware learning, uncertainty-preserving prediction, and evaluation that separates visual plausibility from decision utility.
Figures & tables
| Imperfection | Learning consequence | Representative responses | Central evaluation question |
|---|---|---|---|
| Missing or degraded content | Multiple latent observations may explain the same input | Super-resolution, inpainting, completion, conditional generation, multimodal fusion | Is the recovery faithful, or merely plausible? |
| Imbalanced sampling | Empirical risk is dominated by frequent groups or easy examples | Reweighting, resampling, focal losses, distribution calibration, synthetic augmentation | Does minority performance improve without distorting calibration? |
| Limited supervision | Labels do not uniquely specify the target structure | Multiple-instance learning, pseudo-labels, consistency, distillation, region-word alignment | Does the model discover the intended evidence or a shortcut? |
| Distribution shift | Training correlations are not stable at deployment | Invariant features, domain adaptation, source-free adaptation, domain generalisation | Are gains stable across genuinely unseen shifts? |
| Modality mismatch | Evidence has different statistics or is absent in one channel | Shared embeddings, privileged information, contrastive alignment, fusion | Is complementary information preserved without forced alignment? |