MIP Candy: A Modular PyTorch Framework for Medical Image Processing
Organizations: University of Toronto, Toronto, ON, Canada · Vector Institute, Toronto, ON, Canada · Project Neura, Toronto, ON, Canada · UTMIST, Toronto, ON, Canada · Amplimit, Toronto, ON, Canada
Abstract
Medical image processing demands specialized software that handles high-dimensional volumetric data, heterogeneous file formats, and domain-specific training procedures. Existing frameworks either provide low-level components that require substantial integration effort or impose rigid, monolithic pipelines that resist modification. We present MIP Candy (MIPCandy), a freely available, PyTorch-based framework designed specifically for medical image processing. MIPCandy provides a complete, modular pipeline spanning data loading, training, inference, and evaluation, allowing researchers to obtain a fully functional process workflow by implementing a single method, \texttt{build_network}, while retaining fine-grained control over every component. Central to the design is , a deferred configuration mechanism that enables runtime substitution of convolution, normalization, and activation modules without subclassing. The framework further offers built-in -fold cross-validation, dataset inspection with automatic region-of-interest detection, deep supervision, exponential moving average, multi-frontend experiment tracking (Weights & Biases, Notion, MLflow), training state recovery, and validation score prediction via quotient regression. An extensible bundle ecosystem provides pre-built model implementations that follow a consistent trainer--predictor pattern and integrate with the core framework without modification. MIPCandy is open-source under the Apache-2.0 license and requires Python~3.12 or later. Source code and documentation are available at https://github.com/ProjectNeura/MIPCandy.
Figures & tables
| Feature | nnU-Net | MONAI | TorchIO | MIPCandy |
|---|---|---|---|---|
| Complete training pipeline | ✓ | – | – | ✓ |
| One-method setup | ✓ | – | – | ✓ |
| Modular / individually usable | – | ✓ | ✓ | ✓ |
| Custom architecture swap | Hard | Manual | N/A | build_network |
| Deep supervision | ✓ | Manual | N/A | One flag |
| EMA support | – | Manual | N/A | One flag |
| Module | Responsibility |
|---|---|
| mipcandy.data | Multi-format I/O, dataset classes, -fold cross-validation, transforms, dataset inspection, visualization |
| mipcandy.layer | LayerT configuration, device management, checkpoint I/O, WithNetwork and WithPaddingModule base classes |
| mipcandy.training | Trainer base class, TrainerToolbox dataclass, experiment management, validation score prediction |
| mipcandy.presets | SegmentationTrainer preset with pre-configured loss, optimizer, scheduler, and deep supervision |
| mipcandy.inference | Predictor base class, parse_predictant utility, file-level prediction and export |
| mipcandy.evaluation | Evaluator class, EvalResult container with per-case and aggregate metrics |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.