MIP Candy: A Modular PyTorch Framework for Medical Image Processing
Authors: Tianhao Fu, Yucheng Chen
Organizations: University of Toronto, Toronto, ON, Canada · Vector Institute, Toronto, ON, Canada · Project Neura, Toronto, ON, Canada · UTMIST, Toronto, ON, Canada · Amplimit, Toronto, ON, Canada
Medical image processing demands specialized software that handles high-dimensional volumetric data, heterogeneous file formats, and domain-specific training procedures. Existing frameworks either provide low-level components that require substantial integration effort or impose rigid, monolithic pipelines that resist modification. We present MIP Candy (MIPCandy), a freely available, PyTorch-based framework designed specifically for medical image processing. MIPCandy provides a complete, modular pipeline spanning data loading, training, inference, and evaluation, allowing researchers to obtain a fully functional process workflow by implementing a single method, \texttt{build_network}, while retaining fine-grained control over every component. Central to the design is LayerT, a deferred configuration mechanism that enables runtime substitution of convolution, normalization, and activation modules without subclassing. The framework further offers built-in k-fold cross-validation, dataset inspection with automatic region-of-interest detection, deep supervision, exponential moving average, multi-frontend experiment tracking (Weights & Biases, Notion, MLflow), training state recovery, and validation score prediction via quotient regression. An extensible bundle ecosystem provides pre-built model implementations that follow a consistent trainer--predictor pattern and integrate with the core framework without modification. MIPCandy is open-source under the Apache-2.0 license and requires Python~3.12 or later. Source code and documentation are available at https://github.com/ProjectNeura/MIPCandy.
Figures & tables
Feature
nnU-Net
MONAI
TorchIO
MIPCandy
Complete training pipeline
✓
–
–
✓
One-method setup
✓
–
–
✓
Modular / individually usable
–
✓
✓
✓
Custom architecture swap
Hard
Manual
N/A
build_network
Deep supervision
✓
Manual
N/A
One flag
EMA support
–
Manual
N/A
One flag
Table 1: Feature comparison of active medical image segmentation frameworks.
LayerT configuration, device management, checkpoint I/O, WithNetwork and WithPaddingModule base classes
mipcandy.training
Trainer base class, TrainerToolbox dataclass, experiment management, validation score prediction
mipcandy.presets
SegmentationTrainer preset with pre-configured loss, optimizer, scheduler, and deep supervision
mipcandy.inference
Predictor base class, parse_predictant utility, file-level prediction and export
mipcandy.evaluation
Evaluator class, EvalResult container with per-case and aggregate metrics
Table 2: MIPCandy module overview.
Figure 1: Training progress plots automatically generated by MIPCandy during a U-Net training run on the PH2 dermoscopy dataset. The validation score is the negated combined loss (Section 4.3 ); higher values indicate better performance.
Figure 2: Worst-case prediction previews automatically saved during training. The framework identifies the validation case with the lowest score and generates overlays comparing the ground truth (b) and model prediction (c) against the input image (a). This example is from a U-Net trained on PH2.
Figure 3: 3D volume previews rendered via PyVista. MIPCandy automatically generates 3D visualizations of labels and predictions for volumetric segmentation tasks.
Figure 4: Notion frontend integration. MIPCandy automatically logs experiment metadata, progress, and scores to a Notion database.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Console interface during training state recovery. The output shows a single epoch after recovery: sanity check, training metrics with a structured summary table, per-case validation metrics with per-class statistics, score prediction with ETC, and checkpoint management.
We present APRIL-MedSeg, a YAML-driven modular framework for 2D medical image segmentation. It provides a unified and extensible ecosystem that decomposes segmentation networks into reusable components. Also, the framework integrates a broad spectrum of advanced paradigms, including semi-supervised learning, domain adaptation, knowledge distillation, weakly supervised learning, and text-guided segmentation as well as foundation model support. A registry-based configuration system with inheritance enables flexible and reproducible experiment management, supporting seamless switching across models, datasets, and training strategies. In addition, the framework provides a unified interface for medical datasets, augmentation pipelines, deployment utilities and model ensembling. Overall, APRIL-MedSeg is designed as a general-purpose research and development platform that bridges algorithmic innovation and practical deployment, while also serving as a structured ecosystem for systematically organizing and reproducing advances in medical image segmentation. The code is available at https://github.com/juntaoJianggavin/APRIL-MedSeg under an Apache 2.0 license.
Juntao Jiang, Jinsheng Bai, Linxuan Fan +3
Zhejiang University, APRIL Lab · Vanderbilt University · University of Science and Technology of China
Three-dimensional medical image data and computer-aided decision making, particularly using deep learning, are becoming increasingly important in the medical field. To aid in these developments we introduce PR3DICTR: Platform for Research in 3D Image Classification and sTandardised tRaining. Built using community-standard distributions (PyTorch and MONAI), PR3DICTR provides an open-access, flexible and convenient framework for prediction model development, with an explicit focus on classification using three-dimensional medical image data. By combining modular design principles and standardization, it aims to alleviate developmental burden whilst retaining adjustability. It provides users with a wealth of pre-established functionality, for instance in model architecture design options, hyper-parameter solutions and training methodologies, but still gives users the opportunity and freedom to ``plug in'' their own solutions or modules. PR3DICTR can be applied to any binary or event-based three-dimensional classification task and can work with as little as two lines of code.
Daniel C. MacRae, Luuk van der Hoek, Robert van der Wal +5
Department of Radiation Oncology, University Medical Center Groningen, University of Groningen, Groningen, the Netherlands
Large language models (LLMs) have demonstrated considerable promise in program generation for small-scale and conventional application development; however, they remain limited when applied to complex, domain-specific tasks such as medical image processing. General-purpose models lack explicit domain knowledge and robust validation mechanisms to ensure correctness, often requiring substantial human intervention to produce reliable processing pipelines. To address these limitations, we propose AutoMedImg, a multi-agent framework for fully automated medical image processing code generation. AutoMedImg orchestrates specialised agents across two phases: a Planning Phase that performs dataset analysis and architecture design with semantic and formal verification, and a Coding Phase that generates modules in parallel with static checking, execution testing, and assembly validation. This multi-stage validation mitigates error propagation throughout generation, while comprehensive auto-context engineering combining domain-specific knowledge bases, shared memory, and validation feedback automates context construction without manual prompting. A cross-project adaptive pipeline synthesis mechanism further accumulates validated pipelines and retrieves proven components for new tasks based on project similarity, enhancing generation efficiency through cross-project learning. Extensive evaluation across six diverse and well-established medical imaging datasets with five backbone LLMs demonstrates that AutoMedImg achieves zero human intervention, with Dice scores of up to 0.90 for segmentation tasks and 99% accuracy for classification.
Zixiao Zhao, Jing Sun, Zhe Hou +5
University of Auckland Auckland, New Zealand · Griffith University Brisbane, Australia · Suzhou Industrial Park Monash Research Institute of Science and Technology Suzhou, China +2