Learning from high-resolution 2D images to reconstruct 3D microstructures
Authors: Wenbo Sui, Lars P. Mikkelsen, Jens W. Andreasen, Daniel Lichau, Harold Phelippeau, Philip J. Withers
Organizations: Thermo Fisher Scientific, Bordeaux, 33800, France. · DTU Energy, Technical University of Denmark, Kgs. Lyngby, 2800, Denmark. · DTU Wind, Technical University of Denmark, Roskilde, 4000, Denmark. · Henry Royce Institute, Department of Materials, University of Manchester, Manchester, M13 9PL, United Kingdom.
High-resolution three-dimensional (3D) microstructure characterization remains difficult to undertake at scale because volumetric imaging at the required spatial resolution is expensive, time-consuming or destructive. In many settings, however, fine-resolution two-dimensional (2D) imaging and lower-resolution volumetric measurements are both readily available. We introduce a Privileged Cross-modal Relational Distillation (PCRD) framework that leverages fine-resolution 2D observations during training to improve specimen-specific 3D reconstruction from lower-resolution volumetric imaging alone. We demonstrate PCRD on pultruded carbon fiber-reinforced polymer composites, using scanning electron microscopy as the privileged modality and micro-CT as the inference input. The framework transfers spatial organization by matching similarity distributions defined around corresponding spatial anchors in the teacher and student representations, avoiding direct alignment of heterogeneous feature vectors. In 2D cross-sectional reconstruction, PCRD improves mAP by 1.66 percentage points over direct micro-CT training. Independent 3D validation against high-resolution synchrotron X-ray CT data shows close agreement in fiber volume fraction, diameter and local spacing. The PCRD reconstruction matches 89.97% of the reference fiber centerline length, compared with 75.08% for the interpolated low-resolution volume.
Figures & tables
Figure 1: Privileged cross-modal relational distillation framework for microstructure reconstruction. a , Representative example of imaging inputs, reference and reconstruction quality. Co-registered SEM and micro-CT images are shown with their corresponding pixel sizes, alongside the ground-truth fiber-instance reference and reconstruction quality visualizations for direct student training and ACSD. Each ground-truth fiber is colored according to the IoU of its matched prediction, with four IoU bands separated at 0.25, 0.50 and 0.75, progressing from coral for lower overlap to blue for higher overlap. b , Overview of the PCRD framework. The SEM teacher and CT student extract and fuse features from four backbone depths for instance prediction. Gray paths summarize response-level and feature-level transfer, including direct feature alignment, feature decomposition and relational distillation. The gold path denotes ACSD-based transfer through anchor-conditioned intra-modal similarity distributions at registered spatial locations. The SEM teacher and cross-modal transfer are used only during training, while the CT student is retained for micro-CT-only inference.
Figure 2: Anchored cross-modal similarity distribution distillation. A 4×4 token grid is shown schematically for clarity. The fused teacher and student feature grids are flattened into token sequences, with spatially corresponding anchors outlined in purple and marked by purple dots. For each anchor, cosine similarities with the remaining tokens within the same feature space are converted by masked softmax into teacher and student similarity distributions. ACSD minimizes the KL divergence between corresponding distributions across the shared anchor set, transferring spatial organization from the privileged SEM teacher to the CT student.
Table 1: Comparison of cross-modal knowledge distillation strategies for CFRP fiber-instance reconstruction. Results are reported as mean ± standard deviation over five runs with different random seeds.
Figure 3: Analysis of relational transfer in ACSD. a,b , Effects of similarity-distribution matching, prediction distillation and anchor selection on mAP and [email protected]. c , Effect of the number of spatial anchors on mAP. d , Sensitivity of mAP to the teacher temperature τT and student temperature τS . Results are reported as means over three runs with different random seeds, with standard deviation error bars shown in a, b and c.
Figure 4: 3D reconstruction and validation against synchrotron X-ray CT data. a , Transverse and longitudinal views of the HR reference, with red boxes indicating the region selected for 3D validation. b , Enlarged transverse region showing the HR reference, LR input and PCRD reconstruction. Individual fibers that are poorly resolved in the downsampled input are recovered as distinct instances after reconstruction. c , 3D visualization of the PCRD reconstruction and the fiber centerlines extracted by the subsequent tracking analysis, which are used to quantify fiber trajectory fidelity through the volume.
Metric
HR reference
Interpolated LR
PCRD reconstruction
Fiber volume fraction (%)
71.89
74.52
71.85
Equivalent fiber diameter ( μ m)
7.35 [6.93, 7.74]
–
7.33 [7.08, 7.58]
Nearest-neighbor distance ( μ m)
6.61 [6.20, 7.01]
–
6.83 [6.48, 7.20]
Reference centerline coverage (%) ↑
Ref.
75.08
89.97
Centerline position deviation ( μ m) ↓
Ref.
0.77 [0.44, 1.32]
0.98 [0.63, 1.52]
Orientation deviation ( ∘ ) ↓
Ref.
0.78 [0.00, 2.49]
1.12 [0.00, 2.49]
Table 2: Structural properties and fiber tracking fidelity relative to the high-resolution synchrotron X-ray CT reference. 0 0 footnotetext: Equivalent fiber diameter and nearest-neighbor distance are reported as median [IQR]. Tracking deviations are computed relative to the HR reference and reported as length-weighted median [IQR]. A dash indicates that reliable instance-level measurements could not be obtained from the interpolated LR volume.
Figure S1: Specimen preparation for multimodal imaging. a , Schematic showing the sectioning of a 40×30×5 mm CFRP sample into 5×5×10 mm specimens. b , Metallographic sectioning using a MICRACUT 202 cutting machine. c , Materials used for multi-grain manual polishing, including polishing cloths and 1.0 and 0.3 μ m α -alumina powders and 0.05 μ m γ -alumina powder. d , Mirror-finished specimen end face after polishing, prepared for subsequent microscopic imaging.
Figure S2: Ablation of student feature dimensionality and multi-slice CT input. Results are reported as mean ± standard deviation across three random seeds. a , mAP with student feature dimensions of 192, 384 and 768, with the teacher dimension fixed at 768. Increasing the student dimension from 192 to 384 improves performance, while further expansion to 768 provides no additional gain, supporting the use of a compact 384-dimensional student representation. b , mAP using one to four adjacent CT slices as input. Multi-slice context substantially improves performance over the single-slice setting, with performance remaining relatively stable from two to four slices. Three slices achieve the highest mean mAP and are used in the final model.
Figure S3: Visualization of fine registration between SEM and micro-CT images. Registration results are shown for six pairs. Grayscale images show the SEM observations, with the interpolated micro-CT images overlaid as heat maps. Low- and high-density features in the micro-CT images spatially coincide with the corresponding structures in the SEM images across all specimens, demonstrating consistent cross-modal alignment after fine registration.
Figure S4: Intensity-distribution comparison for synchrotron-to-micro-CT downsampling calibration. Intensity histograms of the downsampled synchrotron X-ray CT volume and experimental micro-CT data after percentile-based rescaling. The Gaussian blur strength was calibrated to align the dominant intensity-distribution peaks of the two datasets.
Generating high-resolution 3D CT volumes with fine details remains challenging due to substantial computational demands and optimization difficulties inherent to existing generative models. In this paper, we propose the Pixel-Level Residual Diffusion Transformer (PRDiT), a scalable generative framework that synthesizes high-quality 3D medical volumes directly at voxel-level. PRDiT introduces a two-stage training architecture comprising 1) a local denoiser in the form of an MLP-based blind estimator operating on overlapping 3D patches to separate low-frequency structures efficiently, and 2) a global residual diffusion transformer employing memory-efficient attention to model and refine high-frequency residuals across entire volumes. This coarse-to-fine modeling strategy simplifies optimization, enhances training stability, and effectively preserves subtle structures without the limitations of an autoencoder bottleneck. Extensive experiments conducted on the LIDC-IDRI and RAD-ChestCT datasets demonstrate that PRDiT consistently outperforms state-of-the-art models, such as HA-GAN, 3D LDM and WDM-3D, achieving significantly lower 3D FID, MMD and Wasserstein distance scores.
Zhenkai Zhang, Markus Hiller, Krista A. Ehinger +1
School of Computing and Information Systems, The University of Melbourne · School of Computing
Computed tomography (CT) provides rich 3D anatomical detail but is often constrained by high radiation exposure, substantial costs, and limited availability. Standard chest X-rays are cost-effective and widely accessible, but provide only 2D projections with limited pathological information. Reconstructing 3D CT volumes from 2D X-rays could markedly increase diagnostic accessibility, yet existing methods rely predominantly on synthetic X-ray projections, limiting clinical generalization. We propose AXON, a multi-stage diffusion-based framework that reconstructs 3D CT volumes directly from real X-rays with substantially improved fidelity over existing approaches. AXON follows a coarse-to-fine paradigm: a Brownian Bridge diffusion model first captures global anatomical structure, and a ControlNet-guided refinement stage then enhances local intensity detail and anatomical realism. To alleviate the depth ambiguity inherent in 2D-to-3D reconstruction, AXON incorporates bi-planar X-ray views, enabling more accurate spatial reasoning and structural recovery. A dedicated super-resolution module further increases the spatial resolution of the generated volumes. Experiments on public and external datasets show that AXON consistently surpasses state-of-the-art approaches while generalizing across diverse clinical distributions. At our highest-resolution bi-planar setting, AXON achieves an 11.9% improvement in PSNR and an 11.0% increase in SSIM over the strongest baseline evaluated at that resolution. In the 1283 single-planar setting, it maintains a lead of 7.8% in PSNR on LIDC-IDRI, with larger margins on the external clinical dataset of 8.0% in PSNR and 16.9% in SSIM. Our code is available at https://github.com/ai-med/AXON/.
Martin Rath, Morteza Ghahremani, Yitong Li +3
Technical University of Munich (TUM), Germany · Munich Center for Machine Learning (MCML), Germany · University of Western Australia (UWA), Australia
Panoramic radiograph (PR) is fundamentally used in routine dental care, but it inherently provides only a two-dimensional (2D) projection of complex three-dimensional (3D) craniofacial anatomy. Most existing learning-based methods attempt to computationally recover this 3D information by directly regressing native cone-beam computed tomography (CBCT) volumes from PR. However, this direct mapping requires the model to simultaneously learn common anatomical structures and patient-specific morphological variations. This entangled formulation makes the ill-posed 2D-to-3D inverse problem highly ambiguous, often producing over-smoothed reconstructions with blurred anatomical boundaries. To address this, we propose HyDAR-Pano3D, a two-stage framework that reformulates PR-to-CBCT reconstruction as a disentangled anatomical recovery problem. In Stage 1, a dual-encoder network integrates radiographic features with SAM-derived semantic priors to reconstruct an arch-normalized canonical volume. In Stage 2, an Anatomical Restoration Network predicts a prior-constrained structured deformation field to map this canonical volume back to the native space, restoring individual morphological variations. Experiments on three large-scale datasets show that HyDAR-Pano3D significantly outperforms baseline methods (p<0.05), achieving a 25.76 dB PSNR, 85.70% SSIM, and an 83.83% overall anatomical Dice score. The synthesized volumes successfully support downstream segmentation of whole teeth (82.4% Dice) and the inferior alveolar canal (72.2% Dice), demonstrating that our disentangled approach preserves clinically relevant structures to enable robust anatomy-aware assessment when CBCT data is unavailable.
Yaoyao Yue, Jérôme Schmid, Xiaoshuang Li +2
School of Computer Science, the University of Sydney, Australia · Geneva School of Health Sciences, HES-SO University of Applied Sciences and Arts Western Switzerland, Geneva, Switzerland · Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, China +1