Mixed-Precision Training

Latest papers

8 papers2 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

Architecture-Aware Mixed Precision for Replica-Exchange Training of Geometry-Aware Neural Operators with Uncertainty Quantification
Nov 2, 2026Ujin Jeon, Michael Penwarden, Wenqian Chen +1Mixed-Precision Training Geometry-Aware Neural Operators
Department of Electrical and Computer Engineering, Purdue University, West Lafayette, 47907, IN, USA · Sandia National Laboratories, 1515 Eubank Blvd SE, Albuquerque, 87123, NM, USA · Pacific Northwest National Laboratory, 3335 Innovation Blvd, Richland, 99354, WA, USA +1
Where Rounding Happens Matters: Rounding-Location Alignment Stabilizes LLM Reinforcement Learning
Oct 23, 2026Yijing Ke, Tianle Cai, Ruosong WangMixed-Precision Training RL for Language Models
ByteDance Seed · Peking University
An FPGA-Based Spiking Neural Network for Spatiotemporal Pattern Learning and Classification
Oct 12, 2026Jingang Jin, Yue Ma, Zhenhang Zhang +1Mixed-Precision Training Spiking Neural Networks
Department of Electrical Engineering and Computer Science Syracuse University, Syracuse, New York, USA
Making Analog Training Scale: Co-Designing Mapping, Optimizer, and Converters
Sep 29, 2026Zhaoxian Wu, Tayfun Gokmen, Omobayode Fagbohungbe +2Mixed-Precision Training Analog Computing
Cornell Tech and Cornell University · IBM T. J. Watson Research Center · Sandia National Laboratories
NeuronFabric: A Software Reference Architecture for On-Chip Transformer Training with Local Adam
Jun 15, 2026Evgeny UkladchikovDeep Learning Optimization Mixed-Precision Training
Independent Researcher
Enhanced 3D Brain Tumor Segmentation Using Assorted Precision Training
May 5, 2026Adwaitt Pandya, Ozioma C. Oguine, Harita Bhargava +1Mixed-Precision Training Medical Image Segmentation
Department of Computer Science, Oriental Institute of Science and Technology, Bhopal, India · Department of Computer Science, University of Abuja, Abuja, Nigeria
Training Time Prediction for Mixed Precision-based Distributed Training
Apr 17, 2026Minchul Kang, Changyong Shin, Jinwoo Jeong +5Deep Learning Optimization Mixed-Precision Training
Korea University · KT Corporation
Improving Continual Learning for Gaussian Splatting based Environments Reconstruction on Commercial Off-the-Shelf Edge Devices
Mar 9, 2026Ivan Zaino, Matteo Risso, Daniele Jahier Pagliari +3Mixed-Precision Training 3DGS Training
Politecnico di Torino · VERSES

Nov 2, 2026J/K move · Enter open · S save

Architecture-Aware Mixed Precision for Replica-Exchange Training of Geometry-Aware Neural Operators with Uncertainty Quantification

Ujin Jeon, Michael Penwarden, Wenqian Chen, Guang Lin

Department of Electrical and Computer Engineering, Purdue University, West Lafayette, 47907, IN, USA · Sandia National Laboratories, 1515 Eubank Blvd SE, Albuquerque, 87123, NM, USA · Pacific Northwest National Laboratory, 3335 Innovation Blvd, Richland, 99354, WA, USA+1

Mixed-Precision Training Geometry-Aware Neural Operators Memory-Efficient Training Deep Learning Optimization Neural Operators

Open paper AskPDF arXiv

Geometry-aware neural operators face substantial computational and memory demands when learning across complex physical domains. We introduce Architecture-Aware Mixed Precision (AA-MP), a replica-exchange training strategy that allocates numerical precision according to replica roles and architecture-specific operations. By retaining FP32 for refinement while selectively employing BF16 for exploration, AA-MP preserves predictive accuracy while improving training efficiency. Across fluid dynamics and three-dimensional structural mechanics benchmarks, AA-MP achieves comparable accuracy to full-precision training over an ensemble of runs, while reducing hot-replica peak GPU memory by 46.9% and training time by up to 25.8%. It also outperforms naive Automatic Mixed Precision (AMP) in memory efficiency and computational cost, with increasing memory advantages at larger model and geometry scales. Ablation studies show that retaining FP32 in the cold-replica is important for predictive accuracy, while additional FP32 operations in the exploratory replica provide limited benefits relative to their computational cost. Finally, conformal prediction for uncertainty calibration shows that AA-MP achieves near-nominal coverage with narrower prediction intervals than the alternatives. These results advance the scalability of geometry-aware scientific machine learning by reducing computational costs without compromising predictive accuracy or uncertainty quantification.