Sustainable Transformer Neural Network Acceleration with Stochastic Photonic Computing
Organizations: Department of Electrical and Computer Engineering, Colorado State University, Fort Collins, CO, USA · Department of Electrical and Computer Engineering, University of Kentucky, Lexington, KY, USA
Abstract
Transformers achieve state-of-the-art performance in natural language processing, vision, and scientific computing, but demand high computation and memory. To address these challenges, we present ASTRA, the first silicon-photonic accelerator leveraging stochastic computing for transformers. ASTRA employs novel optical stochastic multipliers and unary/analog homodyne accumulation in a crosstalk-minimal organization to efficiently process dynamic tensor computations. Evaluations show at least 7.6x speedup and 1.3x lower energy overheads compared to state-of-the-art accelerators, highlighting ASTRA's potential for efficient, scalable, and sustainable transformer inference.
Efficient Transformer Inference
Efficient Transformer Inference studies methods and systems that reduce the latency, memory, and energy needed to run Transformer models.
See all 115 papers