SPARC: Accurate Photonic Matrix Multiplication via Representative Vectors
Organizations: NTT, Inc. Japan · Keio University Japan
Abstract
Photonic neural network accelerators have emerged as a promising solution for reducing the power consumption of deep neural network (DNN) inference by executing matrix multiplication on analog photonic hardware. However, their inference accuracy is limited by low-bit quantization required for photonic processing and by analog photonic noise. This paper proposes an approach for improving inference accuracy by exploiting the high similarity among activation vectors in DNNs. Specifically, our approach constructs a representative vector for each group of similar activation vectors, computes its inner product digitally with high precision, and offloads the residual inner products to the photonic accelerator. Since the activation vectors within each group are highly similar, the residual vectors have much smaller norms, thereby reducing the effects of quantization errors and analog photonic noise. Furthermore, our approach preserves the photonic inner-product computation pipeline, allowing seamless integration into existing and future photonic accelerators without hardware modification. Experimental results on six representative DNN models demonstrate that the proposed approach can improve inference accuracy across diverse network architectures and application domains.
Figures & tables
| Model | Original | Proposed | FP32 | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ResNet50 | 75.73 0.46 | 75.87 0.31 | 72.00 0.69 | 1.20 0.00 | 0.20 0.00 | 77.27 0.46 | 77.00 0.35 | 76.07 0.12 | 8.73 3.95 | 0.53 0.46 | 77.20 | ||
| 52.73 0.46 | 72.13 0.90 | 75.00 0.69 | 69.13 0.76 | 26.73 0.12 | 76.67 0.61 | 77.13 0.46 | 77.13 0.46 | 77.53 0.50 | 74.80 1.40 | ||||
| 0.00 0.00 | 0.33 0.31 | 55.33 0.50 | 72.73 0.12 | 73.80 0.20 | 0.27 0.46 | 75.53 1.42 | 77.00 0.00 | 77.13 0.31 | 77.20 0.00 | ||||
| SSD-ResNet34 | 19.91 0.07 | 17.14 0.02 | 5.85 0.01 | 0.00 0.00 | 0.00 0.00 | 19.95 0.01 | 19.92 0.02 | 17.80 0.02 | 0.45 0.01 | 0.00 0.00 | 20.05 | ||
| 19.31 0.02 | 19.70 0.05 | 17.12 0.01 | 11.40 0.01 | 0.38 0.01 | 19.81 0.05 | 19.88 0.02 | 19.77 0.02 | 18.00 0.04 | 3.95 0.01 | ||||
| Noise variance | Approach | ResNet50 | SSD-ResNet34 | 3D U-Net | RNN-T | BERT-Large | DLRM |
|---|---|---|---|---|---|---|---|
| Original | 75.73 0.81 | 17.16 0.02 | 79.36 0.01 | 96.29 0.04 | 92.56 0.09 | 80.80 0.00 | |
| Proposed | 76.40 0.20 | 19.77 0.02 | 81.21 0.01 | 96.30 0.04 | 92.65 0.09 | 80.91 0.00 | |
| Original | 76.07 0.31 | 17.15 0.02 | 79.42 0.00 | 96.28 0.02 | 92.49 0.03 | 80.80 0.00 | |
| Proposed | 76.27 0.31 | 19.76 0.03 | 81.22 0.00 | 96.30 0.01 | 92.66 0.04 | 80.91 0.00 | |
| Original | 75.00 0.69 | 17.12 0.01 | 79.54 0.01 | 96.24 0.02 | 92.47 0.05 | 80.78 0.01 | |
| Proposed | 77.13 0.46 | 19.77 0.02 | 81.22 0.01 | 96.29 0.01 | 92.82 0.38 | 80.91 0.01 |
| Model | w/o reordering | w/o scaling | Proposed |
|---|---|---|---|
| ResNet50 | 74.73 0.12 | 67.73 1.01 | 77.13 0.46 |
| SSD-ResNet34 | 17.04 0.02 | 17.67 0.01 | 19.77 0.02 |
| 3D U-Net | 81.54 0.01 | 79.76 0.01 | 81.22 0.01 |
| RNN-T | 96.30 0.00 | 96.30 0.00 | 96.31 0.01 |
| BERT-Large | 92.62 0.04 | 92.68 0.08 | 92.82 0.38 |
| DLRM | 80.82 0.01 | 80.76 0.02 | 80.91 0.01 |
Photonic Computing
Photonic computing uses light and optical devices to perform computation, aiming for high throughput and lower energy in machine learning and signal processing.
See all 51 papers