To SIMD or Not to SIMD: Comparing RVV vs. CGRA SoCs for Non-Linear Functions in AI
Organizations: Fujitsu Research of Japan, Fujitsu Limited, Kawasaki, Japan
Abstract
Non-linear functions are an important class of operations in modern large language models. However, chip vendors building for AI have so far relied on traditional vector or scalar processing units for non-linear functions. The performance of these units has not kept pace with matrix multiplication units (MMUs), which can be found in both high-performance and edge-computing devices from almost every vendor. Reconfigurable units such as FPGAs and CGRAs have been proposed in the literature, but have not been systematically compared with traditional units backed by MMUs. In this paper, we build two SoCs: one using an RVV vector processor and the other using a latency-insensitive (elastic) CGRA, both coupled with an MMU and driven by a single-core CPU. We then propose mappings for common feed-forward networks on our SoCs and find that the CGRA reaches a maximum speedup in clock cycles of 7.12x over RVV on the activation stage of simple point-wise non-linear functions like SiLU, whereas the RVV reaches a maximum speedup of 9.8x over the CGRA on complex non-linear functions such as . Over the enclosing feed-forward block the corresponding maxima are 3.16x and 4.24x.
Figures & tables
| Name | Equation |
|---|---|
| SwiGLU [ 66 , 21 , 78 ] | |
| Attention [ 42 , 21 , 73 ] |
| funct7 | Name | Purpose |
|---|---|---|
| 0 | CFG | Scan the configuration bitstream into the fabric — run once before any compute. |
| 1 | SET | Write one configuration register (scratchpad pointer, collector latency, or a DMA base address). |
| 2 | FILL | Start a non-blocking DMA read that fills one scratchpad half with operands. |
| 3 | PUSH2 | Fill the scratchpad from core registers, two operand pairs per instruction. |
| 4 | RUN | Arm the streaming engine over one scratchpad half and return immediately. |
| 5 | WAIT | Ask whether the engine or the DMA has finished, either blocking or polling. |
| SoC | Attention | SwiGLU | |||||||
|---|---|---|---|---|---|---|---|---|---|
| Total | Matmul | Activation | Softmax | Total | Matmul | Activation | Activation | ||
| cycles | cycles | cycles | proportion | cycles | cycles | cycles | proportion | ||
| ME-VE | 16 | 6,943 | 3,628 | 3,299 | 47.52% | 10,955 | 2,240 | 8,715 | 79.55% |
| 32 | 27,610 | 16,582 | 11,022 | 39.92% | 44,687 | 10,068 | 34,619 | 77.47% | |
| 64 | 143,087 | 103,508 | 39,456 | 27.57% | 200,890 | 62,314 | 138,576 | 68.98% | |
| 128 | 862,834 | 713,800 | 149,015 | 17.27% | 983,055 | 430,620 | 552,435 | 56.20% | |
| State | Description | Cycles / element (N=128) |
| RESET | Per chunk fabric reset. | |
| ARM | Re-arm row constants (, ) and stream pointers cleared by the reset. | |
| RDACC | Read and clear the on-fabric accumulator. | |
| RUN | Execute one iteration of a chunk of 24 elements. | |
| FILL | Fetch a chunk over DMA and place it in the scratch pad. | |
| DRAIN | Barrier waiting for RUN to finish. |
| Step | Factor | Cumulative |
|---|---|---|
| Ideal (II ) | — | 1.00 |
| Two passes | 2.00 | |
| Operand duplication | 4.00 | |
| Operand prologue | 5.50 | |
| Streaming rate | 27.53 | |
| Restart fill | 40.55 |