Skip to main content
Back to timeline
arXivSource publication:

Atik puts matmul and attention on one fixed-point MAC mesh, reaching 225.8x peak attention speedup on a Rocket core

Synopsis

The open-source accelerator Atik attaches to a RISC-V RocketChip core over RoCC and runs dense matrix multiplication, attention scoring and probability-value accumulation on one shared fixed-point MAC mesh, while tensors enter and leave as BF16. Against the BF16 scalar reference on the same Rocket core, the 88 configuration reaches 31.5x peak matmul speedup and 225.8x peak attention speedup in cycle-accurate FireSim runs; the 44.2, 77.1 and 32.3 figures for ViT, TinyBERT and GPT-2 prefill are hybrid projections, not full-model hardware replays.

Source-provided article image: Atik: A Shared-Mesh Transformer Accelerator for RISC-V
arXiv

Interpretation

Atik makes one tightly coupled coprocessor carry both of the heaviest transformer inference computations, dense matrix multiplication and attention, rather than accelerating only one of them. Earlier specialized hardware mostly built attention or softmax as standalone blocks, targeted a single narrow operation, or used architectural full-system simulation instead of executing integrated RTL on an FPGA. In cycle-accurate FireSim emulation against a BF16 software reference on the same Rocket core, the 88 configuration reaches 31.5x peak matmul speedup and 225.8x peak attention speedup; all 137 logged records pass their benchmark tolerance, with a largest reported absolute difference of 0.0625.

Matmul, QK scoring and probability-value accumulation share one fixed-point multiply-accumulate mesh, and attention proceeds by streaming online softmax that never materializes the full score matrix. The usual approach provisions a separate datapath per operation or materializes the whole score matrix before normalization, which inflates storage and data movement. The mesh is configurable as 4, 16 or 64 MAC lanes, and all three instances expose the same descriptor ABI and run the same unmodified benchmark binaries; software fills an 80-byte descriptor and polls a status word, while tiling, DMA, packing, numeric conversion and softmax bookkeeping stay in hardware.

Tensors enter and leave as row-major BF16, while the internal datapath computes in 24-bit signed fixed point with 64-bit accumulation and implements exponentiation and reciprocal with 256-entry lookup tables. Most comparable designs commit to quantized or integer datatypes such as INT8, or approximate softmax with lookup tables or powers of two, leaving numerical trade-offs to be evaluated at the workload level. These conversions and lookup tables are approximations, so the reported checks are per-benchmark absolute-error tolerances rather than bit-exact BF16 arithmetic or task-level accuracy.

Perspective

The evaluation covers two standalone kernels and single-layer C reference models of ViT, TinyBERT and GPT-2 prefill, with performance numbers from cycle-accurate FireSim emulation on AWS F2 at an effective target frequency of about 10 MHz; the area study targets standalone AtikCore on Sky130 and excludes RocketTile, ChipTop, routing closure, timing, power and tapeout.

The speedups are measured against the same in-order Rocket core, which has no SIMD or vector unit, so part of the absolute gain reflects how poorly a scalar pipeline handles dense BF16 work; softmax bookkeeping and final normalization run serially on one scalar lane, and that region becomes the dominant sequential cost for very large score tiles; the autoregressive decode stage with a KV cache is not yet accelerated.