AI Analysis

Abstract

Atik is an open-source RoCC accelerator for RocketChip, designed to accelerate transformer inference by executing dense matrix multiplication and attention operations. It utilizes a shared fixed-point MAC mesh and processes BF16 tensors. A streaming online-softmax schedule is employed to avoid materializing the full score matrix. Evaluation on FireSim shows the 8x8 configuration achieves up to 31.5x peak matrix multiplication speedup and 225.8x peak attention speedup compared to a scalar BF16 reference. Hybrid projections for single-layer workloads indicate aggregate speedups for ViT, TinyBERT, and GPT-2 prefill.

Key Highlights

• Atik is an open-source RoCC accelerator for RocketChip.
• It accelerates dense matrix multiplication and attention for transformer inference.
• The accelerator uses one shared fixed-point MAC mesh.
• Atik accepts and returns BF16 tensors.
• A streaming online-softmax schedule is used, which never materializes the full score matrix.
• The 8x8 configuration achieved 31.5x peak matmul speedup and 225.8x peak attention speedup relative to a scalar Rocket BF16 reference.
• All 137 logged FireSim test records passed their benchmark tolerance with zero reported output mismatches.

Technical Details

• Atik is an open-source RoCC accelerator designed for integration with RocketChip.
• It executes matrix multiplication and dense attention operations using a single shared fixed-point MAC mesh.
• The accelerator is designed to accept and return BF16 tensors.
• A streaming online-softmax schedule is implemented to prevent the full score matrix from being materialized.
• Invocation of Atik is managed through a small descriptor-based ABI.
• Evaluation included cycle-by-cycle analysis of three meshes on FireSim and reporting of standalone Sky130 physical-design area.
• The 8x8 configuration demonstrated a 31.5x peak speedup for matrix multiplication and a 225.8x peak speedup for attention, compared to a scalar Rocket BF16 reference.
• Hybrid projections for single-layer workload-derived C models, utilizing directly measured accelerator-kernel cycles, showed aggregate speedups of 44.2x for ViT, 77.1x for TinyBERT, and 32.3x for GPT-2 prefill.

Implications

• Atik provides a hardware accelerator for RISC-V RocketChip systems, specifically for transformer inference tasks.
• Its open-source nature allows for integration and potential modification within the RISC-V ecosystem.
• The shared fixed-point MAC mesh design offers a unified hardware approach for both matrix multiplication and dense attention.
• The use of BF16 tensors and a streaming online-softmax schedule indicates an optimized approach for data handling and memory efficiency in transformer operations.
• The reported peak speedups suggest significant performance gains for core transformer operations compared to scalar processing.

Limitations

• The source content is a feed excerpt or paper abstract, not the full article, and claims have not been independently verified.
• Hybrid projections for workload-derived C models are based on directly measured accelerator-kernel cycles and are not full-model hardware replays.

Source: https://arxiv.org/abs/2611.09522