TALU: A Reconfigurable Tiled-ALU Grid for RISC-V

AI Analysis

Abstract

Key Highlights

• TALU integrates an N×N grid of registered ALU tiles directly into the RISC-V execute stage, replacing the conventional single ALU.
• The architecture efficiently addresses common bottlenecks in lane-based vector processors, such as reductions and deep dependent expression chains, through spatial computation and diagonal forwarding.
• Achieves substantial performance improvements, demonstrating 1.8-5.5x speedup over RISC-V2 and 4.5-15.5x over Vicuna on benchmark kernels.
• Significantly reduces the cost of reductions, with a dot/add ratio of 1.05x compared to 1.96x for RISC-V2 and 1.26x for Vicuna.
• Implemented in 45nm, TALU operates at 1.03 GHz and occupies 0.30 mm², which is half the area of the RISC-V2 baseline.
• Supports scalar, single deep-path, and multi-lane vector execution modes on a single, unified hardware fabric.

Technical Details

• TALU is a 5-stage RISC-V core where the execute (EX) stage consists of an N×N grid of registered ALU tiles.
• Each tile contains a combinational RV32I + MUL ALU and an output register, with operands routed from non-backward neighbors to ensure acyclic dataflow across columns.
• Diagonal (NW/SW) forwarding links enable in-pass reductions, allowing an N-element reduction to collapse spatially to the top-right corner.
• A minimal custom ISA extension includes GCFG (tile configuration), GCFGI (per-tile constant loading), GSTART (grid execution on register-resident operands), and GFLOW (memory region streaming).
• Tile (0,0) functions as the scalar ALU for ordinary RV32I + MUL code, integrating scalar and vector operations within the same fabric.
• The memory model supports streaming 3N words/cycle (2N read, N write) via a banked TCDM, with a 16-bank model showing performance within 3 cycles of ideal.
• A prototype LLVM pass automates mapping of annotated single-pass C loops, classifying them as map or reduction and placing nodes based on DAG depth.
• Physical implementation in 45nm technology achieves a clock frequency of 1.03 GHz with an area of 0.30 mm² for N=4, including pipelined multipliers.

Implications

• TALU offers a more efficient architectural paradigm for handling data-parallel workloads, particularly for reductions and deep dependent expression chains, compared to traditional lane-based vector processors.
• The integration of a reconfigurable spatial datapath directly into the execute stage provides a unified and flexible compute fabric for scalar, single deep-path, and multi-lane vector operations.
• The significant performance gains and reduced area demonstrate the potential for more compact and higher-throughput RISC-V processor designs for DSP and Machine Learning applications.
• The ability to perform segmented reductions efficiently, which conventional adder trees cannot express, expands the range of kernels that can be accelerated in hardware.
• The custom ISA extension provides a programmable interface for fine-grained control over the reconfigurable grid, enabling flexible and optimized kernel execution.
• The design highlights that removing per-element issue bottlenecks is a dominant factor in achieving performance improvements in data-parallel computing, which can then be multiplied by array width.

Limitations

• The current custom extension has a two-operation-per-pass ceiling, which limits the efficiency of multi-pass kernels like FIR, median, and SAD, requiring multiple streaming passes.
• The prototype LLVM pass is limited to single-pass C loops and cannot automatically map multi-pass kernels, which currently require manual mapping.
• The three-input median kernel was rejected by the compiler due to GFLOW supplying only two streams, indicating a limitation in handling kernels requiring more input streams.
• Direct comparison of power and energy per element with RISC-V2 is limited due to different reporting conditions (typical corner vs. worst-case, 0.1-activity power estimate for TALU).
• While the memory model supports high bandwidth, concurrent read streams can cause bank collisions if not offset by a multiple of N words, potentially impacting performance.
• The clock and power figures for TALU and RISC-V2 are not corner-matched, which means direct execution time ratios may not be perfectly accurate.