Update NDP: On Offloading Modifications to Smart Storage with Transactional Guarantees in Near-Data Processing DBMS

Abstract

This work introduces "Update NDP," a novel approach for extending Near-Data Processing (NDP) capabilities to handle database modification operations safely and efficiently. It proposes offloading complex update logic directly onto smart storage hardware to leverage performance gains while minimizing data movement. Crucially, the system provides full transactional guarantees, ensuring atomicity and consistency even when modifications are executed away from the host CPU.

Report

Key Highlights

• Transactional Offloading: The core innovation is moving beyond read-only acceleration in NDP to enable the offloading of complex database modifications (updates and deletes).
• Smart Storage Utilization: The framework specifically targets computational storage devices (CSDs) or Smart SSDs to execute the update logic directly on the storage controller.
• ACID Guarantees: A primary focus is solving the challenge of maintaining transactional integrity (ACID properties) when data manipulation occurs outside the traditional centralized DBMS host.
• Reduced Data Movement: By executing updates near the persistent medium, the system significantly reduces the volume of data that must be transferred back and forth between the host and storage.

Technical Details

• Architecture Modification: The standard DBMS architecture is enhanced with a dedicated component to partition update queries and determine which segments can be safely executed on the storage device.
• Smart Storage Requirements: The implementation necessitates CSDs equipped with sufficient processing power (e.g., custom embedded cores) to handle relational algebra operations related to modifications, indexing, and logging.
• Commit Protocol: A specialized, possibly two-phase, commit protocol is implied to synchronize the state changes performed by the smart storage engine with the overall database transaction managed by the host DBMS.
• Logging and Recovery: The system must incorporate mechanisms for distributed logging, ensuring that partial updates resulting from failures on the smart storage side can be rolled back or recovered consistently upon host restart.

Implications

• NDP Maturity: This research marks a critical step in maturing Near-Data Processing from a specialized query accelerator into a viable foundation for general-purpose transactional database systems.
• RISC-V Ecosystem Demand: The necessity for specialized, low-power, high-performance cores embedded within storage controllers perfectly aligns with the strengths of the RISC-V architecture. These systems create a substantial new market for RISC-V processors designed specifically to handle complex DBMS execution pipelines, index manipulation, and transactional metadata management on the SSD.
• Storage Interface Evolution: The approach necessitates deeper integration and standardization of computational storage interfaces, potentially driving extensions to NVMe or other protocols to natively support transactional commands and data flow control.
• Database Distribution: It enables new paradigms for highly distributed database systems where storage devices are not merely passive memory but active participants in maintaining data consistency and executing core logic.