Reproducibility and Standardization in gem5 Resources v25.0

Abstract

This paper introduces significant improvements in gem5 and gem5 Resources v25.0 to address critical challenges in simulation reproducibility and standardization within computer architecture research. Key innovations include standardizing disk image creation across x86, ARM, and RISC-V using Packer, alongside providing new, validated resources like kernels and benchmarks. Furthermore, the introduction of a class-based exit event system with hypercalls and the implementation of Suites and MultiSim for parallel full-system simulation orchestration eliminate the need for complex external scripting, reducing setup complexity and enhancing standardization.

Report

Key Highlights

• Standardized Artifact Creation: Disk image creation is standardized across major ISAs (x86, ARM, and RISC-V) using the Packer tool.
• New Resource Injection: gem5 Resources v25.0 adds 12 new disk images, 6 new kernels, and over 200 validated workloads, including the pre-annotated NPB and GAPBS benchmark suites.
• Native Multi-Simulation Orchestration: The introduction of Suites and MultiSim enables parallel full-system simulations directly from gem5 configuration scripts, eliminating previously error-prone external scripting workflows.
• Enhanced Guest-Host Communication: The exit event system has been refactored into a class-based model, complemented by new hypercalls for defining custom communication behaviors, along with the gem5-bridge driver for user-space m5 operations.

Technical Details

• Standardization Tool: Packer is the chosen tool for achieving standardized and reproducible disk-image generation across different Instruction Set Architectures (ISAs).
• Architectures Supported: The standardization efforts cover x86, ARM, and RISC-V architectures.
• Benchmark Inclusion: Validated base images come pre-annotated with popular benchmark suites, specifically NPB (NASA Advanced Supercomputing Parallel Benchmarks) and GAPBS (Graph Algorithm Platform Benchmark Suite).
• Communication Mechanism: Exit events are now class-based, allowing researchers to define custom behaviors, utilizing new hypercalls for enhanced guest-host signaling.
• Monitoring Utility: A dedicated utility is provided for remotely monitoring simulation status.

Implications

• Boosting Reproducibility: By standardizing the creation and delivery of artifacts (images, kernels, workloads), the paper directly tackles the 'reproducibility crisis' plaguing simulation-based computer architecture research.
• Lowering Barriers for RISC-V: Explicit standardization of disk images and resources for RISC-V (alongside x86 and ARM) significantly lowers the entry barrier for researchers developing and testing hardware designs on this rapidly emerging open ISA.
• Increased Research Efficiency: The Suites and MultiSim features drastically reduce the need for complex, bespoke external scripts to coordinate multiple simulation runs, saving researchers substantial time and reducing experimental variance.
• Strengthening the Ecosystem: By improving and standardizing the gem5 Resources repository (now containing over 2000 artifacts), the project supports a more robust, extensible, and collaborative environment for the entire computer architecture simulation community.