HadaFS - A File System Bridging the Local and Shared Burst Buffer for Exascale

Explore a 16-minute conference talk from USENIX FAST '23 that introduces HadaFS, an innovative file system designed to bridge local and shared burst buffers for exascale supercomputers. Delve into the Localized Triage Architecture (LTA) that addresses ultra-scale expansion and data sharing challenges. Learn about the full-path indexing approach and metadata synchronization strategies that tackle complex metadata management issues in traditional file systems. Discover how HadaFS integrates Hadash, a data management tool that enhances data query efficiency and accelerates data migration between burst buffers and traditional HPC storage. Gain insights into HadaFS's deployment on the Sunway New-generation Supercomputer (SNS), its ability to serve hundreds of applications, and its support for up to 600,000-client scaling. Examine performance evaluations, including metadata and data performance, data migration, and interference avoidance techniques.