We use cookies to improve your experience with our site.

GekkoFS—一种用于高性能计算应用的临时突发缓冲文件系统

GekkoFS—A Temporary Burst Buffer File System for HPC Applications

  • 摘要: 越来越多的科学领域使用高性能计算(HPC)加工并分析大量实验数据,而在当今HPC环境下,储存系统必须能应对新的访问模式。这些模式包括很多元数据操作、小I/O请求,或者随机文件输入输出(I/O)。通用并行文件系统被优化,以实现对大文件的连续共享访问。突发缓冲文件系统为应用创建一个独立文件系统用以存储临时数据。它们在计算节点内聚合了可用的节点本地储存,或使用专用SSD集群,并提供一个比不干涉情况下后端并行文件系统的峰值带宽更高的峰值带宽。然而,突发缓冲文件系统提供了许多在有限时间内独立运行的科学应用不需要的特征。我们提出了GekkoFS,一个临时、高可扩展的文件系统,它已经针对上述提及的使用场景进行了有针对性的优化。GekkoFS提供了非严格的POSIX语义,此语义仅提供了大多数(非全部)应用实际需要的特征。因此,GekkoFS能够提供可扩展的I/O性能,并能在少量节点上完成数百万的元数据操作,明显优于通用并行文件系统。

     

    Abstract: Many scientific fields increasingly use high-performance computing (HPC) to process and analyze massive amounts of experimental data while storage systems in today's HPC environments have to cope with new access patterns. These patterns include many metadata operations, small I/O requests, or randomized file I/O, while general-purpose parallel file systems have been optimized for sequential shared access to large files. Burst buffer file systems create a separate file system that applications can use to store temporary data. They aggregate node-local storage available within the compute nodes or use dedicated SSD clusters and offer a peak bandwidth higher than that of the backend parallel file system without interfering with it. However, burst buffer file systems typically offer many features that a scientific application, running in isolation for a limited amount of time, does not require. We present GekkoFS, a temporary, highly-scalable file system which has been specifically optimized for the aforementioned use cases. GekkoFS provides relaxed POSIX semantics which only offers features which are actually required by most (not all) applications. GekkoFS is, therefore, able to provide scalable I/O performance and reaches millions of metadata operations already for a small number of nodes, significantly outperforming the capabilities of common parallel file systems.

     

/

返回文章
返回