We use cookies to improve your experience with our site.

海量存储系统

Massive Storage Systems

  • 摘要: 科学计算、多媒体应用、电子商务等对信息存储需求越来越大,数据曾爆炸式增长趋势。存储系统通过研究实现高效的数据布局、灵活的管理等,已从传统的隶属于计算机系统变为直接连接网络的智能存储系统,并面临着要解决高性能、高可扩展性、高可用性、高可靠性和高安全性等难题。本文在国家自然科学基金等项目的资助下,重点研究了五种在线海量存储系统和一种离线海量存储系统。采用异构双通道网络磁盘阵列构成的存储池避免了数据在服务器中的大量存储转发,比采用传统的磁盘阵列构成的系统提高性能 2~3倍。文中介绍了两种用于局域网的高性能分布式存储系统,其一是采用VI作为传输协议的基于虚拟接口的存储系统VISA, 通过设计实现专用于存储系统的vSCSI (VI-attached SCSI) 协议,并在块级实现虚拟化,使得系统性能优于IP SAN;其二是设计并实现了提供高性能和高可靠性的容错PVFS(fault-tolerant parallel virtual file system),通过建模分析了影响系统性能的因素。文中还介绍了一种用于广域网范围的分布式存储系统,其主要特点是采用存储服务提供器提供服务并用作用户代理。对象存储系统除了存储数据外,还在对象中封装有属性和方法,通过探讨自适应策略触发机制、采用机器学习等方法提高存储系统的智能性,实现自组织和自管理的海量存储系统。另外文中还介绍了磁带虚拟化技术这种典型的用于备份和归档的离线海量存储系统技术。最后,给出了一种对于不同类型存储系统进行统一管理的基于域的存储管理方法。

     

    Abstract: To accommodate the explosively increasing amount of data in many areassuch as scientific computing and e-Business, physical storage devicesand control components have been separated from traditional computingsystems to become a scalable, intelligent storage subsystem that, whenappropriately designed, should provide transparent storage interface,effective data allocation, flexible and efficient storage management,and other impressive features. The design goals and desirable featuresof such a storage subsystem include high performance, high scalability,high availability, high reliability and high security. Extensiveresearch has been conducted in this field by researchers all over theworld, yet many issues still remain open and challenging. This paperstudies five different online massive storage systems and one offlinestorage system that we have developed with the research grant supportfrom China. The storage pool with multiple network-attached RAIDs avoidsexpensive store-and-forward data copying between the server and storagesystem, improving data transfer rate by a factor of 2--3 over atraditional disk array. Two types of high performance distributedstorage systems for local-area network storage are introduced in thepaper. One of them is the \it Virtual Interface Storage Architecture(VISA) where VI as a communication protocol replaces the TCP/IP protocolin the system. VISA's performance is shown to achieve better than thatof IP SAN by designing and implementing the vSCSI (VI-attached SCSI)protocol to support SCSI commands in the VI network. The other is afault-tolerant parallel virtual file system that is designed andimplemented to provide high I/O performance and high reliability. Aglobal distributed storage system for wide-area network storage isdiscussed in detail in the paper, where a Storage Service Provider isadded to provide storage service and plays the role of user agent forthe storage system. Object based Storage Systems not only store data butalso adopt the attributes and methods of objects that encapsulate thedata. The adaptive policy triggering mechanism (APTM), which borrowsproven machine learning techniques to improve the scalability of objectstorage systems, is the embodiment of the idea about smart storagedevice and facilitates the self-management of massive storage systems. Atypical offline massive storage system is used to backup data or storedocuments, for which the tape virtualization technology is discussed.Finally, a domain-based storage management framework for different typesof storage systems is presented in the paper.

     

/

返回文章
返回