? 一种基于VM的支持无界流传递的可伸缩通道
Journal of Computer Science and Technology
Quick Search in JCST
 Advanced Search 
      Home | PrePrint | SiteMap | Contact Us | Help
 
Indexed by   SCIE, EI ...
Bimonthly    Since 1986
Journal of Computer Science and Technology 2017, Vol. 32 Issue (6) :1288-1304    DOI: 10.1007/s11390-017-1801-4
Regular Paper << Previous Articles | Next Articles >>
一种基于VM的支持无界流传递的可伸缩通道
Yu Zhang, Senior Member, CCF, Member, ACM, IEEE, Yu-Fen Yu, Hui-Fang Cao, Jian-Kang Chen, Qi-Liang Zhang
School of Computer Science and Technology, University of Science and Technology of China, Hefei 230027, China
CHAUS:Scalable VM-Based Channels for Unbounded Streaming
Yu Zhang, Senior Member, CCF, Member, ACM, IEEE, Yu-Fen Yu, Hui-Fang Cao, Jian-Kang Chen, Qi-Liang Zhang
School of Computer Science and Technology, University of Science and Technology of China, Hefei 230027, China

摘要
参考文献
相关文章
Download: [PDF 834KB]  
摘要 流处理,作为一种特殊的数据流执行模型,为优化和自动化并行提供了广泛的机会。通常,一个流应用可表示成一个通过FIFO通道通信的多个计算阶段组成的图。在共享内存环境中,FIFO通道典型地表示为一个固定大小且被生产者和消费者共享的、需要被同步的缓冲区。然而,随着并发阶段中工作者数量的增加,这种同步开销(如争用和等待时间)也将急剧上升,严重影响应用程序的性能。为此,本文提出一种新型的多线程模型,它缺省地隔离各线程的内存空间,并提供高层编程抽象——Chaus通道(Channel for Unbounded Streaming)——用于支持线程之间可伸缩的单播或多播通信。Chaus模型隐藏了底层的同步细节以简化编程,但要求用户事先声明通道的生产者-消费者关系。Chaus运行时系统负责保证按用户声明的生产-消费关系、在任意用户自定义数据项粒度上实施可靠的并发数据传输。为了达到对流的无界缓存和减少同步开销,我们提出一种基于虚拟内存的解决方案来实现可伸缩的Chaus通道。我们成功地用Chaus多线程编程模型重写PARSEC的dedup和ferret,并用它实现一个类似Phoenix的MapReduce库,从而检验了Chaus的可编程性。实验结果表明,基于Chaus实现的应用程序的运行速度高于对应的Pthreads程序,并且Chaus的可伸缩性最好。其中,有3个应用程序的Chaus版本在16和32个内核上仅花费不超过0.17X的Pthreads版本的运行时间。
关键词流处理   线程模型   流水线并行   无界通道   虚拟内存     
Abstract: Stream processing is a special form of the dataflow execution model that offers extensive opportunities for optimization and automatic parallelism. A streaming application is represented by a graph of computation stages that communicate with each other via FIFO channels. In shared-memory environment, an FIFO channel is classically a common, fixed-size synchronized buffer shared between the producer and the consumer. As the number of concurrent stage workers increases, the synchronization overheads, such as contention and waiting times, rise sharply and severely impair application performance. In this paper, we present a novel multithreaded model which isolates memory between threads by default and provides a higher level abstraction for scalable unicast or multicast communication between threads-CHAUS (Channel for Unbounded Streaming). The CHAUS model hides the underlying synchronization details, but requires the user to declare producer-consumer relationship of a channel in advance. It is the duty of the runtime system to ensure reliable data transmission at data item granularity as declared. To achieve unbounded buffer for streaming and reduce the synchronization overheads, we propose a virtual memory based solution to implement a scalable CHAUS channel. We check the programmability of CHAUS by successfully porting dedup and ferret from PARSEC as well as implementing MapReduce library with Phoenix-like API. The experimental results show that workloads built with CHAUS run faster than those with Pthreads, and CHAUS has the best scalability compared with two Pthread versions. There are three workloads whose CHAUS versions only spend no more than 0.17x runtime of Pthreads on both 16 and 32 cores.
Keywordsstreaming   thread model   pipeline parallelism   unbounded channel   virtual memory     
Received 2016-09-11;
本文基金:

This work was supported by the National Key Research and Development Program of China under Grant No. 2016YFB1000403.

About author: Yu Zhang is an associate professor in School of Computer Science and Technology,University of Science and Technology of China (USTC),Hefei.She received her Ph.D.degree in computer software and theory from USTC,Hefei,in 2005.Her research interests are software security,the construction and evaluation of efficient parallel systems.
引用本文:   
Yu Zhang, Yu-Fen Yu, Hui-Fang Cao, Jian-Kang Chen, Qi-Liang Zhang.一种基于VM的支持无界流传递的可伸缩通道[J]  Journal of Computer Science and Technology , 2017,V32(6): 1288-1304
Yu Zhang, Yu-Fen Yu, Hui-Fang Cao, Jian-Kang Chen, Qi-Liang Zhang.CHAUS:Scalable VM-Based Channels for Unbounded Streaming[J]  Journal of Computer Science and Technology, 2017,V32(6): 1288-1304
链接本文:  
http://jcst.ict.ac.cn:8080/jcst/CN/10.1007/s11390-017-1801-4
Copyright 2010 by Journal of Computer Science and Technology