计算机科学技术学报 ›› 2020,Vol. 35 ›› Issue (1): 194-208.doi: 10.1007/s11390-020-9536-z

• • 上一篇    下一篇

CirroData:另一个基于SQL-on-Hadoop的高性能数据分析引擎

Zheng-Hao Jin1, Haiyang Shi2, Ying-Xin Hu1, Li Zha3,4, Member, CCF, Xiaoyi Lu2, Member, ACM, IEEE   

  1. 1 Business-Intelligence of Oriental Nations Corporation Ltd., Beijing 100102, China;
    2 Department of Computer Science and Engineering, The Ohio State University, Ohio 43210, U.S.A;
    3 Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China;
    4 University of Chinese Academy of Sciences, Beijing 101408, China
  • 收稿日期:2019-07-15 修回日期:2019-10-14 出版日期:2020-01-05 发布日期:2020-01-14
  • 作者简介:Zheng-Hao Jin is the chief architect of Business-Intelligence of Oriental Nations Corporation, Beijing. He received his Master's degree in computer software and theory from Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, in 2001. His current research interests include distributed DBMS, Big Data, cloud computing, and data science.
  • 基金资助:
    This research is supported in part by the Strategic Priority Research Program of the Chinese Academy of Sciences of China under Grant No. XDA19020400.

CirroData: Yet Another SQL-on-Hadoop Data Analytics Engine with High Performance

Zheng-Hao Jin1, Haiyang Shi2, Ying-Xin Hu1, Li Zha3,4, Member, CCF, Xiaoyi Lu2, Member, ACM, IEEE        

  1. 1 Business-Intelligence of Oriental Nations Corporation Ltd., Beijing 100102, China;
    2 Department of Computer Science and Engineering, The Ohio State University, Ohio 43210, U.S.A;
    3 Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China;
    4 University of Chinese Academy of Sciences, Beijing 101408, China
  • Received:2019-07-15 Revised:2019-10-14 Online:2020-01-05 Published:2020-01-14
  • About author:Zheng-Hao Jin is the chief architect of Business-Intelligence of Oriental Nations Corporation, Beijing. He received his Master's degree in computer software and theory from Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing, in 2001. His current research interests include distributed DBMS, Big Data, cloud computing, and data science.
  • Supported by:
    This research is supported in part by the Strategic Priority Research Program of the Chinese Academy of Sciences of China under Grant No. XDA19020400.

PDF (PC)

115

本文介绍了一个针对大数据分析工作负载而设计的高性能SQL-on-Hadoop数据库系统-CirroData。这个自研的面向企业级联机分析处理的大规模并行数据库系统已有超过七年的历史。本文主要介绍CirroData如何实现高性能数据处理的设计与实现细节，其中包含的多种优化技术在文中都有讨论。CirroData客户的日常使用证明了这些技术的有效性和高性能。通过基于TPC-H的标准化测试和多个实际应用案例的评测及检验表明，CirroData的性能优于社区中同类的多种数据库系统，例如Spark+Hive、Spark+HBase、Impala、DB-X/Y、Greenplum和HAWQ等。在TPC-H的查询负载下，CirroData与Greenplum、HAWQ和Spark的性能加速比可达4.99倍。实际应用评估结果表明，CirroData的性能明显优于Spark+Hive、Spark+HBase，性能提升可以分别达到8.4倍和38.8倍。与此同时，与Greenplum、DB-X、Impala、DB-Y和HAWQ相比，CirroData在有些应用负载上可以获得的性能加速比分别可达20倍、100倍、182.5倍、92.6倍和55.5倍。

关键词: CirroData, 高性能, SQL-on-Hadoop, 联机分析处理, 大数据

Abstract: This paper presents CirroData, a high-performance SQL-on-Hadoop system designed for Big Data analytics workloads. As a home-grown enterprise-level online analytical processing (OLAP) system with more than seven-year research and development (R&D) experiences, we share our design details to the community about how to achieve high performance in CirroData. Multiple optimization techniques have been discussed in the paper. The effectiveness and the efficiency of all these techniques have been proved by our customers' daily usage. Benchmark-level studies, as well as several real application case studies of CirroData, have been presented in this paper. Our evaluations show that CirroData can outperform various types of counterpart database systems in the community, such as "Spark+Hive", "Spark+HBase", Impala, DB-X/Y, Greenplum, HAWQ, and others. CirroData can achieve up to 4.99x speedup compared with Greenplum, HAWQ, and Spark in the standard TPC-H queries. Application-level evaluations demonstrate that CirroData outperforms "Spark+Hive" and "Spark+HBase" by up to 8.4x and 38.8x, respectively. In the meantime, CirroData achieves the performance speedups for some application workloads by up to 20x, 100x, 182.5x, 92.6x, and 55.5x as compared with Greenplum, DB-X, Impala, DB-Y, and HAWQ, respectively.

Key words: CirroData, high performance, SQL-on-Hadoop, online analytical processing (OLAP), Big Data

[1] Fox G C, Qiu J, Kamburugamuve S, Jha S, Luckow A. HPC-ABDS high performance computing enhanced Apache big data stack. In Proc. the 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing, May 2015, pp.1057-1066.
[2] Qiu J, Jha S, Luckow A, Fox G C. Towards HPCABDS:An initial high-performance big data stack. http://grids.ucs.indiana.edu/ptliupages/publications/nisthpc-abds.pdf,June 2019.
[3] Shvachko K, Kuang H, Radia S, Chansler R. The Hadoop distributed file system. In Proc. the 26th IEEE Symposium on Mass Storage Systems and Technologies, May 2010, Article No. 9.
[4] Zaharia M, Chowdhury M, Franklin M J, Shenker S, Stoica I. Spark:Cluster computing with working sets. In Proc. the 2nd USENIX Workshop on Hot Topics in Cloud Computing, June 2010, Article No. 5.
[5] Thusoo A, Sarma J S, Jain N, Shao Z, Chakka P, Anthony S, Liu H, Wyckoff P, Murthy R. Hive-A warehousing solution over a Map-Reduce framework. Proceedings of the VLDB Endowment, 2009, 2(2):1626-1629.
[6] Kornacker M, Behm A, Bittorf V et al. Impala:A modern, open-source SQL engine for Hadoop. In Proc. the 7th Biennial Conference on Innovative Data Systems Research, January 2015, Article No. 5.
[7] Chang L, Wang Z W, Ma T et al. HAWQ:A massively parallel processing SQL engine in Hadoop. In Proc. the 2014 ACM SIGMOD International Conference on Management of Data, June 2014, pp.1223-1234.
[8] Costea A, Ionescu A, Raducanu B et al. VectorH:Taking SQL-on-Hadoop to the next level. In Proc. the 2016 International Conference on Management of Data, June 2016, pp.1105-1117.
[9] Hunt P, Konar M, Junqueira F P, Reed B. ZooKeeper:Wait-free coordination for Internet-scale systems. In Proc. the 2010 USENIX Annual Technical Conference, June 2010, Article No. 14.
[10] Chris L, Adve V. LLVM:A compilation framework for lifelong program analysis & transformation. In Proc. the 2nd IEEE/ACM International Symposium on Code Generation and Optimization, March 2004, pp.75-88.
[11] Neumann T. Efficiently compiling efficient query plans for modern hardware. Proceedings of the VLDB Endowment, 2011, 4(9):539-550.
[12] Neumann T, Leis V. Compiling database queries into machine code. IEEE Data Eng. Bull., 2014, 37(1):3-11.
[13] Shamgunov N. The MemSQL in-memory database system. In Proc. the 2nd International Workshop on in Memory Data Management and Analytics, September 2014, Article No. 1.
[14] Tan C K. Vitesse DB:100% PostgreSQL, 100X faster for analytics. The 2nd South Bay PostgreSQL Meetup, 2015. https://www.meetup.com/postgresql-1/events/221039792/,Nov.2019.
[15] Lu X, Liang F, Wang B, Zha L, Xu Z. DataMPI:Extending MPI to Hadoop-like big data computing. In Proc. the 28th International Parallel and Distributed Processing Symposium, May 2014, pp.829-838.
[16] Liang F, Lu X. Accelerating iterative big data computing through MPI. Journal of Computer Science and Technology, Mar. 2015, 30(2):283-294.
[17] Gugnani S, Lu X, Qi H L, Zha L, Panda D K. Characterizing and accelerating indexing techniques on distributed ordered tables. In Proc. the 2017 IEEE International Conference on Big Data, December 2017, pp.173-182.
[18] Kemper A, Neumann T. HyPer:A hybrid OLTP&OLAP main memory database system based on virtual memory snapshots. In Proc. the 27th International Conference on Data Engineering, April 2011, pp.195-206.
[19] Pavlo A, Angulo G, Arulraj J et al. Self-driving database management systems. In Proc. the 8th Biennial Conference on Innovative Data Systems Research, January 2017, Article No. 14.
[20] Thusoo A, Sarma J S, Jain N, Shao Z, Chakka P, Zhang N, Antony S, Liu H, Murthy R. Hive-A petabyte scale data warehouse using Hadoop. In Proc. the 26th IEEE International Conference on Data Engineering, March 2010, pp.996-1005.
[21] Barber R, Garcia-Arellano C, Grosman R et al. Evolving databases for new-gen big data applications. In Proc. the 8th Biennial Conference on Innovative Data Systems Research, January 2017, Article No. 2.
[22] Kallman R, Kimura H, Natkins J et al. H-Store:A highperformance, distributed main memory transaction processing system. Proceedings of the VLDB Endowment, 2008, 1(2):1496-1499.
[23] Pavlo A, Jones E P, Zdonik S. On predictive modeling for optimizing transaction execution in parallel OLTP systems. Proceedings of the VLDB Endowment, 2011, 5(2):85-96.
[24] Serafini M, Mansour E, Aboulnaga A, Salem K, Rafiq T, Minhas U F. Accordion:Elastic scalability for database systems supporting distributed transactions. Proceedings of the VLDB Endowment, 2014, 7(12):1035-1046.
[25] Taft R, Mansour E, Serafini M, Duggan J, Elmore A J, Aboulnaga A, Pavlo A, Stonebraker M. E-store:Finegrained elastic partitioning for distributed transaction processing systems. Proceedings of the VLDB Endowment, 2014, 8(3):245-256.
[26] Mahajan K, Chowdhury M, Akella A, Chawla S. Dynamic query re-planning using QOOP. In Proc. the 13th USENIX Symposium on Operating Systems Design and Implementation, October 2018, pp.253-267.
[27] Cowling J A, Liskov B. Granola:Low-overhead distributed transaction coordination. In Proc. the 2012 USENIX Annual Technical Conference, June 2012, pp.223-235.
[28] Färber F, May N, Lehner W et al. The SAP HANA database-An architecture overview. IEEE Data Eng. Bull., 2012, 35(1):28-33.
[29] Lee J, Kwon Y S, Färber F et al. SAP HANA distributed inmemory database system:Transaction, session, and metadata management. In Proc. the 29th International Conference on Data Engineering, April 2013, pp.1165-1173.
[30] Thomson A, Diamond T, Weng S C, Ren K, Shao P, Abadi D J. Calvin:Fast distributed transactions for partitioned database systems. In Proc. the 2012 ACM SIGMOD International Conference on Management of Data, May 2012, pp.1-12.
[1] Yue-Wen Wu, Yuan-Jia Xu, Heng Wu, Lin-Gang Su, Wen-Bo Zhang, Hua Zhong. 大数据环境下基于数据驱动的云配置优化方法[J]. 计算机科学技术学报, 2021, 36(5): 1184-1199.
[2] Bi-Ying Yan, Chao Yang, Pan Deng, Qiao Sun, Feng Chen, Yang Yu. 一种基于时空因果性的城市感知数据治理方法[J]. 计算机科学技术学报, 2020, 35(5): 1084-1098.
[3] Hong-Mei Wei, Jian Gao, Peng Qing, Kang Yu, Yan-Fei Fang, Ming-Lu Li. MPI-RCDD：一种MPI运行时的通信死锁检测框架[J]. 计算机科学技术学报, 2020, 35(2): 395-411.
[4] André Brinkmann, Kathryn Mohror, Weikuan Yu, Philip Carns, Toni Cortes, Scott A. Klasky, Alberto Miranda, Franz-Josef Pfreundt, Robert B. Ross, Marc-André Vef. 高性能计算专用文件系统[J]. 计算机科学技术学报, 2020, 35(1): 4-26.
[5] Yu-Tong Lu, Peng Cheng, Zhi-Guang Chen. Tianhe-2数据存储与管理系统设计与实现[J]. 计算机科学技术学报, 2020, 35(1): 27-46.
[6] Qi Chen, Kang Chen, Zuo-Ning Chen, Wei Xue, Xu Ji, Bin Yang. 神威存储系统面向应用I/O性能提升的优化介绍[J]. 计算机科学技术学报, 2020, 35(1): 47-60.
[7] Marc-André Vef, Nafiseh Moti, Tim Süß, Markus Tacke, Tommaso Tocci, Ramon Nou, Alberto Miranda, Toni Cortes, André Brinkmann. GekkoFS—一种用于高性能计算应用的临时突发缓冲文件系统[J]. 计算机科学技术学报, 2020, 35(1): 72-91.
[8] Robert B. Ross, George Amvrosiadis, Philip Carns, Charles D. Cranor, Matthieu Dorier, Kevin Harms, Greg Ganger, Garth Gibson, Samuel K. Gutierrez, Robert Latham, Bob Robey, Dana Robinson, Bradley Settlemyer, Galen Shipman, Shane Snyder, Jerome Soumagne, Qing Zheng. Mochi：为高性能计算环境组合数据服务[J]. 计算机科学技术学报, 2020, 35(1): 121-144.
[9] Rui Ren, Jiechao Cheng, Xi-Wen He, Lei Wang, Jian-Feng Zhan, Wan-Ling Gao, Chun-Jie Luo. HybridTune:基于时空数据关联的大数据系统性能诊断[J]. 计算机科学技术学报, 2019, 34(6): 1167-1184.
[10] Xu Tan, Xiao-Wei Shen, Xiao-Chun Ye, Da Wang, Dong-Rui Fan, Lunkai Zhang, Wen-Mi. 一种面向数据流架构的无停顿双缓冲机制[J]. , 2018, 33(1): 145-157.
[11] Ze-Qi Lin, Bing Xie, Yan-Zhen Zou, Jun-Feng Zhao, Xuan-Dong Li, Jun Wei, Hai-Lo. 智能开发环境与软件知识图谱[J]. , 2017, 32(2): 242-249.
[12] Xiao-Wei Shen, Xiao-Chun Ye, Xu Tan, Da Wang, Lunkai Zhang, Wen-Ming Li, Zhi-Min Zhang, Dong-Rui Fan, Ning-Hui Sun. 一种面向数据流架构的高效片上路由结构[J]. , 2017, 32(1): 11-25.
[13] Zhen-Hua Li, Gang Liu, Zhi-Yuan Ji, Roger Zimmermann. 基于腾讯大数据的高成本效益的云下载系统设计[J]. , 2015, 30(6): 1163-1174.
[14] Hui Li, Di Wu, Gao-Xiang Li, Yi-Hao Ke, Wen-Jie Liu, Yuan-Huan Zheng, Xiao-La Lin. 基于大数据的电信用户离网分析和服务质量提升:基础架构、理论模型和实际部署[J]. , 2015, 30(6): 1201-1214.
[15] Tao Liu, Yi Liu, Qin Li, Xiang-Rong Wang, Fei Gao, Yan-Chao Zhu, De-Pei Qian. SEIP:基于分布式平台的高效图像处理系统[J]. , 2015, 30(6): 1215-1232.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] 周笛;. A Recovery Technique for Distributed Communicating Process Systems[J]. , 1986, 1(2): 34 -43 .
[2] 陈世华;. On the Structure of Finite Automata of Which M Is an(Weak)Inverse with Delay τ[J]. , 1986, 1(2): 54 -59 .
[3] C.Y.Chung; 华宣仁;. A Chinese Information Processing System[J]. , 1986, 1(2): 15 -24 .
[4] 孙钟秀; 商陆军;. DMODULA:A Distributed Programming Language[J]. , 1986, 1(2): 25 -31 .
[5] 王建潮; 魏道政;. An Effective Test Generation Algorithm for Combinational Circuits[J]. , 1986, 1(4): 1 -16 .
[6] 陈肇雄; 高庆狮;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[7] 黄河燕;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[8] 郑国梁; 李辉;. The Design and Implementation of the Syntax-Directed Editor Generator(SEG)[J]. , 1986, 1(4): 39 -48 .
[9] 沈理; Stephen Y.H.Su;. Generalized Parallel Signature Analyzers with External Exclusive-OR Gates[J]. , 1986, 1(4): 49 -61 .
[10] 闵应骅; 韩智德;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持：support@magtech.com.cn
总访问量：