计算机科学技术学报 ›› 2019,Vol. 34 ›› Issue (1): 77-93.doi: 10.1007/s11390-019-1900-5

所属专题： Computer Architecture and Systems

• • 上一篇    下一篇

申威26010处理器上k-means算法的高性能并行

Min Li1,2, Student Member, CCF, Chao Yang3,4,5,*, Senior Member, CCF, Member, ACM, IEEE, Qiao Sun1 Wen-Jing Ma1, Wen-Long Cao1,2, Student Member, CCF, and Yu-Long Ao3,4,5, Member, CCF   

  1. 1 Institute of Software, Chinese Academy of Sciences, Beijing 100190, China;
    2 University of Chinese Academy of Sciences, Beijing 100049, China;
    3 School of Mathematical Sciences, Peking University, Beijing 100871, China;
    4 Center for Data Science, Peking University, Beijing 100871, China;
    5 Peng Cheng Laboratory, Shenzhen 518052, China
  • 收稿日期:2018-07-08 修回日期:2018-12-06 出版日期:2019-01-05 发布日期:2019-01-12
  • 通讯作者: Chao Yang E-mail:chao_yang@pku.edu.cn
  • 作者简介:Min Li received her B.S. degree in computer science and technology from Shandong Normal University, Jinan, in 2015. She is currently pursuing her Ph.D. degree in the Institute of Software, Chinese Academy of Sciences, Beijing. Her research interest mainly includes high-performance computing, especially parallel machine learning algorithm.
  • 基金资助:
    This work was supported in part by the National Key Research and Development Plan of China under Grant No. 2016YFB0200603, the National Natural Science Foundation of China under Grant No. 91530323, and the Beijing Natural Science Foundation of China under Grant No. JQ18001.

Enabling Highly Efficient k-Means Computations on the SW26010 Many-Core Processor of Sunway TaihuLight

Min Li1,2, Student Member, CCF, Chao Yang3,4,5,*, Senior Member, CCF, Member, ACM, IEEE, Qiao Sun1 Wen-Jing Ma1, Wen-Long Cao1,2, Student Member, CCF, and Yu-Long Ao3,4,5, Member, CCF   

  1. 1 Institute of Software, Chinese Academy of Sciences, Beijing 100190, China;
    2 University of Chinese Academy of Sciences, Beijing 100049, China;
    3 School of Mathematical Sciences, Peking University, Beijing 100871, China;
    4 Center for Data Science, Peking University, Beijing 100871, China;
    5 Peng Cheng Laboratory, Shenzhen 518052, China
  • Received:2018-07-08 Revised:2018-12-06 Online:2019-01-05 Published:2019-01-12
  • Contact: Chao Yang E-mail:chao_yang@pku.edu.cn
  • About author:Min Li received her B.S. degree in computer science and technology from Shandong Normal University, Jinan, in 2015. She is currently pursuing her Ph.D. degree in the Institute of Software, Chinese Academy of Sciences, Beijing. Her research interest mainly includes high-performance computing, especially parallel machine learning algorithm.
  • Supported by:
    This work was supported in part by the National Key Research and Development Plan of China under Grant No. 2016YFB0200603, the National Natural Science Foundation of China under Grant No. 91530323, and the Beijing Natural Science Foundation of China under Grant No. JQ18001.

PDF (PC)

671

补充材料

1

k-means是无监督学习中一种经典的基于距离计算的聚类算法，由于简单，易实现，无需标记样本等特点，k-means在图像处理、数据挖掘、文本聚类、生物学等领域有着广泛的应用，并越来越多的被用作许多更复杂算法的预处理手段。大数据时代的到来，使得样本数据的特征维度从原来的几十维上升到了数以千计，相应的，对计算速度也提出了更高的要求，因此，研究k-means的并行加速对于实际应用是非常重要且有意义的。我们提出了一种将距离计算与簇标签规约融合的k-means并行算法；同时将其部署在申威26010众核处理器上。该算法减少了对距离矩阵的存储及内存访问次数，使用三层分块策略进行任务划分，以实现数据到硬件资源的高效映射，并利用协作式的核间数据共享方式以提高数据的重用率，使用寄存器通信机制实现簇标签规约。通过使用指令重排、双缓冲等优化技术，我们进一步大幅提升了k-means的性能。此外，我们对分块大小的选择进行了讨论，并建立简单的性能模型进行性能分析。实验显示，我们的方法可以达到348.1 GFlops的浮点计算性能，相比机器最大浮点性能/理论上最大性能，可以获得46.9%/84%的浮点计算效率，相比较当前在CPU和GPU上最好的实现算法，具有很大的优越性。

关键词: 并行k-means, 性能优化, 申威26010处理器, 神威太湖之光

Abstract: With the advent of the big data era, the amounts of sampling data and the dimensions of data features are rapidly growing. It is highly desired to enable fast and efficient clustering of unlabeled samples based on feature similarities. As a fundamental primitive for data clustering, the k-means operation is receiving increasingly more attentions today. To achieve high performance k-means computations on modern multi-core/many-core systems, we propose a matrix-based fused framework that can achieve high performance by conducting computations on a distance matrix and at the same time can improve the memory reuse through the fusion of the distance-matrix computation and the nearest centroids reduction. We implement and optimize the parallel k-means algorithm on the SW26010 many-core processor, which is the major horsepower of Sunway TaihuLight. In particular, we design a task mapping strategy for load-balanced task distribution, a data sharing scheme to reduce the memory footprint and a register blocking strategy to increase the data locality. Optimization techniques such as instruction reordering and double buffering are further applied to improve the sustained performance. Discussions on block-size tuning and performance modeling are also presented. We show by experiments on both randomly generated and real-world datasets that our parallel implementation of k-means on SW26010 can sustain a double-precision performance of over 348.1 Gflops, which is 46.9% of the peak performance and 84% of the theoretical performance upper bound on a single core group, and can achieve a nearly ideal scalability to the whole SW26010 processor of four core groups. Performance comparisons with the previous state-of-the-art on both CPU and GPU are also provided to show the superiority of our optimized k-means kernel.

Key words: parallel k-means, performance optimization, SW26010 processor, Sunway TaihuLight

[1] Wu X, Kumar V, Quinlan J R, Ghosh J, Yang Q, Motoda H, Mclachlan G J, Ng A S, Liu B, Yu P S et al. Top 10 algorithms in data mining. Knowledge and Information Systems, 2007, 14(1):1-37.
[2] Muja M, Lowe D G. Scalable nearest neighbor algorithms for high dimensional data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(11):2227-2240.
[3] You Y, Demmel J, Czechowski K, Song L, Vuduc R. CASVM:Communication-avoiding support vector machines on distributed systems. In Proc. IEEE International Parallel and Distributed Processing Symposium, May 2015, pp.847-859.
[4] Wu J, Leng C, Wang Y, Hu Q, Cheng J. Quantized convolutional neural networks for mobile devices. In Proc. Computer Vision and Pattern Recognition, June 2016, pp.4820-4828.
[5] Narayanan R, Ozisikyilmaz B, Zambreno J, Memik G, Choudhary A. MineBench:A benchmark suite for data mining workloads. In Proc. IEEE International Symposium on Workload Characterization, October 2006, pp.182-188.
[6] Hadian A, Shahrivari S. High performance parallel k-means clustering for disk-resident datasets on multi-core CPUs. Journal of Supercomputing, 2014, 69(2):845-863.
[7] Wang H, Zhao J, Li H, Wang J. Parallel clustering algorithms for image processing on multi-core CPUs. In Proc. International Conference on Computer Science and Software Engineering, December 2008, pp.450-453.
[8] Farivar R, Rebolledo D, Chan E, Campbell R H. A parallel implementation of k-means clustering on GPUs. In Proc. International Conference on Parallel and Distributed Processing Techniques and Applications, July 2008, pp.340-345.
[9] Che S, Boyer M, Meng J, Tarjan D, Sheaffer J W, Skadron K. A performance study of general purpose applications on graphics processors using CUDA. Journal of Parallel & Distributed Computing, 2008, 68(10):1370-1380.
[10] Fang W, Lau K K, Lu M, Xiao X, Chi K L, Yang P Y, He B, Luo Q, Sander P V, Yang K. Parallel data mining on graphics processors. Technical Report, Hong Kong University of Science and Technology, 2008. http://www.comp.hkbu.edu.hk/~youli/Papers/Papers/20-09Fall/Mining/Parallel%20Data%20Mining%20on%20Graphics%20Processorsgpuminer.pdf, September 2018.
[11] Wu R, Zhang B, Hsu M. Clustering billions of data points using GPUs. In Proc. the Combined Workshops on UnConventional High Performance Computing Workshop Plus Memory Access Workshop, May 2009, pp.1-6.
[12] Zechner M, Granitzer M. Accelerating k-means on the graphics processor via CUDA. In Proc. the 1st International Conference on Intensive Applications and Services, April 2009, pp.7-15.
[13] Kohlhoff K J, Pande V S, Altman R B. K-means for parallel architectures using all-prefix-sum sorting and updating steps. IEEE Transactions on Parallel and Distributed Systems, 2013, 24(8):1602-1612.
[14] Li Y, Zhao K, Chu X, Liu J. Speeding up k-means algorithm by GPUs. Journal of Computer and System Sciences, 2013, 79(2):216-229.
[15] Karantasis K I, Polychronopoulos E D, Dimitrakopoulos G N. Accelerating data clustering on GPU-based clusters under shared memory abstraction. In Proc. IEEE International Conference on Cluster Computing Workshops and Posters, September 2010, Article No. 31.
[16] Karantasis K I, Polychronopoulos E D. Programming GPU clusters with shared memory abstraction in software. In Proc. the 19th Euromicro International Conference on Parallel, Distributed and Network-Based Processing, February 2011, pp.223-230.
[17] Wasif M K, Narayanan P J. Scalable clustering using multiple GPUs. In Proc. the 18th International Conference on High Performance Computing, December 2011, Article No. 14.
[18] Kijsipongse E, U-Ruekolan S. Dynamic load balancing on GPU clusters for large-scale k-means clustering. In Proc. the 9th International Joint Conference on Computer Science and Software Engineering, May 2012, pp.346-350.
[19] Stuart J A, Owens J D. Multi-GPU MapReduce on GPU clusters. In Proc. IEEE International Parallel & Distributed Processing Symposium, May 2011, pp.1068-1079.
[20] Wu F, Wu Q, Tan Y, Wei L, Shao L, Gao L. A vectorized k-means algorithm for Intel many integrated core architecture. In Proc. the 10th International Symposium on Advanced Parallel Processing Technologies, August 2013, pp.277-294.
[21] Fu H, Liao J, Yang J, Wang L, Song Z, Huang X, Yang C, Xue W, Liu F, Qiao F et al. The Sunway TaihuLight supercomputer:System and applications. Science China Information Sciences, 2016, 59(7):Article No. 072001.
[22] Yang C, Xue W, Fu H, You H, Wang X, Ao Y, Liu F, Gan L, Xu P, Wang L et al. 10M-core scalable fully-implicit solver for nonhydrostatic atmospheric dynamics. In Proc. International Conference for High Performance Computing, Networking, Storage and Analysis, November 2016, Article No. 6.
[23] Garey M R, Johnson D S, Witsenhausen H S. The complexity of the generalized Lloyd-Max problem (Corresp.). IEEE Transactions on Information Theory, 1982, 28(2):255-256.
[24] Hamerly G, Elkan C. Alternatives to the k-means algorithm that find better clusterings. In Proc. the 11th International Conference on Information and Knowledge Management, November 2002, pp.600-607.
[25] Bradley P S, Fayyad U M. Refining initial points for kmeans clustering. In Proc. the 15th International Conference on Machine Learning, July 1998, pp.91-99.
[26] Arthur D, Vassilvitskii S. K-means++:The advantages of careful seeding. In Proc. the 18th ACM-SIAM Symposium on Discrete Algorithms, January 2007, pp.1027-1035.
[27] Sarje A, Zola J, Aluru S. Accelerating pairwise computations on cell processors. IEEE Transactions on Parallel and Distributed Systems, 2011, 22(1):69-77.
[28] Sarje A, Aluru S. All-pairs computations on many-core graphics processors. Parallel Computing, 2013, 39(2):79-93.
[29] Steuwer M, Friese M, Albers S, Gorlatch S. Introducing and implementing the allpairs skeleton for programming multiGPU systems. International Journal of Parallel Programming, 2014, 42(4):601-618.
[30] Dongarra J J, Croz J D, Hammarling S, Duff I S. A set of level 3 basic linear algebra subprograms. ACM Transactions on Mathematical Software, 1990, 16(1):1-17.
[31] Dhillon I, Modha D. A data clustering algorithm on distributed memory multiprocessors. Lecture Notes in Computer Science, 2000, 1759:245-260.
[32] Zhang J, Zhou C, Wang Y, Ju L, Du Q, Chi X, Xu D, Chen D, Liu Y, Liu Z. Extreme-scale phase field simulations of coarsening dynamics on the Sunway TaihuLight supercomputer. In Proc. International Conference for High Performance Computing, Networking, Storage and Analysis, November 2016, Article No. 4.
[33] Lucas D D, Yver Kwok C, Cameronsmith P, Graven H, Bergmann D, Guilderson T P, Weiss R, Keeling R. Designing optimal greenhouse gas observing networks that consider performance and cost. Geoscientific Instrumentation Methods & Data Systems Discussions, 2014, 4(1):121-137.
[34] Sapsanis C, Georgoulas G, Tzes A, Lymberopoulos D. Improving EMG based classification of basic hand movements using EMD. In Proc. the 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, July 2013, pp.5754-5757.
[35] Altun K, Barshan B, Tuncel O. Comparative study on classifying human activities with miniature inertial and magnetic sensors. Pattern Recognition, 2010, 43(10):3605-3620.
[36] Barshan B, Yüksek M C. Recognizing daily and sports activities in two open source machine learning environments using body-worn sensor units. Computer Journal, 2014, 57(11):1649-1667.
[37] Altun K, Barshan B. Human activity recognition using inertial/magnetic sensor units. In Proc. the 1st International Workshop on Human Behavior Understanding, August 2010, pp.38-51.
[38] Newling J, Fleuret F. Fast k-means with accurate bounds. In Proc. the 33rd International Conference on International Conference on Machine Learning, June 2016, pp.936-944.
[39] Elkan C. Using the triangle inequality to accelerate kmeans. In Proc. the 20th International Conference on Machine Learning, August 2003, pp.147-153.
[40] Drake J, Hamerly G. Accelerated k-means with adaptive distance bounds. In Proc. the 5th NIPS Workshop on Optimization for Machine Learning, December 2012, pp.42-53.
[41] Ding Y, Zhao Y, Shen X, Musuvathi M, Mytkowicz T. Yinyang k-means:A drop-in replacement of the classic kmeans with consistent speedup. In Proc. the 32nd International Conference on Machine Learning, July 2015, pp.579-587.
[42] Bottesch T, Buhler T, Kachele M. Speeding up k-means by approximating Euclidean distances via block vectors. In Proc. the 33rd International Conference on Machine Learning, June 2016, pp.2578-2586.
[43] Wu J, Hong B. An efficient k-means algorithm on CUDA. In Proc. IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum, May 2011, pp.1740-1749.
[44] Lin Z, Lo C, Chow P. K-means implementation on FPGA for high-dimensional data using triangle inequality. In Proc. the 22nd International Conference on Field Programmable Logic and Applications, August 2012, pp.437-442.
[45] Winterstein F, Bayliss S, Constantinides G A. FPGAbased k-means clustering using tree-based data structures. In Proc. the 23rd International Conference on Field Programmable Logic and Applications, September 2013, Article No. 18.
[46] Tang Q Y, Khalid M A S. Acceleration of k-means algorithm using Altera SDK for OpenCL. ACM Transactions on Reconfigurable Technology and Systems, 2016, 10(1):Article No. 6.
[47] Jiang L, Yang C, Ao Y, Yin W, Ma W, Sun Q, Liu F, Lin R, Zhang P. Towards highly efficient DGEMM on the emerging SW26010 many-core processor. In Proc. the 46th International Conference on Parallel Processing, August 2017, pp.422-431.
[48] Fang J, Fu H, Zhao W, Chen B, Zheng W, Yang G. swDNN:A library for accelerating deep learning applications on Sunway TaihuLight. In Proc. IEEE International Parallel & Distributed Processing Symposium, May 2017, pp.615-624.
[49] Zhao W, Ma H, He Q. Parallel k-means clustering based on MapReduce. In Proc. the 1st IEEE International Conference on Cloud Computing, December 2009, pp.674-679.
[50] Cordeiro R L F, Traina A J M, Kang U, Faloutsos C et al. Clustering very large multi-dimensional datasets with MapReduce. In Proc. the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, August 2011, pp.690-698.
[51] Li Q, Wang P, Wang W, Hu H, Li Z, Li J. An efficient k-means clustering algorithm on MapReduce. In Proc. the 19th International Conference on Database Systems for Advanced Applications, April 2014, pp.357-371.
[1] Jason Liu, Pedro Espina, Xian-He Sun. 关于储存系统建模和优化的综述[J]. 计算机科学技术学报, 2021, 36(1): 71-89.
[2] Lan Huang, Da-Lin Li, Kang-Ping Wang, Teng Gao, Adriano Tavares. 一个关于高级综合工具性能优化的综述[J]. 计算机科学技术学报, 2020, 35(3): 697-720.
[3] Qi Chen, Kang Chen, Zuo-Ning Chen, Wei Xue, Xu Ji, Bin Yang. 神威存储系统面向应用I/O性能提升的优化介绍[J]. 计算机科学技术学报, 2020, 35(1): 47-60.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] 刘明业; 洪恩宇;. Some Covering Problems and Their Solutions in Automatic Logic Synthesis Systems[J]. , 1986, 1(2): 83 -92 .
[2] 陈世华;. On the Structure of (Weak) Inverses of an (Weakly) Invertible Finite Automaton[J]. , 1986, 1(3): 92 -100 .
[3] 高庆狮; 张祥; 杨树范; 陈树清;. Vector Computer 757[J]. , 1986, 1(3): 1 -14 .
[4] 陈肇雄; 高庆狮;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[5] 黄河燕;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[6] 闵应骅; 韩智德;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[7] 唐同诰; 招兆铿;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[8] 闵应骅;. Easy Test Generation PLAs[J]. , 1987, 2(1): 72 -80 .
[9] 朱鸿;. Some Mathematical Properties of the Functional Programming Language FP[J]. , 1987, 2(3): 202 -216 .
[10] 李明慧;. CAD System of Microprogrammed Digital Systems[J]. , 1987, 2(3): 226 -235 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持：support@magtech.com.cn
总访问量：