? BENCHIP: Benchmarking Intelligence Processors
Journal of Computer Science and Technology
Quick Search in JCST
 Advanced Search 
      Home | PrePrint | SiteMap | Contact Us | FAQ
Indexed by   SCIE, EI ...
Bimonthly    Since 1986
Journal of Computer Science and Technology 2018, Vol. 33 Issue (1) :1-23    DOI: 10.1007/s11390-018-1805-8
Computer Architecture and Systems Current Issue | Archive | Adv Search << | Next Articles >>
BENCHIP: Benchmarking Intelligence Processors
Jin-Hua Tao1,2,3, Zi-Dong Du1,3,4, Qi Guo1,3,4, Member, CCF, Hui-Ying Lan1,3, Lei Zhang1,3, Sheng-Yuan Zhou1,3, Ling-Jie Xu5, Cong Liu6, Hai-Feng Liu7, Shan Tang8, Allen Rush9, Willian Chen9, Shao-Li Liu1,3,4, Yun-Ji Chen1,2,3, Distinguished Member, CCF, Tian-Shi Chen1,3,4
1 State Key Laboratory of Computer Architecture, Institute of Computing Technology Chinese Academy of Sciences, Beijing 100190, China;
2 School of Computer and Control Engineering, University of Chinese Academy of Sciences, Beijing 100049, China;
3 Intelligent Processor Research Center, Institute of Computing Technology, Chinese Academy of Sciences Beijing 100190, China;
4 Cambricon Ltd., Beijing 100190, China;
5 Alibaba Infrastructure Service, Alibaba Group, Hangzhou 311121, China;
6 Iflytek Co., Ltd., Hefei 230088, China;
7 Beijing Jingdong Century Trading Co., Ltd., Beijing 100176, China;
8 RDA Microelectronics, Inc., Shanghai 201203, China;
9 Advanced Micro Devices, Inc., Sunnyvale, CA 94085, U.S.A

Related Articles
Download: [PDF 1433KB]     Export: BibTeX or EndNote (RIS)  
Abstract The increasing attention on deep learning has tremendously spurred the design of intelligence processing hardware. The variety of emerging intelligence processors requires standard benchmarks for fair comparison and system optimization (in both software and hardware). However, existing benchmarks are unsuitable for benchmarking intelligence processors due to their non-diversity and nonrepresentativeness. Also, the lack of a standard benchmarking methodology further exacerbates this problem. In this paper, we propose BenchIP, a benchmark suite and benchmarking methodology for intelligence processors. The benchmark suite in BenchIP consists of two sets of benchmarks:microbenchmarks and macrobenchmarks. The microbenchmarks consist of single-layer networks. They are mainly designed for bottleneck analysis and system optimization. The macrobenchmarks contain state-of-the-art industrial networks, so as to offer a realistic comparison of different platforms. We also propose a standard benchmarking methodology built upon an industrial software stack and evaluation metrics that comprehensively reflect various characteristics of the evaluated intelligence processors. BenchIP is utilized for evaluating various hardware platforms, including CPUs, GPUs, and accelerators. BenchIP will be open-sourced soon.
Articles by authors
Keywordsdeep learning   intelligence processor   benchmark     
Received 2017-09-10;

This work is partially supported by the National Key Research and Development Program of China under Grant No. 2017YFB1003101, the National Natural Science Foundation of China under Grant Nos. 61472396, 61432016, 61473275, 61522211, 61532016, 61521092, 61502446, 61672491, 61602441, 61602446, 61732002, and 61702478, Beijing Science and Technology Projects under Grant No. Z151100000915072, the Science and Technology Service Network Initiative (STS) Projects of Chinese Academy of Sciences, and the National Basic Research 973 Program of China under Grant No. 2015CB358800.

About author: Jin-Hua Tao received his B.S. degree in statistics from University of Science and Technology of China, Hefei, in 2013. He is currently a Ph.D. student at Institute of Computing Technology, Chinese Academy of Sciences, Beijing. His research interests include computer architecture and computational intelligence.
Cite this article:   
Jin-Hua Tao, Zi-Dong Du, Qi Guo, Hui-Ying Lan, Lei Zhang, Sheng-Yuan Zhou, Ling-Jie Xu, Cong Liu, Hai-Feng Liu, Shan Tang, Allen Rush, Willian Chen, Shao-Li Liu, Yun-Ji Chen, Tian-Shi Chen.BENCHIP: Benchmarking Intelligence Processors[J]  Journal of Computer Science and Technology, 2018,V33(1): 1-23
Copyright 2010 by Journal of Computer Science and Technology