Loading [MathJax]/jax/output/SVG/jax.js
We use cookies to improve your experience with our site.

Indexed in:

SCIE, EI, Scopus, INSPEC, DBLP, CSCD, etc.

Submission System
(Author / Reviewer / Editor)
Advanced Search
Volume 35 Issue 2
March  2020
Turn off MathJax
Article Contents
Abstract
References
Related Articles
Supplements
Shu-Quan Wang, Lei Wang, Yu Deng, Zhi-Jie Yang, Sha-Sha Guo, Zi-Yang Kang, Yu-Feng Guo, Wei-Xia Xu. SIES: A Novel Implementation of Spiking Convolutional Neural Network Inference Engine on Field-Programmable Gate Array[J]. Journal of Computer Science and Technology, 2020, 35(2): 475-489. DOI: 10.1007/s11390-020-9686-z
Citation: Shu-Quan Wang, Lei Wang, Yu Deng, Zhi-Jie Yang, Sha-Sha Guo, Zi-Yang Kang, Yu-Feng Guo, Wei-Xia Xu. SIES: A Novel Implementation of Spiking Convolutional Neural Network Inference Engine on Field-Programmable Gate Array[J]. Journal of Computer Science and Technology, 2020, 35(2): 475-489. DOI: 10.1007/s11390-020-9686-z
shu

SIES: A Novel Implementation of Spiking Convolutional Neural Network Inference Engine on Field-Programmable Gate Array

  • College of Computer Science and Technology, National University of Defense Technology, Changsha 430041, China

Funds: The work was supported by the HeGaoJi Program of China under Grant Nos. 2017ZX01028103-002 and 2017ZX01038104-002, and the National Natural Science Foundation of China under Grant No. 61472432.
More Information
  • Author Bio:

    Shu-Quan Wang was a Master student in College of Computer Science and Technology, National University of Defense Technology, Changsha. He received his B.S. degree in computer science and technology in 2011. His research interests include computer architecture, asynchronous circuit, artificial intelligence and neuromorphic computation.

  • Corresponding author:

    Lei Wang E-mail: arrowya@gmail.com

  • Received Date: April 30, 2019
  • Revised Date: February 12, 2020
  • Published Date: March 04, 2020
    Abstract
  • Neuromorphic computing is considered to be the future of machine learning, and it provides a new way of cognitive computing. Inspired by the excellent performance of spiking neural networks (SNNs) on the fields of low-power consumption and parallel computing, many groups tried to simulate the SNN with the hardware platform. However, the efficiency of training SNNs with neuromorphic algorithms is not ideal enough. Facing this, Michael et al. proposed a method which can solve the problem with the help of DNN (deep neural network). With this method, we can easily convert a well-trained DNN into an SCNN (spiking convolutional neural network). So far, there is a little of work focusing on the hardware accelerating of SCNN. The motivation of this paper is to design an SNN processor to accelerate SNN inference for SNNs obtained by this DNN-to-SNN method. We propose SIES (Spiking Neural Network Inference Engine for SCNN Accelerating). It uses a systolic array to accomplish the task of membrane potential increments computation. It integrates an optional hardware module of max-pooling to reduce additional data moving between the host and the SIES. We also design a hardware data setup mechanism for the convolutional layer on the SIES with which we can minimize the time of input spikes preparing. We implement the SIES on FPGA XCVU440. The number of neurons it supports is up to 4 000 while the synapses are 256 000. The SIES can run with the working frequency of 200 MHz, and its peak performance is 1.562 5 TOPS.
    • FullText(HTML)
    • References (42)
  • [1]
    Akopyan F, Sawada J, Cassidy A, Alvarez-Icaza R, Arthur J, Merolla P, Imam N, Nakamura Y, Datta P, Nam G J. TrueNorth:Design and tool flow of a 65mW 1 million neuron programmable neurosynaptic chip. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2015, 34(10):1537-1557.
    [2]
    Geddes J, Lloyd S, Simpson A C et al. NeuroGrid:Using grid technology to advance neuroscience. In Proc. the 18th IEEE Symposium on Computer-Based Medical Systems, June 2005, pp.570-572.
    [3]
    Schemmel J, Grübl A, Hartmann S et al. Live demonstration:A scaled-down version of the BrainScaleS wafer-scale neuromorphic system. In Proc. the 2012 IEEE International Symposium on Circuits Systems, May 2012, p.702.
    [4]
    Furber S B, Lester D R, Plana L A, Garside J D, Painkras E, Temple S, Brown A D. Overview of the spiNNaker system architecture. IEEE Transactions on Computers, 2013, 62(12):2454-2467.
    [5]
    Davies M, Jain S, Liao Y et al. Loihi:A neuromorphic manycore processor with on-chip learning. IEEE Micro, 2018, 38(1):82-99.
    [6]
    Diehl P U, Neil D, Binas J, Cook M, Liu S C, Pfeiffer M. Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In Proc. the 2015 International Joint Conference on Neural Networks, July 2015.
    [7]
    Rueckauer B, Lungu I A, Hu Y, Pfeiffer M, Liu S C. Conversion of continuous-valued deep networks to efficient eventdriven networks for image classification. Frontiers in Neuroscience, 2017, 11:Article No. 682.
    [8]
    Rueckauer B, Lungu L A, Hu Y H, Pfeiffer M. Theory and tools for the conversion of analog to spiking convolutional neural networks. arXiv:1612.04052, 2016. https://arxiv.org/pdf/1612.04052.pdf,Nov.2019.
    [9]
    Du Z D, Fasthuber R, Chen T S, Ienne P, Li L, Luo T, Feng X B, Chen Y J, Temam O. ShiDianNao:Shifting vision processing closer to the sensor. In Proc. the 42nd ACM/IEEE International Symposium on Computer Architecture, June 2015, pp.92-104.
    [10]
    Guan Y J, Yuan Z H, Sun G Y, Cong J. FPGA-based accelerator for long short-term memory recurrent neural networks. In Proc. the 22nd Asia and South Pacific Design Automation Conference, January 2017, pp.629-634.
    [11]
    Zhou Y M, Jiang J F. An FPGA-based accelerator implementation for deep convolutional neural networks. In Proc. the 4th International Conference on Computer Science Network Technology, December 2016, pp.829-832.
    [12]
    Neil D, Liu S C. Minitaur, an event-driven FPGA-based spiking network accelerator. IEEE Transactions on Very Large Scale Integration Systems, 2014, 22(12):2621-2628.
    [13]
    Wang R, Thakur C S, Cohen G, Hamilton T J, Tapson J, van Schaik A. Neuromorphic hardware architecture using the neural engineering framework for pattern recognition. IEEE Trans. Biomed Circuits Syst., 2017, 11(3):574-584.
    [14]
    Glackin B, Mcginnity T M, Maguire L P, Wu Q X, Belatreche A. A novel approach for the implementation of large scale spiking neural networks on FPGA hardware. In Lecture Notes in Computer Science 3512, Cabestany J, Prieto A, Sandoral (eds.), Springer, 2005, pp.552-563.
    [15]
    Cheung K, Schultz S R, Luk W. A large-scale spiking neural network accelerator for FPGA systems. In Proc. the 22nd International Conference on Artificial Neural Networks, September 2012, pp.113-130.
    [16]
    Benton A L. Foundations of physiological psychology. Neurology, 1968, 18(6):609-612.
    [17]
    Hodgkin A L, Huxley A F, Katz B. Measurement of currentvoltage relations in the membrane of the giant axon of Loligo. J. Physiol., 1952, 116(4):424-448.
    [18]
    Izhikevich E M. Simple model of spiking neurons. IEEE Transactions on Neural Networks, 2003, 14(6):1569-1572.
    [19]
    Brunel N, van Rossum M C W. Lapicque's 1907 paper:From frogs to integrate-and-fire. Biological Cybernetics, 2007, 97(5/6):337-339.
    [20]
    Liu Y H, Wang X J. Spike-frequency adaptation of a generalized leaky integrate-and-fire model neuron. Journal of Computational Neuroscience, 2001, 10(1):25-45.
    [21]
    Brette R, Gerstner W. Adaptive exponential integrate-andfire model as an effective description of neuronal activity. Journal of Neurophysiology, 2005, 94(5):3637-3642.
    [22]
    Paninski L, Pillow J W, Simoncelli E P. Maximum likelihood estimation of a stochastic integrate-and-fire neural encoding model. Neural Computation, 2014, 16(12):2533-2561.
    [23]
    Tsumoto K, Kitajima H, Yoshinaga T, Aihara K, Kawakami H. Bifurcations in Morris-Lecar neuron model. Neurocomputing, 2006, 69(4-6):293-316.
    [24]
    Linares-Barranco B, Sanchez-Sinencio E, RodriguezVazquez A, Huertas J L. A CMOS implementation of the Fitzhugh-Nagumo neuron model. IEEE Journal of SolidState Circuits, 1991, 26(7):956-965.
    [25]
    Yadav R N, Kalra P K, John J. Time series prediction with single multiplicative neuron model. Applied Soft Computing, 2007, 7(4):1157-1163.
    [26]
    Maguire L P, Mcginnity T M, Glackin B, Ghani A, Belatreche A, Harkin J. Challenges for large-scale implementations of spiking neural networks on FPGAs. Neurocomputing, 2007, 71(1):13-29.
    [27]
    Gerstner W, Kistler W. Spiking Neuron Models:Single Neurons, Populations, Plasticity (1st edition). Cambridge University Press, 2002.
    [28]
    Gerstner W. Spiking neuron models. In Encyclopedia of Neuroscience, Squire L R (ed.), Academic Press, 2009, pp.277-280.
    [29]
    Lopresti D P. P-NAC:A systolic array for comparing nucleic acid sequences. Computer, 1987, 20(7):98-99.
    [30]
    Samajdar A, Zhu Y, Whatmough P, Mattina M, Krishna T. SCALE-Sim:Systolic CNN accelerator simulator. Distributed, Parallel, and Cluster Computing, 2018.
    [31]
    Jouppi N P, Young C, Patil N et al. In-datacenter performance analysis of a tensor processing unit. In Proc. International Symposium on Computer Architecture, May 2017.
    [32]
    Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In Proc. the 3rd International Conference on Learning Representations, May 2015, Article No. 4.
    [33]
    Shen J C, Ma D, Gu Z H, Zhang M, Zhu X L, Xu X Q, Xu Q, Shen Y J, Pan G. Darwin:A neuromorphic hardware co-processor based on spiking neural networks. SCIENCE CHINA Information Sciences, 2016, 59(2):Article No. 023401.
    [34]
    Kousanakis E, Dollas A, Sotiriades E et al. An architecture for the acceleration of a hybrid leaky integrate and fire SNN on the convey HC-2ex FPGA-based processor. In Proc. the 25th IEEE International Symposium on Fieldprogrammable Custom Computing Machines, April 2017, pp.56-63.
    [35]
    Fang H, Shrestha A, Ma D et al. Scalable NoCbased neuromorphic hardware learning and inference. arXiv:1810.09233, 2018. https://arxiv.org/pdf/1810.0923-3v1.pdf,Dec.2019.
    [36]
    Cheung K, Schultz S R, Luk W. NeuroFlow:A general purpose spiking neural network simulation platform using customizable processors. Frontiers in Neuroscience, 2015, 9:Article No. 516.
    [37]
    Albericio J, Judd P, Hetherington T et al. Cnvlutin:Ineffectual-neuron-free deep neural network computing. ACM SIGARCH Computer Architecture News, 2016, 44(3):1-13.
    [38]
    Guo S, Wang L, Chen B, Dou Q. An overhead-free maxpooling method for SNN. IEEE Embedded Systems Letters. doi: 10.1109/LES.2019.2919244.
    [39]
    Sengupta A, Ye Y T, Wang R, Liu C, Roy K. Going deeper in spiking neural networks:VGG and residual architectures. arXiv:1802.02627, 2018. https://arxiv.org/pdf/1802.02627.pdf,Dec.2019.
    [40]
    LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 1998, 86(11):2278-2324.
    [41]
    Netzer Y, Wang T, Coates A, Bissacco A, Wu B, Ng A Y. Reading digits in natural images with unsupervised feature learning. In Proc. the NIPS Workshop on Deep Learning and Unsupervised Feature Learning, December 2011.
    [42]
    Krizhevsky A. Learning multiple layers of features from tiny images. Technical Report, University of Toronto, 2009. http://www.cs.utoronto.ca/~kriz/learning-features-2009-TR.pdf,Dec.2019.
    • Relative Articles

  • Related Articles

    [1]Li-Xian Ma, Le-Ping Wang, En Shao, Rong-Yu Cao, Guang-Ming Tan. VastPipe: A High-Throughput Inference System via Adaptive Space-Division Multiplexing for Diverse Accelerators[J]. Journal of Computer Science and Technology, 2025, 40(2): 444-463. DOI: 10.1007/s11390-024-3773-5
    [2]Xiao-Fei Liao, Wen-Ju Zhao, Hai Jin, Peng-Cheng Yao, Yu Huang, Qing-Gang Wang, Jin Zhao, Long Zheng, Yu Zhang, Zhi-Yuan Shao. Towards High-Performance Graph Processing: From a Hardware/Software Co-Design Perspective[J]. Journal of Computer Science and Technology, 2024, 39(2): 245-266. DOI: 10.1007/s11390-024-4150-0
    [3]Zhe Fan, Yi-Fan Hao, Tian Zhi, Qi Guo, Zi-Dong Du. Hardware Acceleration for SLAM in Mobile Systems[J]. Journal of Computer Science and Technology, 2023, 38(6): 1300-1322. DOI: 10.1007/s11390-021-1523-5
    [4]Shao-Feng Zhao, Fang Wang, Bo Liu, Dan Feng, Yang Liu. LayCO: Achieving Least Lossy Accuracy for Most Efficient RRAM-Based Deep Neural Network Accelerator via Layer-Centric Co-Optimization[J]. Journal of Computer Science and Technology, 2023, 38(2): 328-347. DOI: 10.1007/s11390-023-2545-y
    [5]Tong Li, Hong-Lan Jiang, Hai Mo, Jie Han, Lei-Bo Liu, Zhi-Gang Mao. Approximate Processing Element Design and Analysis for the Implementation of CNN Accelerators[J]. Journal of Computer Science and Technology, 2023, 38(2): 309-327. DOI: 10.1007/s11390-023-2548-8
    [6]Jia-Jun Li, Ke Wang, Hao Zheng, Ahmed Louri. GShuttle: Optimizing Memory Access Efficiency for Graph Convolutional Neural Network Accelerators[J]. Journal of Computer Science and Technology, 2023, 38(1): 115-127. DOI: 10.1007/s11390-023-2875-9
    [7]Ying Wang, Lei Zhang, Yin-He Han, Hua-Wei Li. Reinventing Memory System Design for Many-Accelerator Architecture[J]. Journal of Computer Science and Technology, 2014, 29(2): 273-280. DOI: 10.1007/s11390-014-1429-6
    [8]Tong Xin, Tang Zesheng. Hardware Assisted Fast Volume Rendering with Boundary Enhancement[J]. Journal of Computer Science and Technology, 1998, 13(5): 393-401.
    [9]Xing Hancheng, Li Chunlin, Xing Dongsheng. Improvements to the Control Techniques of Sequential Inference Machines——from Instructions to Hardware Organization[J]. Journal of Computer Science and Technology, 1991, 6(1): 66-73.
    [10]Liu Mingye, Guo Shuming, Yang Huai, Jia Liangyu, Hong Enyu. A Logic Design Automation System for Generating Logic Diagram from Hardware Description[J]. Journal of Computer Science and Technology, 1989, 4(3): 245-254.
    • Supplements (2)

  • Others

  • Cited by

    Periodical cited type(28)

    1. HOU Yue, XIANG Shuiying, ZOU Tao, et al. Implementation of a High-Efficiency, Lightweight Residual Spiking Neural Network Processor Based on FPGA. Acta Physica Sinica, 2025, 74(14): 0. DOI:10.7498/aps.74.20250390
    2. Mehrzad Karamimanesh, Ebrahim Abiri, Mahyar Shahsavari, et al. Spiking neural networks on FPGA: A survey of methodologies and recent advancements. Neural Networks, 2025, 186: 107256. DOI:10.1016/j.neunet.2025.107256
    3. Yuehai Chen, Wujian Ye, Yijun Liu, et al. SiBrain: A Sparse Spatio-Temporal Parallel Neuromorphic Architecture for Accelerating Spiking Convolution Neural Networks With Low Latency. IEEE Transactions on Circuits and Systems I: Regular Papers, 2024, 71(12): 6482. DOI:10.1109/TCSI.2024.3393233
    4. Zilin Wang, Zehong Ou, Yi Zhong, et al. NeuroREC: A 28-nm Efficient Neuromorphic Processor for Radar Emitter Classification. IEEE Transactions on Circuits and Systems I: Regular Papers, 2024, 71(12): 6215. DOI:10.1109/TCSI.2024.3427385
    5. Jindong Li, Guobin Shen, Dongcheng Zhao, et al. FireFly v2: Advancing Hardware Support for High-Performance Spiking Neural Network With a Spatiotemporal FPGA Accelerator. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2024, 43(9): 2647. DOI:10.1109/TCAD.2024.3380550
    6. Zilin Wang, Yi Zhong, Zehong Ou, et al. Marmotini: A Weight Density Adaptation Architecture With Hybrid Compression Method for Spiking Neural Network. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2024, 32(12): 2293. DOI:10.1109/TVLSI.2024.3453897
    7. Lei Kang, Xu Yang, Chi Zhang, et al. A 24.3 μJ/Image SNN Accelerator for DVS-Gesture With WS-LOS Dataflow and Sparse Methods. IEEE Transactions on Circuits and Systems II: Express Briefs, 2023, 70(11): 4226. DOI:10.1109/TCSII.2023.3282589
    8. Yuehai Chen, Yijun Liu, Wujian Ye, et al. The High-Performance Design of a General Spiking Convolution Computation Unit for Supporting Neuromorphic Hardware Acceleration. IEEE Transactions on Circuits and Systems II: Express Briefs, 2023, 70(9): 3634. DOI:10.1109/TCSII.2023.3266516
    9. Wujian Ye, Yuehai Chen, Yijun Liu. The Implementation and Optimization of Neuromorphic Hardware for Supporting Spiking Neural Networks With MLP and CNN Topologies. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2023, 42(2): 448. DOI:10.1109/TCAD.2022.3179246
    10. Jindong Li, Guobin Shen, Dongcheng Zhao, et al. FireFly: A High-Throughput Hardware Accelerator for Spiking Neural Networks With Efficient DSP and Memory Optimization. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2023, 31(8): 1178. DOI:10.1109/TVLSI.2023.3279349
    11. Vijay Kakani, Xingyou Li, Xuenan Cui, et al. Implementation of Field-Programmable Gate Array Platform for Object Classification Tasks Using Spike-Based Backpropagated Deep Convolutional Spiking Neural Networks. Micromachines, 2023, 14(7): 1353. DOI:10.3390/mi14071353
    12. Qinyu Chen, Chang Gao, Yuxiang Fu. Cerebron: A Reconfigurable Architecture for Spatiotemporal Sparse Spiking Neural Networks. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 2022, 30(10): 1425. DOI:10.1109/TVLSI.2022.3196839
    13. Jeong-Jun Lee, Wenrui Zhang, Yuan Xie, et al. SaARSP: An Architecture for Systolic-Array Acceleration of Recurrent Spiking Neural Networks. ACM Journal on Emerging Technologies in Computing Systems, 2022, 18(4): 1. DOI:10.1145/3510854
    14. Jan Sommer, M. Akif Ozkan, Oliver Keszocze, et al. Efficient Hardware Acceleration of Sparsely Active Convolutional Spiking Neural Networks. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2022, 41(11): 3767. DOI:10.1109/TCAD.2022.3197512
    15. Ling Zhang, Jing Yang, Cong Shi, et al. A Cost-Efficient High-Speed VLSI Architecture for Spiking Convolutional Neural Network Inference Using Time-Step Binary Spike Maps. Sensors, 2021, 21(18): 6006. DOI:10.3390/s21186006
    16. Jingjing Tao, Mingsheng Zhang. Application of Intelligent Systems in Multi-modal Information Analytics. Lecture Notes on Data Engineering and Communications Technologies, DOI:10.1007/978-3-031-05237-8_114
    17. Zilin Wang, Yi Zhong, Guang Chen, et al. A Hybrid Heterogeneous Neural Network Accelerator based on Systolic Array. 2024 IEEE 6th International Conference on AI Circuits and Systems (AICAS), DOI:10.1109/AICAS59952.2024.10595910
    18. Hanwen Liu, Yi Chen, Zihang Zeng, et al. A Low Power and Low Latency FPGA-Based Spiking Neural Network Accelerator. 2023 International Joint Conference on Neural Networks (IJCNN), DOI:10.1109/IJCNN54540.2023.10191153
    19. Shaogang Hu, Tianchen Li, Yue Zuo, et al. A Binarized Systolic Array-Based Neuromorphic Architecture with High Efficiency. 2022 6th International Conference on Information Technology, Information Systems and Electrical Engineering (ICITISEE), DOI:10.1109/ICITISEE57756.2022.10057911
    20. Rui Gong, Lei Wang, Wei Shi, et al. A Hardware Security Isolation Architecture for Intelligent Accelerator. 2022 IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), DOI:10.1109/TrustCom56396.2022.00135
    21. Minhui Hu, Jianhua Fan, Yongyang Hu, et al. Modeling and optimizing PE utilization rate for systolic array based CNN accelerators. Eighth International Conference on Electronic Technology and Information Science (ICETIS 2023), DOI:10.1117/12.2682498
    22. Jeong-Jun Lee, Wenrui Zhang, Peng Li. Parallel Time Batching: Systolic-Array Acceleration of Sparse Spiking Neural Computation. 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA), DOI:10.1109/HPCA53966.2022.00031
    23. Ayesha Siddique, Khaza Anuarul Hoque. Improving Reliability of Spiking Neural Networks through Fault Aware Threshold Voltage Optimization. 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE), DOI:10.23919/DATE56975.2023.10136999
    24. Patrick Plagwitz, Frank Hannig, Jürgen Teich, et al. Applied Reconfigurable Computing. Architectures, Tools, and Applications. Lecture Notes in Computer Science, DOI:10.1007/978-3-031-55673-9_1
    25. Rasha Karakchi. A Scratchpad Spiking Neural Network Accelerator. 2024 IEEE 3rd International Conference on Computing and Machine Intelligence (ICMI), DOI:10.1109/ICMI60790.2024.10586065
    26. Jeong-Jun Lee, Peng Li. Reconfigurable Dataflow Optimization for Spatiotemporal Spiking Neural Computation on Systolic Array Accelerators. 2020 IEEE 38th International Conference on Computer Design (ICCD), DOI:10.1109/ICCD50377.2020.00027
    27. Jeong-Jun Lee, Jianhao Chen, Wenrui Zhang, et al. Systolic-Array Spiking Neural Accelerators with Dynamic Heterogeneous Voltage Regulation. 2021 International Joint Conference on Neural Networks (IJCNN), DOI:10.1109/IJCNN52387.2021.9534037
    28. Yijie Miao, Makoto Ikeda. Artificial Intelligence Applications and Innovations. IFIP Advances in Information and Communication Technology, DOI:10.1007/978-3-031-34107-6_42

    Other cited types(0)

Catalog

    Get Citation
    PDF
    XML
    Read Online
    Article views (120) PDF downloads (437) Cited by(28)
    Related

    Copyright ©2025 JCST, All Rights Reserved     京ICP备05002829号-1

    Institute of Computing Technology, Chinese Academy of Sciences
    P.O. Box 2704, Beijing 100190, P.R. China

    Tel.: 86-10-62610746 E-mail: jcst@ict.ac.cn

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return