We use cookies to improve your experience with our site.
Advanced Search
Volume 36 Issue 5
September  2021
Turn off MathJax
Article Contents
Abstract
References
Supplements
Feng Wang, Guo-Jie Luo, Guang-Yu Sun, Yu-Hao Wang, Di-Min Niu, Hong-Zhong Zheng. Area Efficient Pattern Representation of Binary Neural Networks on RRAM[J]. Journal of Computer Science and Technology, 2021, 36(5): 1155-1166. DOI: 10.1007/s11390-021-0906-y
Citation: Feng Wang, Guo-Jie Luo, Guang-Yu Sun, Yu-Hao Wang, Di-Min Niu, Hong-Zhong Zheng. Area Efficient Pattern Representation of Binary Neural Networks on RRAM[J]. Journal of Computer Science and Technology, 2021, 36(5): 1155-1166. DOI: 10.1007/s11390-021-0906-y
shu

Area Efficient Pattern Representation of Binary Neural Networks on RRAM

  • 1 Center for Energy-Efficient Computing and Applications, Peking University, Beijing 100871, China;
    2 Pingtouge, Alibaba Group, Hangzhou 310052, China

Funds: This work is partly supported by the National Key Research and Development Program of China under Grant No. 2020AAA0130400, Beijing Municipal Science and Technology Program of China under Grant No. Z201100004220007, the National Natural Science Foundation of China under Grant No. 62090021, Beijing Academy of Artificial Intelligence (BAAI), and Alibaba Innovative Research (AIR) Program.
More Information
  • Author Bio:

    Feng Wang received his Ph.D. degree in computer science from Peking University, Beijing, in 2021. He is currently a research scientist in LEDA Technology. His research interests include processing-in-memory and EDA algorithms.

  • Corresponding author:

    Guo-Jie Luo E-mail: gluo@pku.edu.cn

  • Received Date: August 13, 2020
  • Revised Date: August 25, 2021
  • Published Date: September 29, 2021
    Abstract
  • Resistive random access memory (RRAM) has been demonstrated to implement multiply-and-accumulate (MAC) operations using a highly parallel analog fashion, which dramatically accelerates the convolutional neural networks (CNNs). Since CNNs require considerable converters between analog crossbars and digital peripheral circuits, recent studies map the binary neural networks (BNNs) onto RRAM and binarize the weights to {+1, -1}. However, two mainstream representations for BNN weights introduce patterns of redundant 0s and 1s when dealing with negative weights. In this work, we reduce the area of redundant 0s and 1s by proposing a BNN weight representation framework based on the novel pattern representation and a corresponding architecture. First, we spilt the weight matrix into several small matrices by clustering adjacent columns together. Second, we extract 1s' patterns, i.e., the submatrices only containing 1s, from the small weight matrix, such that each final output can be represented by the sum of several patterns. Third, we map these patterns onto RRAM crossbars, including pattern computation crossbars (PCCs) and pattern accumulation crossbars (PACs). Finally, we compare the pattern representation with two mainstream representations and adopt the more area efficient one. The evaluation results demonstrate that our framework can save over 20% of crossbar area effectively, compared with two mainstream representations.
    • FullText(HTML)
    • References (21)
  • [1]
    Hinton G, Deng L, Yu D et al. Deep neural networks for acoustic modeling in speech recognition:The shared views of four research groups. IEEE Signal Process Mag., 2012, 29(6):82-97. DOI: 10.1109/MSP.2012.2205597.
    [2]
    Akinaga H, Shima H. Resistive random access memory (ReRAM) based on metal oxides. Proc. IEEE, 2010, 98(12):2237-2251. DOI: 10.1109/JPROC.2010.2070830.
    [3]
    Chi P, Li S, Xu C et al. PRIME:A novel processing-inmemory architecture for neural network computation in ReRAM-based main memory. In Proc. the 43rd International Symposium on Computer Architecture, Jun. 2016, pp.27-39. DOI: 10.1109/ISCA.2016.13.
    [4]
    Chen L, Li J, Chen Y et al. Accelerator-friendly neuralnetwork training:Learning variations and defects in RRAM crossbar. In Proc. the Design, Automation & Test in Europe Conference & Exhibition, Mar. 2017, pp.19-24. DOI: 10.23919/DATE.2017.7926952.
    [5]
    Liu C, Yan B, Yang C et al. A spiking neuromorphic design with resistive crossbar. In Proc. the 52nd Design Automation Conference, Jun. 2015. DOI: 10.1145/2744769.2744783.
    [6]
    Rastegari M, Ordonez V, Redmon J, Farhadi A. XNORNet:ImageNet classification using binary convolutional neural networks. In Proc. the 14th European Conference on Computer Vision, Oct. 2016, pp.525-542. DOI: 10.1007/978-3-319-46493-032.
    [7]
    Alemdar H, Leroy V, Prost-Boucle A, Pétrot F. Ternary neural networks for resource-efficient AI applications. In Proc. the International Joint Conference on Neural Networks, May 2017, pp.2547-2554. DOI: 10.1109/IJCNN.2017.7966166.
    [8]
    Tang T, Xia L, Li B, Wang Y, Yang H. Binary convolutional neural network on RRAM. In Proc. the 22nd Asia and South Pacific Design Automation Conference, Jan. 2017, pp.782-787. DOI: 10.1109/ASPDAC.2017.7858419.
    [9]
    Ni L, Liu Z, Song W et al. An energy-efficient and highthroughput bitwise CNN on sneak-path-free digital ReRAM crossbar. In Proc. the 2017 IEEE/ACM International Symposium on Low Power Electronics and Design, Jul. 2017. DOI: 10.1109/ISLPED.2017.8009177.
    [10]
    Sun X, Yin S, Peng X, Liu R, Seo J, Yu S. XNOR-RRAM:A scalable and parallel resistive synaptic architecture for binary neural networks. In Proc. the Design, Automation & Test in Europe Conference & Exhibition, Mar. 2018, pp.1423-1428. DOI: 10.23919/DATE.2018.8342235.
    [11]
    Sun X, Peng X, Chen P Y, Liu R, Seo J, Yu S. Fully parallel RRAM synaptic array for implementing binary neural network with (+1, -1) weights and (+1, 0) neurons. In Proc. the 23rd Asia and South Pacific Design Automation Conference, Jan. 2018, pp.574-579. DOI: 10.1109/ASPDAC.2018.8297384.
    [12]
    Wang P, Ji Y, Hong C, Lyu Y, Wang D, Xie Y. SNrram:An efficient sparse neural network computation architecture based on resistive random-access memory. In Proc. the 55th ACM/ESDA/IEEE Design Automation Conference, Jun. 2018. DOI: 10.1109/DAC.2018.8465793.
    [13]
    Chi C C, Jiang J H R. Logic synthesis of binarized neural networks for efficient circuit implementation. IEEE Trans. Comput. Des. Integr. Circuits Syst.. DOI: 10.1109/TCAD.2021.3078606.
    [14]
    Garey M R, Johnson D S, Stockmeyer L. Some simplified NP-complete problems. In Proc. the 6th ACM Symposium on Theory of Computing, Apr. 30-May 2, 1974, pp.47-63. DOI: 10.1145/800119.803884.
    [15]
    Kazemi A, Alessandri C, Seabaugh A C, Sharon H X, Niemier M, Joshi S. A device non-ideality resilient approach for mapping neural networks to crossbar arrays. In Proc. the 57th ACM/IEEE Design Automation Conference, Jul. 2020. DOI: 10.1109/DAC18072.2020.9218544.
    [16]
    Song L, Qian X, Li H, Chen Y. PipeLayer:A pipelined ReRAM-based accelerator for deep learning. In Proc. the International Symposium on High Performance Computer Architecture, Feb. 2017, pp.541-552. DOI: 10.1109/HPCA.2017.55.
    [17]
    Shafiee A, Nag A, Muralimanohar N et al. ISAAC:A convolutional neural network accelerator with in-situ analog arithmetic in crossbars. ACM SIGARCH Comput. Archit. News, 2016, 44(3):14-26. DOI: 10.1145/3007787.3001139.
    [18]
    Zhu Z, Sun H, Lin Y et al. A configurable multi-precision CNN computing framework based on single bit RRAM. In Proc. the 56th ACM/IEEE Design Automation Conference, Jun. 2019, Article No. 56. DOI: 10.1145/3316781.3317739.
    [19]
    Peng X, Liu R, Yu S. Optimizing weight mapping and dataflow for convolutional neural networks on processing-in-memory architectures. IEEE Trans. Circuits Syst. I Regul. Pap., 2020, 67(4):1333-1343. DOI: 10.1109/TCSI.2019.2958568.
    [20]
    Cheng M, Xia L, Zhu Z et al. TIME:A training-in-memory architecture for RRAM-based deep neural networks. IEEE Trans. Comput. Des. Integr. Circuits Syst., 2019, 38(5):834-847. DOI: 10.1109/TCAD.2018.2824304.
    [21]
    Zhu Z, Lin J, Cheng M et al. Mixed size crossbar based RRAM CNN accelerator with overlapped mapping method. In Proc. the International Conference on Computer-Aided Design, Nov. 2018, Article No. 69. DOI: 10.1145/3240765.3240825.
    • Relative Articles
  • Supplements (2)

  • Others

Catalog

    Get Citation
    PDF
    XML
    Read Online
    Article views (137) PDF downloads (0) Cited by()
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return