We use cookies to improve your experience with our site.

Indexed in:

SCIE, EI, Scopus, INSPEC, DBLP, CSCD, etc.

Submission System
(Author / Reviewer / Editor)
Advanced Search
Volume 36 Issue 2
March  2021
Turn off MathJax
Article Contents
Abstract
References
Supplements
Zhang-Jin Huang, Xiang-Xiang He, Fang-Jun Wang, Qing Shen. A Real-Time Multi-Stage Architecture for Pose Estimation of Zebrafish Head with Convolutional Neural Networks[J]. Journal of Computer Science and Technology, 2021, 36(2): 434-444. DOI: 10.1007/s11390-021-9599-5
Citation: Zhang-Jin Huang, Xiang-Xiang He, Fang-Jun Wang, Qing Shen. A Real-Time Multi-Stage Architecture for Pose Estimation of Zebrafish Head with Convolutional Neural Networks[J]. Journal of Computer Science and Technology, 2021, 36(2): 434-444. DOI: 10.1007/s11390-021-9599-5
shu

A Real-Time Multi-Stage Architecture for Pose Estimation of Zebrafish Head with Convolutional Neural Networks

  • 1 School of Computer Science and Technology, University of Science and Technology of China, Hefei 230027, China;
    2 School of Data Science, University of Science and Technology of China, Hefei 230027, China;
    3 Anhui Province Key Laboratory of Software in Computing and Communication, Hefei 230027, China

Funds: This work was supported in part by the National Key Research and Development Program of China under Grant No. 2018YFC1504104, the Fundamental Research Funds for the Central Universities of China under Grant No. WK6030000109, and the National Natural Science Foundation of China under Grant No. 61877056.
More Information
  • Author Bio:

    Zhang-Jin Huang received his B.S. and Ph.D. degrees in computational mathematics from University of Science and Technology of China (USTC), Hefei, in 1999 and 2005, respectively. He is currently an associate professor with the School of Computer Science and Technology, and the School of Data Science, USTC, Hefei. His current research interests include computer graphics, computer vision, machine learning and deep learning.

  • Received Date: March 29, 2019
  • Revised Date: June 02, 2019
  • Published Date: March 04, 2021
    Abstract
  • In order to conduct optical neurophysiology experiments on a freely swimming zebrafish, it is essential to quantify the zebrafish head to determine exact lighting positions. To efficiently quantify a zebrafish head's behaviors with limited resources, we propose a real-time multi-stage architecture based on convolutional neural networks for pose estimation of the zebrafish head on CPUs. Each stage is implemented with a small neural network. Specifically, a light-weight object detector named Micro-YOLO is used to detect a coarse region of the zebrafish head in the first stage. In the second stage, a tiny bounding box refinement network is devised to produce a high-quality bounding box around the zebrafish head. Finally, a small pose estimation network named tiny-hourglass is designed to detect keypoints in the zebrafish head. The experimental results show that using Micro-YOLO combined with RegressNet to predict the zebrafish head region is not only more accurate but also much faster than Faster R-CNN which is the representative of two-stage detectors. Compared with DeepLabCut, a state-of-the-art method to estimate poses for user-defined body parts, our multi-stage architecture can achieve a higher accuracy, and runs 19x faster than it on CPUs.
    • FullText(HTML)
    • References (22)
  • [1]
    Cong L, Wang Z, Chai Y, Han W, Shang C, Yang W, Bai L, Du J, Wang K, Wen Q. Rapid whole brain imaging of neural activity in freely behaving larval zebrafish (Danio rerio). Elife, 2017, 6:Article No. e28158. DOI: 10.7554/elife.28158.
    [2]
    Xu Z P, Cheng X E. Zebrafish tracking using convolutional neural networks. Scientific Reports, 2017, 7:Article No. 42815. DOI: 10.1038/srep42815.
    [3]
    Mathis A, Mamidanna P, Cury K M, Abe T, Murthy V N, Mathis M W, Bethge M. DeepLabCut:Markerless pose estimation of user-defined body parts with deep learning. Nature Neuroscience, 2018, 21:1281-1289. DOI: 10.1038/s41593-018-0209-y.
    [4]
    Girshick R, Donahue J, Darrell T, Malik J. Rich feature hierarchies for accurate object detection and semantic segmentation. In Proc. the 2014 IEEE Conference on Computer Vision and Pattern Recognition, June 2014, pp.580-587. DOI: 10.1109/CVPR.2014.81.
    [5]
    Girshick R. Fast R-CNN. In Proc. the 2015 IEEE International Conference on Computer Vision, December 2015, pp.1440-1448. DOI: 10.1109/ICCV.2015.169.
    [6]
    Ren S, He K, Girshick R, Sun J. Faster R-CNN:Towards real-time object detection with region proposal networks. In Proc. the 29th Annual Conference on Neural Information Processing Systems, December 2015, pp.91-99.
    [7]
    Dai J, Li Y, He K, Sun J. R-FCN:Object detection via region-based fully convolutional networks. In Proc. the 30th Annual Conference on Neural Information Processing Systems, December 2016, pp.379-387.
    [8]
    Uijlings J R, van de Sande K E, Gevers T, Smeulders A W. Selective search for object recognition. International Journal of Computer Vision, 2013, 104(2):154-171. DOI: 10.1007/s11263-013-0620-5.
    [9]
    Redmon J, Divvala S, Girshick R, Farhadi A. You only look once:Unified, real-time object detection. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.779-788. DOI: 10.1109/CVPR.2016.91.
    [10]
    Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu C Y, Berg A C. SSD:Single shot multibox detector. In Proc. the 14th European Conference on Computer Vision, October 2016, pp.21-37. DOI: 10.1007/978-3-319-46448-02.
    [11]
    Cai Z, Vasconcelos N. Cascade R-CNN:Delving into high quality object detection. In Proc. the 2018 IEEE Conference on Computer Vision and Pattern Recognition, June 2018, pp.6154-6162. DOI: 10.1109/CVPR.2018.00644.
    [12]
    Toshev A, Szegedy C. DeepPose:Human pose estimation via deep neural networks. In Proc. the 2014 IEEE Conference on Computer Vision and Pattern Recognition, June 2014, pp.1653-1660. DOI: 10.1109/CVPR.2014.214.
    [13]
    Pfister T, Simonyan K, Charles J, Zisserman A. Deep convolutional neural networks for efficient pose estimation in gesture videos. In Proc. the 12th Asian Conference on Computer Vision, November 2014, pp.538-552. DOI: 10.1007/978-3-319-16865-435.
    [14]
    Carreira J, Agrawal P, Fragkiadaki K, Malik J. Human pose estimation with iterative error feedback. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.4733-4742. DOI: 10.1109/CVPR.2016.512.
    [15]
    Pfister T, Charles J, Zisserman A. Flowing ConvNets for human pose estimation in videos. In Proc. the 2015 IEEE International Conference on Computer Vision, December 2015, pp.1913-1921. DOI: 10.1109/ICCV.2015.222.
    [16]
    Wei S E, Ramakrishna V, Kanade T, Sheikh Y. Convolutional pose machines. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.4724-4732. DOI: 10.1109/CVPR.2016.511.
    [17]
    Newell A, Yang K, Deng J. Stacked hourglass networks for human pose estimation. In Proc. the 14th European Conference on Computer Vision, October 2016, pp.483-499. DOI: 10.1007/978-3-319-46484-829.
    [18]
    Pishchulin L, Insafutdinov E, Tang S, Andres B, Andriluka M, Gehler P V, Schiele B. DeepCut:Joint subset partition and labeling for multi person pose estimation. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.4929-4937. DOI: 10.1109/CVPR.2016.533.
    [19]
    Insafutdinov E, Pishchulin L, Andres B, Andriluka M, Schiele B. Deepercut:A deeper, stronger, and faster multiperson pose estimation model. In Proc. the 14th European Conference on Computer Vision, October 2016, pp.34-50. DOI: 10.1007/978-3-319-46466-43.
    [20]
    Cao Z, Simon T, Wei S E, Sheikh Y. Realtime multiperson 2D pose estimation using part affinity fields. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, July 2017, pp.1302-1310. DOI: 10.1109/CVPR.2017.143.
    [21]
    He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.770-778. DOI: 10.1109/CVPR.2016.90.
    [22]
    Li S, Fang Z, Song W, Hao A, Qin H. Bidirectional optimization coupled lightweight networks for efficient and robust multi-person 2D pose estimation. Journal of Computer Science and Technology, 2019, 34(3):522-536. DOI: 10.1007/s11390-019-1924-x.
    • Relative Articles
  • Supplements (2)

  • Others

Catalog

    Get Citation
    PDF
    XML
    Read Online
    Article views (54) PDF downloads (0) Cited by()
    Related

    Copyright ©2025 JCST, All Rights Reserved     京ICP备05002829号-1

    Institute of Computing Technology, Chinese Academy of Sciences
    P.O. Box 2704, Beijing 100190, P.R. China

    Tel.: 86-10-62610746 E-mail: jcst@ict.ac.cn

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return