Journal of Computer Science and Technology ›› 2021, Vol. 36 ›› Issue (2): 323-333.doi: 10.1007/s11390-021-0782-5

Special Issue: Emerging Areas

• Special Section on AI and Big Data Analytics in Biology and Medicine • Previous Articles     Next Articles

Seg-CapNet: A Capsule-Based Neural Network for the Segmentation of Left Ventricle from Cardiac Magnetic Resonance Imaging

Yang-Jie Cao1, Member, CCF, Shuang Wu1, Chang Liu1, Nan Lin1, Yuan Wang2, Cong Yang1,*, Member, CCF, and Jie Li1,3, Senior Member, IEEE        

  1. 1 School of Software, Zhengzhou University, Zhengzhou 450000, China;
    2 Center of Modern Analysis and Gene Sequencing, Zhengzhou University, Zhengzhou 450000, China;
    3 Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200000, China
  • Received:2020-07-05 Revised:2021-03-09 Online:2021-03-05 Published:2021-04-01
  • Contact: Cong Yang E-mail:wangyuanyc@zzu.edu.cn
  • About author:Yang-Jie Cao is currently an associate professor of the School of Software, Zhengzhou University, Zhengzhou. He received his Ph.D. degree in computer science from Xi'an Jiaotong University, Xi'an, in 2012, and his M.S. degree in computer science from Zhengzhou University, Zhengzhou, in 2006. His current research interests include computer vision and intelligent computing, artificial intelligence, and high-performance computing.
  • Supported by:
    This work was supported by the Collaborative Innovation Major Project of Zhengzhou under Grant No. 20XTZX06013, and the National Natural Science Foundation of China under Grant No. 61932014.

PDF (PC)

48

Supplement

1

Deep neural networks (DNNs) have been extensively studied in medical image segmentation. However, existing DNNs often need to train shape models for each object to be segmented, which may yield results that violate cardiac anatomical structure when segmenting cardiac magnetic resonance imaging (MRI). In this paper, we propose a capsulebased neural network, named Seg-CapNet, to model multiple regions simultaneously within a single training process. The Seg-CapNet model consists of the encoder and the decoder. The encoder transforms the input image into feature vectors that represent objects to be segmented by convolutional layers, capsule layers, and fully-connected layers. And the decoder transforms the feature vectors into segmentation masks by up-sampling. Feature maps of each down-sampling layer in the encoder are connected to the corresponding up-sampling layers, which are conducive to the backpropagation of the model. The output vectors of Seg-CapNet contain low-level image features such as grayscale and texture, as well as semantic features including the position and size of the objects, which is beneficial for improving the segmentation accuracy. The proposed model is validated on the open dataset of the Automated Cardiac Diagnosis Challenge 2017 (ACDC 2017) and the Sunnybrook Cardiac Magnetic Resonance Imaging (MRI) segmentation challenge. Experimental results show that the mean Dice coefficient of Seg-CapNet is increased by 4.7% and the average Hausdorff distance is reduced by 22%. The proposed model also reduces the model parameters and improves the training speed while obtaining the accurate segmentation of multiple regions.

Key words: capsule neural network; image segmentation; left ventricle segmentation; cardiac magnetic resonance imaging;

[1] Cootes T F, Taylor C J, Cooper D H et al. Active shape models-Their training and application. Computer Vision and Image Understanding, 1995, 61(1):38-59. DOI:10.1006/cviu.1995.1004.
[2] Soliman A, Khalifa F, Elnakib A et al. Accurate lungs segmentation on CT chest images by adaptive appearance-guided shape modeling. IEEE Transactions on Medical Imaging, 2016, 36(1):263-276. DOI:10.1109/TMI.2016.2606370.
[3] Cootes T F, Edwards G J, Taylor C J. Active appearance models. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2001, 23(6):681-685. DOI:10.1109/34.927467.
[4] Matthews l, Baker S. Active appearance models revisited. International Journal of Computer Vision, 2004, 60:135-164. DOI:10.1023/B:VISI.0000029666.37597.d3
[5] Wachinger C, Fritscher K, Sharp G et al. Contourdriven atlas-based segmentation. IEEE Transactions on Medical Imaging, 2015, 34(12):2492-2505. DOI:10.1109/TMI.2015.2442753.
[6] Maintz J B, Viergever M A. A survey of medical image registration. Medical Image Analysis, 1998, 2(1):1-36. DOI:10.1016/S1361-8415(01)80026-8.
[7] Litjens G, Kooi T, Bejnordi B E et al. A survey on deep learning in medical image analysis. Medical Image Analysis, 2017, 42:60-88. DOI:10.1016/j.media.2017.07.005.
[8] LeCun Y, Bengio Y, Hinton G E. Deep learning. Nature, 2015, 521(7553):436-444. DOI:10.1038/nature14539.
[9] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. In Proc. the 26th Int. Conference on Neural Information Processing Systems, December 2012, pp.1097-1105. DOI:10.5555/2999134.2999257.
[10] Badrinarayanan V, Handa V, Cipolla R. SegNet:A deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling. arXiv:1505.07293, 2015. https://arxiv.org/pdf/1505.07293.pdf, March, 2020.
[11] Badrinarayanan V, Kendall A, Cipolla R. SegNet:A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495. DOI:10.1109/TPAMI.2016.2644615.
[12] Noh H, Hong S, Han B. Learning deconvolution network for semantic segmentation. In Proc. the 2015 IEEE International Conference on Computer Vision, December 2015, pp.1520-1528. DOI:10.1109/ICCV.2015.178.
[13] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In Proc. the 2015 IEEE International Conference on Computer Vision and Pattern Recognition, June 2015, pp.3431-3440. DOI:10.1109/CVPR.2015.7298965.
[14] Ronneberger O, Fischer P, Brox T. U-Net:Convolutional networks for biomedical image segmentation. In Proc. the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, October 2015, pp.234-241. DOI:10.1007/978-3-319-24574-428.
[15] Gu Z W, Cheng J, Fu H Z et al. CE-Net:Context encoder network for 2D medical image segmentation. IEEE Transactions on Medical Imaging, 2019, 38(10):2281-2292. DOI:10.1109/TMI.2019.2903562.
[16] Wang G, Liu X, Li C et al. A noise-robust framework for automatic segmentation of COVID-19 pneumonia lesions from CT images. IEEE Transactions on Medical Imaging, 2020, 39(8):2653-2663. DOI:10.1109/TMI.2020.3000314.
[17] Hu X, Li F, Samaras D et al. Topology-preserving deep image segmentation. In Proc. the 33rd Annual Conference of Neural Information Processing Systems, December 2019, pp.5658-5669.
[18] Karimi D, Salcudean S E. Reducing the Hausdorff Distance in medical image segmentation with convolutional neural networks. IEEE Transactions on Medical Imaging, 2020, 39(2):499-513. DOI:10.1109/TMI.2019.2930068.
[19] Moltz J H, Hänsch A, Lassen-Schmidt B et al. Learning a loss function for segmentation:A feasibility study. In Proc. the 17th IEEE Int. Biomedical Imaging Symp., April 2020, pp.357-360. DOI:10.1109/ISBI45749.2020.9098557.
[20] Hinton G E, Alex K, Wang S D. Transforming autoencoders. In Proc. the 21st Int. Conference on Artificial Neural Networks, June 2011, pp.44-51. DOI:10.1007/978-3-642-21735-76.
[21] Sabour S, Frosst N, Hinton G E. Dynamic routing between capsules. In Proc. the 31 st Int. Conference on Neural Information Processing Systems, December 2017, pp.3856-3866.
[22] LaLonde R, Bagci U. Capsules for object segmentation. arXiv:1804.04241, 2018. https://arxiv.org/pdf/1804.0424-1v1.pdf, March, 2020.
[23] Kromm C, Rohr K. Inception capsule network for retinal blood vessel segmentation and centerline extraction. In Proc. the 17th IEEE Int. Biomedical Imaging Symp., April 2020, pp.1223-1226. DOI:10.1109/ISBI45749.2020.9098538.
[24] He Y, Qin W, Wu Y et al. Automatic left ventricle segmentation from cardiac magnetic resonance images using a capsule network. Journal of X-Ray Science and Technology, 2020, 28(3):541-553. DOI:10.3233/XST-190621.
[25] Hara K, Saito D, Shouno H. Analysis of function of rectified linear unit used in deep learning. In Proc. the 2015 International Joint Conference on Neural Networks, July 2015. DOI:10.1109/IJCNN.2015.7280578.
[26] Loffe S, Szegedy C. Batch normalization:Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167, 2015. https://arxiv.org/pdf/1502.03167.pdf, March 2020.
[27] Bernard O, Lalande A, Zotti C et al. Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis:Is the problem solved? IEEE Transactions on Medical Imaging, 2018, 37(11):2514-2525. DOI:10.1109/TMI.2018.2837502.
[28] Chen L, Papandreou G, Schroff F et al. Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587, 2017. https://arxiv.org/abs/1706.05587, June 2020.
[29] Zhou Z W, Siddiquee M, Tajbakhsh N et al. UNet++:A nested U-Net architecture for medical image segmentation. In Proc. the 4th International Workshop on Deep Learning in Medical Image Analysis, September 2018, pp.3-11. DOI:10.1007/978-3-030-00889-51.
[1] Wen-Qian Deng, Xue-Mei Li, Xifeng Gao, Cai-Ming Zhang. A Modified Fuzzy C-Means Algorithm for Brain MR Image Segmentation and Bias Field Correction [J]. , 2016, 31(3): 501-511.
[2] Mahsa Chitsaz, and Chaw Seng Woo, Member, IEEE. Software Agent with Reinforcement Learning Approach for Medical Image Segmentation [J]. , 2011, 26(2): 247-255.
[3] Ma Feng; Xia Shaowei;. A Multiscale Approach to Automatic Medical Image Segmentation Using Self-Organizing Map [J]. , 1998, 13(5): 402-409.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] Li Wanxue;. Almost Optimal Dynamic 2-3 Trees[J]. , 1986, 1(2): 60 -71 .
[2] C.Y.Chung; H.R.Hwa;. A Chinese Information Processing System[J]. , 1986, 1(2): 15 -24 .
[3] Zhang Cui; Zhao Qinping; Xu Jiafu;. Kernel Language KLND[J]. , 1986, 1(3): 65 -79 .
[4] Huang Xuedong; Cai Lianhong; Fang Ditang; Chi Bianjin; Zhou Li; Jiang Li;. A Computer System for Chinese Character Speech Input[J]. , 1986, 1(4): 75 -83 .
[5] Shi Zhongzhi;. Knowledge-Based Decision Support System[J]. , 1987, 2(1): 22 -29 .
[6] Tang Tonggao; Zhao Zhaokeng;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[7] Xia Peisu; Fang Xinwo; Wang Yuxiang; Yan Kaiming; Zhang Tingjun; Liu Yulan; Zhao Chunying; Sun Jizhong;. Design of Array Processor Systems[J]. , 1987, 2(3): 163 -173 .
[8] Sun Yongqiang; Lu Ruzhan; Huang Xiaorong;. Termination Preserving Problem in the Transformation of Applicative Programs[J]. , 1987, 2(3): 191 -201 .
[9] Lin Qi; Xia Peisu;. The Design and Implementation of a Very Fast Experimental Pipelining Computer[J]. , 1988, 3(1): 1 -6 .
[10] Xie Li; Chen Peipei; Yang Peigen; Sun Zhongxiu;. The Design and Implementation of an OA System ZGL1[J]. , 1988, 3(1): 75 -80 .

ISSN 1000-9000(Print)

         1860-4749(Online)
CN 11-2296/TP

 Home
Editorial Board
Author Guidelines
Subscription		 Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
Tel.:86-10-62610746
E-mail: jcst@ict.ac.cn
 
  Copyright ©2015 JCST, All Rights Reserved