计算机科学技术学报 ›› 2021,Vol. 36 ›› Issue (2): 323-333.doi: 10.1007/s11390-021-0782-5

所属专题： Emerging Areas

• • 上一篇    下一篇

Seg-CapNet:一种用于心脏核磁共振左心室图像分割的胶囊神经网络

Yang-Jie Cao1, Member, CCF, Shuang Wu1, Chang Liu1, Nan Lin1, Yuan Wang2, Cong Yang1,*, Member, CCF, and Jie Li1,3, Senior Member, IEEE   

  1. 1 School of Software, Zhengzhou University, Zhengzhou 450000, China;
    2 Center of Modern Analysis and Gene Sequencing, Zhengzhou University, Zhengzhou 450000, China;
    3 Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200000, China
  • 收稿日期:2020-07-05 修回日期:2021-03-09 出版日期:2021-03-05 发布日期:2021-04-01
  • 通讯作者: Cong Yang E-mail:wangyuanyc@zzu.edu.cn
  • 作者简介:Yang-Jie Cao is currently an associate professor of the School of Software, Zhengzhou University, Zhengzhou. He received his Ph.D. degree in computer science from Xi'an Jiaotong University, Xi'an, in 2012, and his M.S. degree in computer science from Zhengzhou University, Zhengzhou, in 2006. His current research interests include computer vision and intelligent computing, artificial intelligence, and high-performance computing.
  • 基金资助:
    This work was supported by the Collaborative Innovation Major Project of Zhengzhou under Grant No. 20XTZX06013, and the National Natural Science Foundation of China under Grant No. 61932014.

Seg-CapNet: A Capsule-Based Neural Network for the Segmentation of Left Ventricle from Cardiac Magnetic Resonance Imaging

Yang-Jie Cao1, Member, CCF, Shuang Wu1, Chang Liu1, Nan Lin1, Yuan Wang2, Cong Yang1,*, Member, CCF, and Jie Li1,3, Senior Member, IEEE        

  1. 1 School of Software, Zhengzhou University, Zhengzhou 450000, China;
    2 Center of Modern Analysis and Gene Sequencing, Zhengzhou University, Zhengzhou 450000, China;
    3 Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200000, China
  • Received:2020-07-05 Revised:2021-03-09 Online:2021-03-05 Published:2021-04-01
  • Contact: Cong Yang E-mail:wangyuanyc@zzu.edu.cn
  • About author:Yang-Jie Cao is currently an associate professor of the School of Software, Zhengzhou University, Zhengzhou. He received his Ph.D. degree in computer science from Xi'an Jiaotong University, Xi'an, in 2012, and his M.S. degree in computer science from Zhengzhou University, Zhengzhou, in 2006. His current research interests include computer vision and intelligent computing, artificial intelligence, and high-performance computing.
  • Supported by:
    This work was supported by the Collaborative Innovation Major Project of Zhengzhou under Grant No. 20XTZX06013, and the National Natural Science Foundation of China under Grant No. 61932014.

PDF (PC)

48

补充材料

1

1、研究背景（context）。
图像分割根据灰度、颜色、纹理、形状等特征将每个像素划分到不重叠的区域。图像分割是计算机视觉和图像处理的一项基本任务，是实现目标跟踪、计算机辅助诊断等高级计算机视觉任务的基础。在医学影像中，准确的组织分割可以定量的测量病变的组织形态学参数等病理指标，为临床诊断、治疗和病理研究提供可靠的依据。深度神经网络在医学图像分割方向已进行了广泛的研究，但由于图像对比度差、噪声和区域重叠等原因使得准确地分割仍具有一定的挑战性。
2、目的（Objective）：
研究目的是在一次训练过程中对多个重叠目标同时建模，以同时分割左心室的心内膜和心外膜。
3、方法（Method）：
Seg-CapNet由卷积层，胶囊层，全连接层和反卷积层组成。卷积层用来提取图像底层信息，并组成原始胶囊层。胶囊层通过动态路由产生包含目标物体底层和高层语义信息的特征向量。动态路由导致的物体组成部分间的空间位置关系在全连接层中得以恢复。在反卷积过程中，通过跳跃连接加速反向传播过程，从而缩短训练时间。
4、结果（Result&Findings）：
实验结果表明，Seg-CapNet的平均Dice（Dice Coefficient）提高了4.7%，平均HD（Hausdorff Distance）降低了22%。该模型在保证重叠区域准确分割的同时，减少了参数的数量，提高了训练速度。
5、结论（Conclusions）：
在本文中，我们提出了一种基于胶囊网络的神经网络模型Seg-CapNet与一个新的损失函数。相较于主流方法，Seg-CapNet不仅可以同时提取左心室的心内膜和心外膜，并在ACDC 2017和Sunnybrook数据上的Dice和HD两种评价指标上也有更好的表现。另外，Seg-CapNet的参数量少，计算成本低且网络结构易于扩展。

关键词: 胶囊网络, 图像分割, 左心室分割, 心脏核磁共振

Abstract: Deep neural networks (DNNs) have been extensively studied in medical image segmentation. However, existing DNNs often need to train shape models for each object to be segmented, which may yield results that violate cardiac anatomical structure when segmenting cardiac magnetic resonance imaging (MRI). In this paper, we propose a capsulebased neural network, named Seg-CapNet, to model multiple regions simultaneously within a single training process. The Seg-CapNet model consists of the encoder and the decoder. The encoder transforms the input image into feature vectors that represent objects to be segmented by convolutional layers, capsule layers, and fully-connected layers. And the decoder transforms the feature vectors into segmentation masks by up-sampling. Feature maps of each down-sampling layer in the encoder are connected to the corresponding up-sampling layers, which are conducive to the backpropagation of the model. The output vectors of Seg-CapNet contain low-level image features such as grayscale and texture, as well as semantic features including the position and size of the objects, which is beneficial for improving the segmentation accuracy. The proposed model is validated on the open dataset of the Automated Cardiac Diagnosis Challenge 2017 (ACDC 2017) and the Sunnybrook Cardiac Magnetic Resonance Imaging (MRI) segmentation challenge. Experimental results show that the mean Dice coefficient of Seg-CapNet is increased by 4.7% and the average Hausdorff distance is reduced by 22%. The proposed model also reduces the model parameters and improves the training speed while obtaining the accurate segmentation of multiple regions.

Key words: capsule neural network, image segmentation, left ventricle segmentation, cardiac magnetic resonance imaging

[1] Cootes T F, Taylor C J, Cooper D H et al. Active shape models-Their training and application. Computer Vision and Image Understanding, 1995, 61(1):38-59. DOI:10.1006/cviu.1995.1004.
[2] Soliman A, Khalifa F, Elnakib A et al. Accurate lungs segmentation on CT chest images by adaptive appearance-guided shape modeling. IEEE Transactions on Medical Imaging, 2016, 36(1):263-276. DOI:10.1109/TMI.2016.2606370.
[3] Cootes T F, Edwards G J, Taylor C J. Active appearance models. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2001, 23(6):681-685. DOI:10.1109/34.927467.
[4] Matthews l, Baker S. Active appearance models revisited. International Journal of Computer Vision, 2004, 60:135-164. DOI:10.1023/B:VISI.0000029666.37597.d3
[5] Wachinger C, Fritscher K, Sharp G et al. Contourdriven atlas-based segmentation. IEEE Transactions on Medical Imaging, 2015, 34(12):2492-2505. DOI:10.1109/TMI.2015.2442753.
[6] Maintz J B, Viergever M A. A survey of medical image registration. Medical Image Analysis, 1998, 2(1):1-36. DOI:10.1016/S1361-8415(01)80026-8.
[7] Litjens G, Kooi T, Bejnordi B E et al. A survey on deep learning in medical image analysis. Medical Image Analysis, 2017, 42:60-88. DOI:10.1016/j.media.2017.07.005.
[8] LeCun Y, Bengio Y, Hinton G E. Deep learning. Nature, 2015, 521(7553):436-444. DOI:10.1038/nature14539.
[9] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. In Proc. the 26th Int. Conference on Neural Information Processing Systems, December 2012, pp.1097-1105. DOI:10.5555/2999134.2999257.
[10] Badrinarayanan V, Handa V, Cipolla R. SegNet:A deep convolutional encoder-decoder architecture for robust semantic pixel-wise labelling. arXiv:1505.07293, 2015. https://arxiv.org/pdf/1505.07293.pdf, March, 2020.
[11] Badrinarayanan V, Kendall A, Cipolla R. SegNet:A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495. DOI:10.1109/TPAMI.2016.2644615.
[12] Noh H, Hong S, Han B. Learning deconvolution network for semantic segmentation. In Proc. the 2015 IEEE International Conference on Computer Vision, December 2015, pp.1520-1528. DOI:10.1109/ICCV.2015.178.
[13] Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In Proc. the 2015 IEEE International Conference on Computer Vision and Pattern Recognition, June 2015, pp.3431-3440. DOI:10.1109/CVPR.2015.7298965.
[14] Ronneberger O, Fischer P, Brox T. U-Net:Convolutional networks for biomedical image segmentation. In Proc. the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, October 2015, pp.234-241. DOI:10.1007/978-3-319-24574-428.
[15] Gu Z W, Cheng J, Fu H Z et al. CE-Net:Context encoder network for 2D medical image segmentation. IEEE Transactions on Medical Imaging, 2019, 38(10):2281-2292. DOI:10.1109/TMI.2019.2903562.
[16] Wang G, Liu X, Li C et al. A noise-robust framework for automatic segmentation of COVID-19 pneumonia lesions from CT images. IEEE Transactions on Medical Imaging, 2020, 39(8):2653-2663. DOI:10.1109/TMI.2020.3000314.
[17] Hu X, Li F, Samaras D et al. Topology-preserving deep image segmentation. In Proc. the 33rd Annual Conference of Neural Information Processing Systems, December 2019, pp.5658-5669.
[18] Karimi D, Salcudean S E. Reducing the Hausdorff Distance in medical image segmentation with convolutional neural networks. IEEE Transactions on Medical Imaging, 2020, 39(2):499-513. DOI:10.1109/TMI.2019.2930068.
[19] Moltz J H, Hänsch A, Lassen-Schmidt B et al. Learning a loss function for segmentation:A feasibility study. In Proc. the 17th IEEE Int. Biomedical Imaging Symp., April 2020, pp.357-360. DOI:10.1109/ISBI45749.2020.9098557.
[20] Hinton G E, Alex K, Wang S D. Transforming autoencoders. In Proc. the 21st Int. Conference on Artificial Neural Networks, June 2011, pp.44-51. DOI:10.1007/978-3-642-21735-76.
[21] Sabour S, Frosst N, Hinton G E. Dynamic routing between capsules. In Proc. the 31 st Int. Conference on Neural Information Processing Systems, December 2017, pp.3856-3866.
[22] LaLonde R, Bagci U. Capsules for object segmentation. arXiv:1804.04241, 2018. https://arxiv.org/pdf/1804.0424-1v1.pdf, March, 2020.
[23] Kromm C, Rohr K. Inception capsule network for retinal blood vessel segmentation and centerline extraction. In Proc. the 17th IEEE Int. Biomedical Imaging Symp., April 2020, pp.1223-1226. DOI:10.1109/ISBI45749.2020.9098538.
[24] He Y, Qin W, Wu Y et al. Automatic left ventricle segmentation from cardiac magnetic resonance images using a capsule network. Journal of X-Ray Science and Technology, 2020, 28(3):541-553. DOI:10.3233/XST-190621.
[25] Hara K, Saito D, Shouno H. Analysis of function of rectified linear unit used in deep learning. In Proc. the 2015 International Joint Conference on Neural Networks, July 2015. DOI:10.1109/IJCNN.2015.7280578.
[26] Loffe S, Szegedy C. Batch normalization:Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167, 2015. https://arxiv.org/pdf/1502.03167.pdf, March 2020.
[27] Bernard O, Lalande A, Zotti C et al. Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis:Is the problem solved? IEEE Transactions on Medical Imaging, 2018, 37(11):2514-2525. DOI:10.1109/TMI.2018.2837502.
[28] Chen L, Papandreou G, Schroff F et al. Rethinking atrous convolution for semantic image segmentation. arXiv:1706.05587, 2017. https://arxiv.org/abs/1706.05587, June 2020.
[29] Zhou Z W, Siddiquee M, Tajbakhsh N et al. UNet++:A nested U-Net architecture for medical image segmentation. In Proc. the 4th International Workshop on Deep Learning in Medical Image Analysis, September 2018, pp.3-11. DOI:10.1007/978-3-030-00889-51.
[1] Wei Du, Yu Sun, Hui-Min Bao, Liang Chen, Ying Li, Yan-Chun Liang. 基于迁移学习与深度学习的人类血液分泌蛋白预测框架[J]. 计算机科学技术学报, 2021, 36(2): 234-247.
[2] Wen-Qian Deng, Xue-Mei Li, Xifeng Gao, Cai-Ming Zhang. 一种MRI脑部组织图像偏场校正和分割的FCM新算法[J]. , 2016, 31(3): 501-511.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] 李万学;. Almost Optimal Dynamic 2-3 Trees[J]. , 1986, 1(2): 60 -71 .
[2] C.Y.Chung; 华宣仁;. A Chinese Information Processing System[J]. , 1986, 1(2): 15 -24 .
[3] 章萃; 赵沁平; 徐家福;. Kernel Language KLND[J]. , 1986, 1(3): 65 -79 .
[4] 黄学东; 蔡莲红; 方棣棠; 迟边进; 周立; 蒋力;. A Computer System for Chinese Character Speech Input[J]. , 1986, 1(4): 75 -83 .
[5] 史忠植;. Knowledge-Based Decision Support System[J]. , 1987, 2(1): 22 -29 .
[6] 唐同诰; 招兆铿;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[7] 夏培肃; 方信我; 王玉祥; 严开明; 张廷军; 刘玉兰; 赵春英; 孙继忠;. Design of Array Processor Systems[J]. , 1987, 2(3): 163 -173 .
[8] 孙永强; 陆汝占; 黄小戎;. Termination Preserving Problem in the Transformation of Applicative Programs[J]. , 1987, 2(3): 191 -201 .
[9] 林琦; 夏培肃;. The Design and Implementation of a Very Fast Experimental Pipelining Computer[J]. , 1988, 3(1): 1 -6 .
[10] 谢立; 陈珮珮; 杨培根; 孙钟秀;. The Design and Implementation of an OA System ZGL1[J]. , 1988, 3(1): 75 -80 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持：support@magtech.com.cn
总访问量：