|
计算机科学技术学报 ›› 2021,Vol. 36 ›› Issue (2): 334-346.doi: 10.1007/s11390-021-0861-7
所属专题： Emerging Areas
基于深度学习与波束偏转的穿刺针定位与增强算法
Jun Gao1,2, Paul Liu2, Guang-Di Liu3, and Le Zhang1,4,5,*, Senior Member, CCF, Member, ACM
- 1 College of Computer Science, Sichuan University, Chengdu 610065, China;
2 Stork Healthcare, Chengdu 610000, China;
3 College of Computer and Information Science, Southwest University, Chongqing 400715, China;
4 West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu 610065, China;
5 Pera Corporation Ltd., Beijing 100025, China
Robust Needle Localization and Enhancement Algorithm for Ultrasound by Deep Learning and Beam Steering Methods
Jun Gao1,2, Paul Liu2, Guang-Di Liu3, and Le Zhang1,4,5,*, Senior Member, CCF, Member, ACM 
- 1 College of Computer Science, Sichuan University, Chengdu 610065, China;
2 Stork Healthcare, Chengdu 610000, China;
3 College of Computer and Information Science, Southwest University, Chongqing 400715, China;
4 West China Biomedical Big Data Center, West China Hospital, Sichuan University, Chengdu 610065, China;
5 Pera Corporation Ltd., Beijing 100025, China
1、研究背景（Context）：
目前，超声在活检，麻醉，治疗性注射等介入性手术中应用非常广泛。由于超声系统成像的特点以及穿刺针表面光滑的特性，因此使得穿刺针在超声图像中非常模糊，甚至不可见。为了精确定位与增强穿刺针，更好的服务超声介入的临床应用，目前有很多基于硬件和基于软件的方法试图解决这个问题。基于硬件的方法主要包括：基于磁导航的方、基于红外导航的方法、基于针尖附件传感器的方法等。这些方法可以精确的检测并强化穿刺针，但是它们普遍价格昂贵、对手术环境要求高以及妨碍传统的临床应用工作流，因此应用并不广泛。基于软件的方法主要包括基于线检测的算法、基于3D的算法、基于投影的算法、基于学习的方法等。目前除了基于学习的算法外，其他所有算法都表现的很不稳定，尤其当穿刺针的反射强度很弱的时候，这些算法都不能准确的定位与强化显示穿刺针。随着机器学习和深度学习的发展，很多基于学习的算法被提出。很多基于学习的方法可以准确的定位穿刺针的位置并强化显示，但是由于算法的复杂度等原因，它们不能实时处理的运用在临床应用中。有些算法目前可以实时的定位穿刺针的针尖位置，但是不能定位穿刺针针体的位置，所以此类方法不能强化并显示穿刺针。因此，开发一个能精确且实时的穿刺针定位与强化的算法是一项具有挑战与具有现实意义的工作。
2、目的（Objective）：
我们的研究目标是提出一种可以精确的、实时的穿刺针定位与强化算法，可以满足超声介入手术的临床需求。
3、方法（Method）：
我们提出了一种基于深度学习与偏转扫描的穿刺针定位与强化算法（NLEM）。首先，通过增加超声偏转波束，并使偏转波束始终垂直于穿刺针，这样将最大程度的增加穿刺针的反射信号，为后续处理奠定良好基础。然后，我们通过修改U-net的网络架构设计了一个多任务神经网络。将U-net网络的最后一个上采样层修改成两个分支，一个分支用来分割穿刺针针体的位置，另一个分支用来分割穿刺针针尖的LandMark。同时，在U-net网络的最后一个下采层后增加一个分支网络用于判别图像中是否包含穿刺针。在网络训练中，我们使用对抗网络（PatchGan）进行监督训练。对抗训练用于去除分割伪影，提高分割精度。最后，运用多任务神经网络的输出结果，我们设计了一个穿刺针融合框架，此框架通过一系列基本的图像处理算法提取并强化显示穿刺针。
4、结果（Result&Findings）：
实验结果表明NLEM方法可以精确并且实时的定位与强化穿刺针。针尖的定位精度大约为0.29±0.02毫米，穿刺针针体的定位精度为0.27±0.02度，总处理时间为0.0149±0.0001秒。同时，我们发现偏转扫描引入的超声Grating Lobe伪影噪声对穿刺针针尖的定位精度影响较大，当其正好覆盖到穿刺针针尖位置时，穿刺针针尖的定位精度会下降。另外，我们也发现穿刺针插入较短的样本，对于我们的分类模型是困难样本，分类失败的测试样本基本都发生在短针样本中。
5、结论（Conclusions）：
我们提出的NLEM方法可以精准并且实时的定位与增强显示穿刺针，满足临床超声介入手术的需求。将来我们可以通过超声信号处理的方法，在偏转波束上抑制Grating Lobe伪影噪声，进一步提高穿刺针针尖的定位精度。另外，在活体的人体组织上进行试验验证也是将来工作之一。
| [1] Gulsen F, Kantarci F. Ultrasound-guided intervention around the hip joint. American Journal of Roentgenology, 2012, 198(1):W95. DOI:10.2214/AJR.11.7627. [2] Chin K J, Perlas A, Chan V W S, Brull R. Needle visualization in ultrasound-guided regional anesthesia:Challenges and solutions. Regional Anesthesia and Pain Medicine, 2008, 33(6):532-544. DOI:10.1016/j.rapm.2008.06.002. [3] Fevre M C, Vincent C, Picard J, Vighetti A, Chapuis C, Detavernier M, Allenet B, Payen J F, Bosson J L, Albaladejo P. Reduced variability and execution time to reach a target with a needle GPS system:Comparison between physicians, residents and nurse anaesthetists. Anaesthesia Critical Care & Pain Medicine, 2018, 37(1):55-60. DOI:10.1016/j.accpm.2016.05.008. [4] Stolka P J, Foroughi P, Rendina M, Weiss C R, Hager G D, Boctor E M. Needle guidance using handheld stereo vision and projection for ultrasound-based interventions. In Proc. the 17th International Conference on Medical Image Computing and Computer-Assisted Intervention, September 2014, pp.684-691. DOI:10.1007/978-3-319-10470-685. [5] Lu H, Li J, Lu Q, Bharat S, Erkamp R, Chen B, Drysdale J, Vignon F, Jain A. A new sensor technology for 2D ultrasound-guided needle tracking. In Proc. the 17th International Conference on Medical Image Computing and Computer-Assisted Intervention, September 2014, pp.389-396. DOI:10.1007/978-3-319-10470-649. [6] Xia W, West S J, Finlay M C, Mari J M, Ourselin S, David A L, Desjardins A E. Looking beyond the imaging plane:3D needle tracking with a linear array ultrasound probe. Sci. Rep., 2017, 7(1):Article No. 3674. DOI:10.1038/s41598-017-03886-4. [7] Ding M, Fenster A. Projection-based needle segmentation in 3D ultrasound images. In Proc. the 6th International Conference on Medical Image Computing and ComputerAssisted Intervention, November 2003, pp.319-327. DOI:10.3109/10929080500079321. [8] Qiu W, Yuchi M, Ding M. Phase grouping-based needle segmentation in 3-D trans-rectal ultrasoundguided prostate trans-perineal therapy. Ultrasound in Medicine & Biology, 2014, 40(4):804-816. DOI:10.1016/j.ultrasmedbio.2013.11.004. [9] Beigi P, Rohling R, Salcudean T, Lessoway V A, Ng G C. Needle trajectory and tip localization in realtime 3-D ultrasound using a moving stylus. Ultrasound in Medicine & Biology, 2015, 41(7):2057-2070. DOI:10.1016/j.ultrasmedbio.2015.03.013. [10] Zhao Y, Shen Y, Bernard A, Cachard C, Liebgott H. Evaluation and comparison of current biopsy needle localization and tracking methods using 3D ultrasound. Ultrasonics, 2017, 73:206-220. DOI:10.1016/j.ultras.2016.09.006. [11] Ayvali E, Desai J P. Optical flow-based tracking of needles and needle-tip localization using circular hough transform in ultrasound images. Annals of Biomedical Engineering, 2015, 43(8):1828-1840. DOI:10.1007/s10439-014-1208-0. [12] Uhercik M, Kybic J, Liebgott H, Cachard C. Model fitting using RANSAC for surgical tool localization in 3D ultrasound images. IEEE Transactions on Biomedical Engineering, 2010, 57(8):1907-1916. DOI:10.1109/TBME.2010.2046416. [13] Zhao Y, Cachard C, Liebgott H. Automatic needle detection and tracking in 3D ultrasound using an ROI-based RANSAC and Kalman method. Ultrasonic Imaging, 2013, 35(4):283-306. DOI:10.1177/0161734613502004. [14] Mwikirize C, Nosher J L, Hacihaliloglu I. Learning needle tip localization from digital subtraction in 2D ultrasound. International Journal of Computer Assisted Radiology and Surgery, 2019, 14(6):1017-1026. DOI:10.1007/s11548-019-01951-z. [15] Mwikirize C, Nosher J L, Hacihaliloglu I. Convolution neural networks for real-time needle detection and localization in 2D ultrasound. International Journal of Computer Assisted Radiology and Surgery, 2018, 13(5):647-657. DOI:10.1007/s11548-018-1721-y. [16] Beigi P, Rohling R, Salcudean S E, Ng G C. CASPER:Computer-aided segmentation of imperceptible motion-A learning-based tracking of an invisible needle in ultrasound. International Journal of Computer Assisted Radiology and Surgery, 2017, 12(11):1857-1866. DOI:10.1007/s11548-017-1631-4. [17] Pourtaherian A, Ghazvinian Zanjani F, Zinger S, Mihajlovic N, Ng G C, Korsten H H M, De With P H N. Robust and semantic needle detection in 3D ultrasound using orthogonal-plane convolutional neural networks. International Journal of Computer Assisted Radiology and Surgery, 2018, 13(9):1321-1333. DOI:10.1007/s11548-018-1798-3. [18] Arif M, Moelker A, Van Walsum T. Automatic needle detection and real-time Bi-planar needle visualization during 3D ultrasound scanning of the liver. Medical Image Analysis, 2019, 53:104-110. DOI:10.1016/j.media.2019.02.002. [19] Mwikirize C, Nosher J L, Hacihaliloglu I. Single shot needle tip localization in 2D ultrasound. In Proc. the 22nd International Conference on Medical Image Computing and Computer Assisted Intervention, October 2019, pp.637-645. DOI:10.1007/978-3-030-32254-071. [20] Hatt C R, Ng G, Parthasarathy V. Enhanced needle localization in ultrasound using beam steering and learning-based segmentation. Computerized Medical Imaging and Graphics, 2015, 41:46-54. DOI:10.1016/j.compmedimag.2014.06.016. [21] Ronneberger O, Fischer P, Brox T. U-Net:Convolutional networks for biomedical image segmentation. In Proc. the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, October 2015, pp. 234-241. DOI:10.1007/978-3-319-24574-428. [22] Gao J, Liu P, Liu G D, Zhang L. Supplementary material. Technical Report, Sichuan University, 2020. https://github.com/gaojun0821/NLEM/blob/main/doc/-Supplementary Material.pdf, Dec. 2020. [23] Jiang B, Struthers A, Sun Z, Feng Z, Zhao X, Zhao K, Dai W, Zhou X, Berens M E, Zhang L. Employing graphics processing unit technology, alternating direction implicit method and domain decomposition to speed up the numerical diffusion solver for the biomedical engineering research. International Journal for Numerical Methods in Biomedical Engineering, 2011, 27(11):1829-1849. DOI:10.1002/cnm.1444. [24] Jiang B N, Dai W Z, Khaliq A, Carey M, Zhou X B, Zhang L. Novel 3D GPU based numerical parallel diffusion algorithms in cylindrical coordinates for health care simulation. Mathematics and Computers in Simulation, 2015, 109:1-19. DOI:10.1016/j.matcom.2014.07.003. [25] Zhang L, Jiang B, Wu Y, Strouthos C, Sun P Z, Su J, Zhou X. Developing a multiscale, multi-resolution agentbased brain tumor model by graphics processing units. Theoretical Biology and Medical Modelling, 2011, 8(1):Article No. 46. DOI:10.1186/1742-4682-8-46. [26] Isola P, Zhu J Y, Zhou T, Efros A. Image-to-image translation with conditional adversarial networks. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition, July 2017, pp.5967-5976. DOI:10.1109/CVPR.2017.632. [27] Wu W, Song L, Yang Y, Wang J, Liu H, Zhang L. Exploring the dynamics and interplay of human papillomavirus and cervical tumorigenesis by integrating biological data into a mathematical model. BMC Bioinformatics, 2020, 21(Suppl 7):Article No. 152. DOI:10.1186/s12859-020-3454-5. [28] Ren S, He K, Girshick R, Sun J. Faster R-CNN:Towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6):1137-1149. DOI:10.1109/TPAMI.2016.2577031. [29] Xu Z, Huo Y, Park J, Landman B, Milkowski A, Grbic S, Zhou S. Less is more:Simultaneous view classification and landmark detection for abdominal ultrasound images. In Proc. the 21st International Conference on Medical Image Computing and Computer-Assisted Intervention, September 2018, pp.711-719. DOI:10.1007/978-3-030-00934-279. [30] Powers D. Evaluation:From precision, recall and Ffactor to ROC, informedness, markedness & correlation. arXiv:2010.16061, 2008. https://arxiv.org/abs/2010.16061, Jan. 2021. [31] Xia Y, Yang C, Hu N, Yang Z, He X, Li T, Zhang L. Exploring the key genes and signaling transduction pathways related to the survival time of glioblastoma multiforme patients by a novel survival analysis model. BMC Genomics, 2017, 18(Suppl 1):Article No. 950. DOI:10.1186/s12864-016-3256-3. [32] Zhang L, Bai W, Yuan N, Du Z. Comprehensively benchmarking applications for detecting copy number variation. PLoS Computational Biology, 2019, 15(5):Article No. e1007069. DOI:10.1371/journal.pcbi.1007069. [33] Zhang L, Li J, Yin K, Jiang Z, Li T, Hu R, Yu Z, Feng H, Chen Y. Computed tomography angiography-based analysis of high-risk intracerebral haemorrhage patients by employing a mathematical model. BMC Bioinformatics, 2019, 20(Suppl 7):Article No. 193. DOI:10.1186/s12859-019-2741-5. [34] Zhang L, Liu Y, Wang M, Wu Z, Li N, Zhang J, Yang C. EZH2-, CHD4-, and IDH-linked epigenetic perturbation and its association with survival in glioma patients. J. Mol. Cell Biol., 2017, 9(6):477-488. DOI:10.1093/jmcb/mjx056. [35] Kremkau F W, Taylor K J. Artifacts in ultrasound imaging. Journal of Ultrasound in Medicine, 1986, 5(4):227-237. DOI:10.7863/jum.1986.5.4.227. [36] Paul Y, Barthez D, Léveillé R, Peter V, Scrivani D. Side lobes and grating lobes artifacts in ultrasound imaging. Veterinary Radiology & Ultrasound, 1997, 38(5):387-393. DOI:10.1111/j.1740-8261.1997.tb02104.x. [37] Matalon T A, Silver B. US guidance of interventional procedures. Radiology, 1990, 174(1):43-47. DOI:10.1148/radiology.174.1.2403684. [38] Sofka C, Collins A J, Adler R. Use of ultrasonographic guidance in interventional musculoskeletal procedures:A review from a single institution. Journal of Ultrasound in Medicine:Official Journal of the American Institute of Ultrasound in Medicine, 2001, 20(1):21-26. DOI:10.7863/jum.2001.20.1.21. |
| [1] | 张鑫, 陆思源, 王水花, 余翔, 王甦菁, 姚仑, 潘毅, 张煜东. 通过新型深度学习架构诊断COVID-19肺炎[J]. 计算机科学技术学报, 2022, 37(2): 330-343. |
| [2] | 孔雀屏, 王子彦, 黄袁, 陈湘萍, 周晓聪, 郑子彬, 黄罡. 定义和检测智能合约中低效率的Gas模式[J]. 计算机科学技术学报, 2022, 37(1): 67-82. |
| [3] | Yu-Wei Wu, Qing-Gang Wang, Long Zheng, Xiao-Fei Liao, Hai Jin, Wen-Bin Jiang, Ran Zheng, Kan Hu. FDGLib：在数据中心中支持高效大规模图计算加速的通信库[J]. 计算机科学技术学报, 2021, 36(5): 1051-1070. |
| [4] | Yi Zhong, Jian-Hua Feng, Xiao-Xin Cui, Xiao-Le Cui. 机器学习辅助的抗逻辑块加密密钥猜测攻击范式[J]. 计算机科学技术学报, 2021, 36(5): 1102-1117. |
| [5] | Xiao-Jing Zha, Yin-Shui Xia, Shang-Luan Xie, Zhu-Fei Chu. 面向时延优化的CMOL电路容错映射[J]. 计算机科学技术学报, 2021, 36(5): 1118-1132. |
| [6] | Hui-Ming Tian, Zhu-Fei Chu. 基于具有互补属性器件的反相器优化策略[J]. 计算机科学技术学报, 2021, 36(5): 1145-1154. |
| [7] | Songjie Niu, Shimin Chen. TransGPerf：利用迁移学习建模分布式图计算性能[J]. 计算机科学技术学报, 2021, 36(4): 778-791. |
| [8] | Zhi-Xin Qi, Hong-Zhi Wang, An-Jie Wang. 针对劣质数据对分类和聚类模型影响的实验评估[J]. 计算机科学技术学报, 2021, 36(4): 806-821. |
| [9] | Jing-Xuan Zhang, Chuan-Qi Tao, Zhi-Qiu Huang, Xin Chen. 使用文本分类检测API规范中的API指令[J]. 计算机科学技术学报, 2021, 36(4): 922-943. |
| [10] | Yu-Wen Huang, Gong-Ping Yang, Kui-Kui Wang, Hai-Ying Liu, Yi-Long Yin. 基于多尺度深度级联双边森林的心电身份识别[J]. 计算机科学技术学报, 2021, 36(3): 617-632. |
| [11] | Sheng-Luan Hou, Xi-Kun Huang, Chao-Qun Fei, Shu-Han Zhang, Yang-Yang Li, Qi-Lin Sun, Chuan-Qing Wang. 基于深度学习的文本摘要研究综述[J]. 计算机科学技术学报, 2021, 36(3): 633-663. |
| [12] | Wen-Jun Yang, Bei-Ji Zou, Kai-Wen Li, Shu Liu. 面向多方向场景文字检测的字符流框架[J]. 计算机科学技术学报, 2021, 36(3): 465-477. |
| [13] | Lan Chen, Juntao Ye, Xiaopeng Zhang. 基于多特征超分网络的布料褶皱合成[J]. 计算机科学技术学报, 2021, 36(3): 478-493. |
| [14] | Yu-Jie Yuan, Yukun Lai, Tong Wu, Lin Gao, Li-Gang Liu. 回顾形状编辑技术：从几何角度到神经网络方法[J]. 计算机科学技术学报, 2021, 36(3): 520-554. |
| [15] | Wei Du, Yu Sun, Hui-Min Bao, Liang Chen, Ying Li, Yan-Chun Liang. 基于迁移学习与深度学习的人类血液分泌蛋白预测框架[J]. 计算机科学技术学报, 2021, 36(2): 234-247. |
|
| 版权所有 © 《计算机科学技术学报》编辑部 本系统由北京玛格泰克科技发展有限公司设计开发 技术支持：support@magtech.com.cn 总访问量： |