Abstract Computer-aided diagnosis (CAD) technology can improve the efficiency of a physician diagnosing lung lesions, especially sign recognition, which is important for identifying benign and malignant nodules. This paper proposes a new sign recognition method based on image retrieval for lung nodules. First, we construct a deep learning framework to extract semantic features that can effectively represent sign information. Then, we translate the high-dimensional image features into compact binary codes with principal component analysis (PCA) and supervised hashing. Next, we retrieve similar lung nodule images with the presented adaptive-weighted similarity calculation method. Finally, we recognize nodule signs from the retrieval results, which can also provide decision support for diagnosis of lung lesions. The proposed method is validated on the publicly available databases Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) and Lung Computed Tomography (CT) Imaging Signs (LISS). The experimental results demonstrate our method substantially improves retrieval performance and can achieve 87.29%. The entire recognition rate on the basis of the retrieval results can achieve 93.52%. Moreover, our method is also effective for real-life diagnosis data.
This work was supported in part by the National Natural Science Foundation of China under Grant No. 61373100, the Virtual Reality Technology and Systems National Key Laboratory of Open Foundation of China under Grant No. BUAA-VR-16KF-13 and the Shanxi Scholarship Council of China under Grant No. 2016-038.
Corresponding Authors: 10.1007/s11390-017-1736-9
About author: Juan-Juan Zhao is a professor in the College of Computer Science and Technology at Taiyuan University of Technology (TYUT), Taiyuan. She received her Ph.D. degree in computer application technology from TYUT, Taiyuan, in 2010. Her current research topics include medical image processing and the Internet of Things.
Cite this article:
Juan-Juan Zhao, Ling Pan, Peng-Fei Zhao, Xiao-Xian Tang.Medical Sign Recognition of Lung Nodules based on Image Retrieval with Semantic Feature and Supervised Hashing[J] Journal of Computer Science and Technology, 2017,V32(3): 457-469
 Han H, Li L, Han F et al. Fast and adaptive detection of pulmonary nodules in thoracic CT images using a hierarchical vector quantization scheme. IEEE Journal of Biomedical & Health Informatics, 2014, 19(2): 648-659. Zhao J, Ji G, Qiang Y et al. A new method of detecting pulmonary nodules with PET/CT based on an improved watershed algorithm. PloS One, 2015, 10(4): e0123694. Tarando S R, Fetita C, Faccinetto A, Brillet P. Increasing CAD system efficacy for lung texture analysis using a convolutional network. In Proc. SPIE Medical Imaging, March 2016, pp.97850Q. Han G, Liu X, Han F et al. The LISS-A public database of common imaging signs of lung diseases for computeraided detection and diagnosis research and medical education. IEEE Trans. Biomedical Engineering, 2015, 62(2): 648-656. Dhara A K, Mukhopadhyay S, Saha P et al. Differential geometry-based techniques for characterization of boundary roughness of pulmonary nodules in CT images. International Journal of Computer Assisted Radiology & Surgery, 2015, 11(3): 337-349. Hu H, Wang Q, Tang H et al. Multi-slice computed tomography characteristics of solitary pulmonary ground-glass nodules: Differences between malignant and benign. Thoracic Cancer, 2016, 7(1): 80-87. Gao F, Ge X, Li M et al. CT features of lung scar cancer. Journal of Thoracic Disease, 2015, 7(3): 273-80. Ciompi F, Jacobs C, Scholten E T et al. Automatic detection of spiculation of pulmonary nodules in computed tomography images. In Proc. SPIE Medical Imaging, March 2015. Song L, Liu X, Ma L et al. Using HOG-LBP features and MMP learning to recognize imaging signs of lung lesions. In Proc. the 25th International Symposium on ComputerBased Medical Systems, June 2012. Ma L, Liu X, Li S et al. A new classifier fusion method based on historical and on-line classification reliability for recognizing common CT imaging signs of lung diseases. Computerized Medical Imaging & Graphics, 2015, 40: 39-48. Silpa-Anan C, Hartley R. Optimised KD-trees for fast image descriptor matching. In Proc. IEEE Conference on Computer Vision & Pattern Recognition. June 2008. Jégou H, Douze M, Schmid C. Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(1): 117-128. Gionis A, Indyk P, Motwani R. Similarity search in high dimensions via hashing. In Proc. the 25th International Conference on Very Large Data Bases, September 1999, pp.518-529. Kulis B, Grauman K. Kernelized locality-sensitive hashing for scalable image search. In Proc. the 12th IEEE International Conference on Computer Vision, Sept.27-Oct.4, 2009, pp.2130-2137. Weiss Y, Torralba A, Fergus R. Spectral hashing. In Proc. the 22nd Annual Conference on Neural Information Processing Systems, December 2008, pp.1753-1760. Gong Y, Lazebnik S, Gordo A et al. Iterative quantization: A procrustean approach to learning binary codes for largescale image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(12): 2916-2929. Wang J, Kumar S, Chang S F. Semi-supervised hashing for scalable image retrieval. In Proc. the 23rd IEEE Conference on Computer Vision and Pattern Recognition, June 2010, pp.3424-3431. Wang J, Kumar S, Chang S F. Sequential projection learning for hashing with compact codes. In Proc. the 27th International Conference on Machine Learning, June 2010, pp.1127-1134. Salakhutdinov R, Hinton G. Semantic hashing. International Journal of Approximate Reasoning, 2009, 50(7): 969- 978. Liu W, Wang J, Ji R et al. Supervised hashing with kernels. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2012, pp.2074-2081. Kulis B, Darrell T. Learning to hash with binary reconstructive embeddings. In Proc. the 23rd Annual Conference on Neural Information Processing Systems, December 2009, pp.1042-1050. Liu J, Zhang S, Liu W et al. Scalable mammogram retrieval using Anchor Graph Hashing. In Proc. the 11th IEEE International Symposium on Biomedical Imaging, April 29-May 2, 2014, pp.898-901. Zhang X, Yang L, Liu W et al. Mining histopathological images via composite hashing and online learning. In Proc. the 17th International Conference on Medical Image Computing & Computer-Assisted Intervention, September 2014, pp.479-486. Yu X, Zhang S, Liu B et al. Large scale medical image search via unsupervised PCA hashing. In Proc. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), June 2013, pp.393-398. Zhang X, Liu W, Dundar M et al. Towards large-scale histopathological image analysis: Hashing-based image retrieval. IEEE Transactions on Medical Imaging, 2015, 34(2): 496-506. Li W J, Wang S, Kang W C. Feature learning based deep supervised hashing with pairwise labels. arXiv:1511.03855. https://arxin.org/abs/1511.03855v1, Mar. 2017. Wei Y, Xia W, Huang J et al. CNN: Single-label to multi-label. arXiv:1406.5726, 2014. https://arxiv.org/abs-/1406.5726, Mar. 2017. Jiang Y G, Wang J, Chang S F. Lost in binarization: Queryadaptive ranking for similar image search with compact codes. In Proc. the 1st ACM International Conference on Multimedia Retrieval (ICMR), April 2011, p.16. Armato S G, Mclennan G, Bidaut L et al. The lung image database consortium (LIDC) and image database resource initiative (IDRI): A completed reference database of lung nodules on CT scans. Medical Physics, 2011, 38(2): 915- 931. Wang X J, Zhang L, Jing F et al. AmoSearch: Image autoamotation by search. In Proc. IEEE CVPR, June 2006, pp.1483-1490 Zhu Y, Tan Y, Hua Y et al. Feature selection and performance evaluation of support vector machine (SVM)- based classifier for differentiating benign and malignant pulmonary nodules by computed tomography. Journal of Digital Imaging, 2010, 23(1): 51-65. Keshani M, Azimifar Z, Tajeripour F et al. Lung nodule segmentation and recognition using SVM classifier and active contour modeling: A complete intelligent system. Computers in Biology and Medicine, 2013, 43(4): 287-300. Wang Q, Zhu W, Wang B. Three-dimensional SVM with latent variable: Application for detection of lung lesions in CT images. Journal of Medical Systems, 2015, 39(1): 171:1- 171:8. Jing Z, Li B, Tian L. Lung nodule classification combining rule-based and SVM. In Proc. the 5th International Conference on Bio-Inspired Computing: Theories and Applications (BIC-TA), September 2010, pp.1033-1036. Huang G B, Zhou H, Ding X et al. Extreme learning machine for regression and multiclass classification. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2012, 42(2): 513-529.