›› 2017, Vol. 32 ›› Issue (3): 457-469.doi: 10.1007/s11390-017-1736-9

Special Issue: Artificial Intelligence and Pattern Recognition; Computer Graphics and Multimedia

• Special Section of CVM 2017 • Previous Articles     Next Articles

Medical Sign Recognition of Lung Nodules based on Image Retrieval with Semantic Feature and Supervised Hashing

Juan-Juan Zhao1, Member, CCF, Ling Pan1, Peng-Fei Zhao1, Xiao-Xian Tang2   

  1. 1. College of Computer Science and Technology, Taiyuan University of Technology, Taiyuan 030024, China;
    2. Shanxi Provincial People's Hospital, Taiyuan 030012, China
  • Received:2016-12-27 Revised:2017-03-09 Online:2017-05-05 Published:2017-05-05
  • Contact: 10.1007/s11390-017-1736-9
  • About author:Juan-Juan Zhao is a professor in the College of Computer Science and Technology at Taiyuan University of Technology (TYUT), Taiyuan. She received her Ph.D. degree in computer application technology from TYUT, Taiyuan, in 2010. Her current research topics include medical image processing and the Internet of Things.
  • Supported by:

    This work was supported in part by the National Natural Science Foundation of China under Grant No. 61373100, the Virtual Reality Technology and Systems National Key Laboratory of Open Foundation of China under Grant No. BUAA-VR-16KF-13 and the Shanxi Scholarship Council of China under Grant No. 2016-038.

Computer-aided diagnosis (CAD) technology can improve the efficiency of a physician diagnosing lung lesions, especially sign recognition, which is important for identifying benign and malignant nodules. This paper proposes a new sign recognition method based on image retrieval for lung nodules. First, we construct a deep learning framework to extract semantic features that can effectively represent sign information. Then, we translate the high-dimensional image features into compact binary codes with principal component analysis (PCA) and supervised hashing. Next, we retrieve similar lung nodule images with the presented adaptive-weighted similarity calculation method. Finally, we recognize nodule signs from the retrieval results, which can also provide decision support for diagnosis of lung lesions. The proposed method is validated on the publicly available databases Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) and Lung Computed Tomography (CT) Imaging Signs (LISS). The experimental results demonstrate our method substantially improves retrieval performance and can achieve 87.29%. The entire recognition rate on the basis of the retrieval results can achieve 93.52%. Moreover, our method is also effective for real-life diagnosis data.

[1] Han H, Li L, Han F et al. Fast and adaptive detection of pulmonary nodules in thoracic CT images using a hierarchical vector quantization scheme. IEEE Journal of Biomedical & Health Informatics, 2014, 19(2): 648-659.

[2] Zhao J, Ji G, Qiang Y et al. A new method of detecting pulmonary nodules with PET/CT based on an improved watershed algorithm. PloS One, 2015, 10(4): e0123694.

[3] Tarando S R, Fetita C, Faccinetto A, Brillet P. Increasing CAD system efficacy for lung texture analysis using a convolutional network. In Proc. SPIE Medical Imaging, March 2016, pp.97850Q.

[4] Han G, Liu X, Han F et al. The LISS-A public database of common imaging signs of lung diseases for computeraided detection and diagnosis research and medical education. IEEE Trans. Biomedical Engineering, 2015, 62(2): 648-656.

[5] Dhara A K, Mukhopadhyay S, Saha P et al. Differential geometry-based techniques for characterization of boundary roughness of pulmonary nodules in CT images. International Journal of Computer Assisted Radiology & Surgery, 2015, 11(3): 337-349.

[6] Hu H, Wang Q, Tang H et al. Multi-slice computed tomography characteristics of solitary pulmonary ground-glass nodules: Differences between malignant and benign. Thoracic Cancer, 2016, 7(1): 80-87.

[7] Gao F, Ge X, Li M et al. CT features of lung scar cancer. Journal of Thoracic Disease, 2015, 7(3): 273-80.

[8] Ciompi F, Jacobs C, Scholten E T et al. Automatic detection of spiculation of pulmonary nodules in computed tomography images. In Proc. SPIE Medical Imaging, March 2015.

[9] Song L, Liu X, Ma L et al. Using HOG-LBP features and MMP learning to recognize imaging signs of lung lesions. In Proc. the 25th International Symposium on ComputerBased Medical Systems, June 2012.

[10] Ma L, Liu X, Li S et al. A new classifier fusion method based on historical and on-line classification reliability for recognizing common CT imaging signs of lung diseases. Computerized Medical Imaging & Graphics, 2015, 40: 39-48.

[11] Silpa-Anan C, Hartley R. Optimised KD-trees for fast image descriptor matching. In Proc. IEEE Conference on Computer Vision & Pattern Recognition. June 2008.

[12] Jégou H, Douze M, Schmid C. Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(1): 117-128.

[13] Gionis A, Indyk P, Motwani R. Similarity search in high dimensions via hashing. In Proc. the 25th International Conference on Very Large Data Bases, September 1999, pp.518-529.

[14] Kulis B, Grauman K. Kernelized locality-sensitive hashing for scalable image search. In Proc. the 12th IEEE International Conference on Computer Vision, Sept.27-Oct.4, 2009, pp.2130-2137.

[15] Weiss Y, Torralba A, Fergus R. Spectral hashing. In Proc. the 22nd Annual Conference on Neural Information Processing Systems, December 2008, pp.1753-1760.

[16] Gong Y, Lazebnik S, Gordo A et al. Iterative quantization: A procrustean approach to learning binary codes for largescale image retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(12): 2916-2929.

[17] Wang J, Kumar S, Chang S F. Semi-supervised hashing for scalable image retrieval. In Proc. the 23rd IEEE Conference on Computer Vision and Pattern Recognition, June 2010, pp.3424-3431.

[18] Wang J, Kumar S, Chang S F. Sequential projection learning for hashing with compact codes. In Proc. the 27th International Conference on Machine Learning, June 2010, pp.1127-1134.

[19] Salakhutdinov R, Hinton G. Semantic hashing. International Journal of Approximate Reasoning, 2009, 50(7): 969- 978.

[20] Liu W, Wang J, Ji R et al. Supervised hashing with kernels. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2012, pp.2074-2081.

[21] Kulis B, Darrell T. Learning to hash with binary reconstructive embeddings. In Proc. the 23rd Annual Conference on Neural Information Processing Systems, December 2009, pp.1042-1050.

[22] Liu J, Zhang S, Liu W et al. Scalable mammogram retrieval using Anchor Graph Hashing. In Proc. the 11th IEEE International Symposium on Biomedical Imaging, April 29-May 2, 2014, pp.898-901.

[23] Zhang X, Yang L, Liu W et al. Mining histopathological images via composite hashing and online learning. In Proc. the 17th International Conference on Medical Image Computing & Computer-Assisted Intervention, September 2014, pp.479-486.

[24] Yu X, Zhang S, Liu B et al. Large scale medical image search via unsupervised PCA hashing. In Proc. IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), June 2013, pp.393-398.

[25] Zhang X, Liu W, Dundar M et al. Towards large-scale histopathological image analysis: Hashing-based image retrieval. IEEE Transactions on Medical Imaging, 2015, 34(2): 496-506.

[26] Li W J, Wang S, Kang W C. Feature learning based deep supervised hashing with pairwise labels. arXiv:1511.03855. https://arxin.org/abs/1511.03855v1, Mar. 2017.

[27] Wei Y, Xia W, Huang J et al. CNN: Single-label to multi-label. arXiv:1406.5726, 2014. https://arxiv.org/abs-/1406.5726, Mar. 2017.

[28] Jiang Y G, Wang J, Chang S F. Lost in binarization: Queryadaptive ranking for similar image search with compact codes. In Proc. the 1st ACM International Conference on Multimedia Retrieval (ICMR), April 2011, p.16.

[29] Armato S G, Mclennan G, Bidaut L et al. The lung image database consortium (LIDC) and image database resource initiative (IDRI): A completed reference database of lung nodules on CT scans. Medical Physics, 2011, 38(2): 915- 931.

[30] Wang X J, Zhang L, Jing F et al. AmoSearch: Image autoamotation by search. In Proc. IEEE CVPR, June 2006, pp.1483-1490

[31] Zhu Y, Tan Y, Hua Y et al. Feature selection and performance evaluation of support vector machine (SVM)- based classifier for differentiating benign and malignant pulmonary nodules by computed tomography. Journal of Digital Imaging, 2010, 23(1): 51-65.

[32] Keshani M, Azimifar Z, Tajeripour F et al. Lung nodule segmentation and recognition using SVM classifier and active contour modeling: A complete intelligent system. Computers in Biology and Medicine, 2013, 43(4): 287-300.

[33] Wang Q, Zhu W, Wang B. Three-dimensional SVM with latent variable: Application for detection of lung lesions in CT images. Journal of Medical Systems, 2015, 39(1): 171:1- 171:8.

[34] Jing Z, Li B, Tian L. Lung nodule classification combining rule-based and SVM. In Proc. the 5th International Conference on Bio-Inspired Computing: Theories and Applications (BIC-TA), September 2010, pp.1033-1036.

[35] Huang G B, Zhou H, Ding X et al. Extreme learning machine for regression and multiclass classification. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 2012, 42(2): 513-529.
No related articles found!
Full text



[1] Liu Mingye; Hong Enyu;. Some Covering Problems and Their Solutions in Automatic Logic Synthesis Systems[J]. , 1986, 1(2): 83 -92 .
[2] Chen Shihua;. On the Structure of (Weak) Inverses of an (Weakly) Invertible Finite Automaton[J]. , 1986, 1(3): 92 -100 .
[3] Gao Qingshi; Zhang Xiang; Yang Shufan; Chen Shuqing;. Vector Computer 757[J]. , 1986, 1(3): 1 -14 .
[4] Chen Zhaoxiong; Gao Qingshi;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[5] Huang Heyan;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[6] Min Yinghua; Han Zhide;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[7] Tang Tonggao; Zhao Zhaokeng;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[8] Min Yinghua;. Easy Test Generation PLAs[J]. , 1987, 2(1): 72 -80 .
[9] Zhu Hong;. Some Mathematical Properties of the Functional Programming Language FP[J]. , 1987, 2(3): 202 -216 .
[10] Li Minghui;. CAD System of Microprogrammed Digital Systems[J]. , 1987, 2(3): 226 -235 .

ISSN 1000-9000(Print)

CN 11-2296/TP

Editorial Board
Author Guidelines
Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
E-mail: jcst@ict.ac.cn
  Copyright ©2015 JCST, All Rights Reserved