Journal of Computer Science and Technology ›› 2019, Vol. 34 ›› Issue (3): 634-644.doi: 10.1007/s11390-019-1932-x

Special Issue: Artificial Intelligence and Pattern Recognition; Computer Graphics and Multimedia

• Artificial Intelligence and Pattern Recognition • Previous Articles     Next Articles

PVSS: A Progressive Vehicle Search System for Video Surveillance Networks

Xin-Chen Liu, Member, CCF, IEEE, Wu Liu, Member, CCF, IEEE Hua-Dong Ma*, Fellow, CCF, Senior Member, IEEE, Member, ACM, Shuang-Qun Li   

  1. Beijing Key Laboratory of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing 100876, China
  • Received:2018-10-20 Revised:2019-03-22 Online:2019-05-05 Published:2019-05-06
  • Contact: Hua-Dong Ma
  • About author:Xin-Chen Liu received his Ph.D. degree at the Beijing Key Laboratory of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing, in 2018, and B.E. degree in computer science from Northwest Agricultural and Forestry University, Xi'an, in 2011. He received the Best Student Paper Award at ICME in 2016. His research interests include multimedia content analysis and computer vision.
  • Supported by:
    This work was supported by the Funds for International Cooperation and Exchange of the National Natural Science Foundation of China (NSFC) under Grant No. 61720106007, the NSFC-Guangdong Joint Fund under Grant No. U1501254, the National Key Research and Development Plan of China under Grant No. 2016YFC0801005, the NFSC under Grant No. 61602049, and the 111 Project under Grant No. B18008.

This paper is focused on the task of searching for a specific vehicle that appears in the surveillance networks. Existing methods usually assume the vehicle images are well cropped from the surveillance videos, and then use visual attributes, like colors and types, or license plate numbers to match the target vehicle in the image set. However, a complete vehicle search system should consider the problems of vehicle detection, representation, indexing, storage, matching, and so on. Besides, it is very difficult for attribute-based search to accurately find the same vehicle due to intra-instance changes in different cameras and the extremely uncertain environment. Moreover, the license plates may be mis-recognized in surveillance scenes due to the low resolution and noise. In this paper, a progressive vehicle search system, named as PVSS, is designed to solve the above problems. PVSS is constituted of three modules:the crawler, the indexer, and the searcher. The vehicle crawler aims to detect and track vehicles in surveillance videos and transfer the captured vehicle images, metadata and contextual information to the server or cloud. Then multi-grained attributes, such as the visual features and license plate fingerprints, are extracted and indexed by the vehicle indexer. At last, a query triplet with an input vehicle image, the time range, and the spatial scope is taken as the input by the vehicle searcher. The target vehicle will be searched in the database by a progressive process. Extensive experiments on the public dataset from a real surveillance network validate the effectiveness of PVSS.

Key words: multi-modal data analysis; progressive search system; vehicle search; video surveillance network;

[1] Ma H D. Internet of Things:Objectives and scientific challenges. Journal of Computer Science and Technology (JCST), 2011, 26(9):919-924.
[2] Brin S, Page L. The anatomy of a large-scale hypertextual Web search engine. Computer Networks, 1998, 30(1/2/3/4/5/6/7):107-117.
[3] Ma H D, Liu W. A progressive search paradigm for the Internet of things. IEEE MultiMedia, 2018, 25(1):76-86.
[4] Zhang J P, Wang F Y, Wang K F et al. Data-driven intelligent transportation systems:A survey. IEEE Trans. Intelligent Transportation Systems, 2011, 12(4):1624-1639.
[5] Yuan J Z, Chen H, Zhao B, Xu Y Y. Estimation of vehicle pose and position with monocular camera at urban road intersections. JCST, 2017, 32(6):1150-1161.
[6] Valera M, Velastin S A. Intelligent distributed surveillance systems:A review. IEE Proceedings-Vision, Image and Signal Processing, 2005, 152(2):192-204.
[7] Feris R, Siddiquie B, Zhai Y et al. Attribute-based vehicle search in crowded surveillance videos. In Proc. the 1st Int. Conf. Multimedia Retrieval, April 2011, Article No. 18.
[8] Feris R S, Siddiquie B, Petterson J et al. Large-scale vehicle detection, indexing, and search in urban surveillance videos. IEEE Transactions on Multimedia, 2012, 14(1):28-42.
[9] Wang Q, Ding Y D. A novel fine-grained method for vehicle type recognition based on the locally enhanced PCANet neural network. JCST, 2018, 33(2):335-350.
[10] Liu X C, Liu W, Ma H S, Fu H Y. Large-scale vehicle reidentification in urban surveillance videos. In Proc. the 2016 IEEE International Conference on Multimedia and Expo, July 2016, Article No. 145.
[11] Liu H, Tian Y, Yang Y, Pang L, Huang T. Deep relative distance learning:Tell the difference between similar vehicles. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016, pp.2167- 2175.
[12] Liu X C, Liu W, Mei T, Ma H D. A deep learning based approach to progressive vehicle re-identification for urban surveillance. In Proc. the 14th European Conference on Computer Vision, October 2016, pp.869-884.
[13] Shen Y, Xiao T, Li H, Yi S, Wang X. Learning deep neural networks for vehicle re-ID with visual-spatio-temporal path proposals. In Proc. the 2017 IEEE International Conference on Computer Vision (ICCV), October 2017, pp.1918- 1927.
[14] Wang Z, Tang L, Liu X et al. Orientation invariant feature embedding and spatial temporal regularization for vehicle re-identification. In Proc. the 2017 ICCV, October 2017, pp.379-387.
[15] Liu X C, Liu W, Ma H D, Li S Q. A progressive vehicle search system for video surveillance networks. In Proc. the 4th IEEE Int. Conf. Multimedia Big Data, September 2018, Article No. 40.
[16] Sivic J, Zisserman A. Video Google:A text retrieval approach to object matching in videos. In Proc. the 9th ICCV, October 2003, pp.1470-1477.
[17] Lew M S, Sebe N, Djeraba C, Jain R. Content-based multimedia information retrieval:State of the art and challenges. ACM Transactions on Multimedia Computing, Communications, and Applications, 2006, 2(1):1-19.
[18] Hu W, Xie N, Li L et al. A survey on visual content-based video indexing and retrieval. IEEE Transactions on Systems, Man, and Cybernetics, Part C, 2011, 41(6):797-819.
[19] Mei T, Rui Y, Li S, Tian Q. Multimedia search reranking:A literature survey. ACM Computing Surveys, 2014, 46(3):Article No. 38.
[20] Liu W, Mei T, Zhang Y D. Instant mobile video search with layered audio-video indexing and progressive transmission. IEEE Transactions on Multimedia, 2014, 16(8):2242-2255.
[21] Zheng L, Yang Y, Tian Q. SIFT meets CNN:A decade survey of instance retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(5):1224-1244.
[22] Lin Y L, Tsai M K, Hsu W H, Chen C W. Investigating 3-D model and part information for improving content-based vehicle retrieval. IEEE Transactions on Circuits and Systems for Video Technology, 2013, 23(3):401-413.
[23] Farhadi A, Endres I, Hoiem D et al. Describing objects by their attributes. In Proc. the 2009 CVPR, June 2009, pp.1778-1785.
[24] Zheng L, Wang S, Liu Z, Tian Q. Fast image retrieval:Query pruning and early termination. IEEE Transactions on Multimedia, 2015, 17(5):648-659.
[25] Gong S, Cristani M, Yan S C, Loy C C. Person ReIdentification. Springer-Verlag London, 2014.
[26] Zheng L, Shen L, Tian L et al. Scalable person reidentification:A benchmark. In Proc. the 2015 ICCV, December 2015, pp.1116-1124.
[27] Zheng L, Yang Y, Hauptmann A G. Person reidentification:Past, present and future. arXiv:1610.02984, 2016., March 2019.
[28] Sun Y, Zheng L, Deng W, Wang S. SVDNet for pedestrian retrieval. In Proc. the 2017 ICCV, October 2017, pp.3820- 3828.
[29] Li W, Zhao R, Xiao T, Wang X. DeepReID:Deep filter pairing neural network for person re-identification. In Proc. the 2014 CVPR, June 2014, pp.152-159.
[30] Liao S, Hu Y, Zhu X, Li S Z. Person re-identification by local maximal occurrence representation and metric learning. In Proc. the 2015 CVPR, June 2015, pp.2197-2206.
[31] Zhang L, Xiang T, Gong S. Learning a discriminative null space for person re-identification. In Proc. the 2016 CVPR, June 2016, pp.1239-1248.
[32] Hermans A, Beyer L, Leibe B. In defense of the triplet loss for person re-identification. arXiv:1703.07737, 2017., March 2019.
[33] Feris R, Bobbitt R, Brown L, Pankanti S. Attribute-based people search:Lessons learnt from a practical surveillance system. In Proc. the 2014 Int. Conf. Multimedia Retrieval, April 2014, Article No. 153.
[34] Xu J, Jagadeesh V, Ni Z et al. Graph-based topic-focused retrieval in distributed camera network. IEEE Transactions on Multimedia, 2013, 15(8):2046-2057.
[35] Yang L, Luo P, Change Loy C, Tang X. A large-scale car dataset for fine-grained categorization and verification. In Proc. the 2015 CVPR, June 2015, pp.3973-3981.
[36] Liu X C, Ma H D, Fu H Y, Zhou M. Vehicle retrieval and trajectory inference in urban traffic surveillance scene. In Proc. the 2014 Int. Conf. Distributed Smart Cameras, November 2014, Article No. 26.
[37] Ren S, He K, Girshick R, Sun J. Faster R-CNN:Towards real-time object detection with region proposal networks. In Proc. the 2015 Annual Conference on Neural Information Processing Systems, December 2015, pp.91-99.
[38] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In Proc. the 2016 CVPR, June 2016, pp.770-778.
[39] Russakovsky O, Deng J, Su H et al. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 2015, 115(3):211-252.
[40] Chopra S, Hadsell R, LeCun Y. Learning a similarity metric discriminatively, with application to face verification. In Proc. the 2015 CVPR, June 2005, pp.539-546.
[41] Muja M, Lowe D G. Scalable nearest neighbor algorithms for high dimensional data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(11):2227- 2240.
[42] Liu X C, Liu W, Mei T, Ma H D. PROVID:Progressive and multimodal vehicle reidentification for large-scale urban surveillance. IEEE Transactions on Multimedia, 2018, 20(3):645-658.
[43] Yan Y, Ni B, Song Z et al. Person re-identification via recurrent feature aggregation. In Proc. the 14th European Conference on Computer Vision, Part VI, October 2016, pp.701- 716.
[44] You J, Wu A, Li X, Zheng W S. Top-push video-based person re-identification. In Proc. the 2016 CVPR, June 2016, pp.1345-1353.
No related articles found!
Full text



[1] Sun Zhongxiu; Shang Lujun;. DMODULA:A Distributed Programming Language[J]. , 1986, 1(2): 25 -31 .
[2] Zhou Di; Xu Xiangwen;. A Distributed Error Recovery Technique and Its Implementation and Application on UNIX[J]. , 1990, 5(2): 127 -138 .
[3] Han Jianchao; Shi Zhongzhi;. Formalizing Default Reasoning[J]. , 1990, 5(4): 374 -378 .
[4] Xu Jiepan; Wang Lei;. A New Approach to Database Auto-Design by Logic[J]. , 1991, 6(2): 201 -204 .
[5] Harald E. Otto;. UNDO, An Aid for Explorative Learning?[J]. , 1992, 7(3): 226 -236 .
[6] I.V.Vel bitsky; A.L.Kovalev; I.V.Kasatkina; Wang Lei;. R-Technology of Programming: Basic Notions and Implementation[J]. , 1992, 7(4): 345 -355 .
[7] Luo Junzhou; Gu Guanqun;. CIMS Network Protocol and Its Net Models[J]. , 1997, 12(5): 476 -481 .
[8] Chen Yangjun;. Counting and Topological Order[J]. , 1997, 12(6): 497 -509 .
[9] Chi Xuebin;. Parallel Implementation of Linear Algebra Problems on Dawning-1000[J]. , 1998, 13(2): 141 -146 .
[10] David de Frutos-Escrig; Luis Liana-Diaz; Manuel Nunez;. An invitation to Friendly Testing[J]. , 1998, 13(6): 531 -545 .

ISSN 1000-9000(Print)

CN 11-2296/TP

Editorial Board
Author Guidelines
Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
  Copyright ©2015 JCST, All Rights Reserved