计算机科学技术学报 ›› 2019,Vol. 34 ›› Issue (3): 634-644.doi: 10.1007/s11390-019-1932-x

所属专题: Artificial Intelligence and Pattern Recognition Computer Graphics and Multimedia

• Artificial Intelligence and Pattern Recognition • 上一篇    下一篇

一种面向城市监控网络的渐进式车辆搜索系统

Xin-Chen Liu, Member, CCF, IEEE, Wu Liu, Member, CCF, IEEE Hua-Dong Ma*, Fellow, CCF, Senior Member, IEEE, Member, ACM, Shuang-Qun Li   

  1. Beijing Key Laboratory of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing 100876, China
  • 收稿日期:2018-10-20 修回日期:2019-03-22 出版日期:2019-05-05 发布日期:2019-05-06
  • 通讯作者: Hua-Dong Ma E-mail:mhd@bupt.edu.cn
  • 作者简介:Xin-Chen Liu received his Ph.D. degree at the Beijing Key Laboratory of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing, in 2018, and B.E. degree in computer science from Northwest Agricultural and Forestry University, Xi'an, in 2011. He received the Best Student Paper Award at ICME in 2016. His research interests include multimedia content analysis and computer vision.
  • 基金资助:
    This work was supported by the Funds for International Cooperation and Exchange of the National Natural Science Foundation of China (NSFC) under Grant No. 61720106007, the NSFC-Guangdong Joint Fund under Grant No. U1501254, the National Key Research and Development Plan of China under Grant No. 2016YFC0801005, the NFSC under Grant No. 61602049, and the 111 Project under Grant No. B18008.

PVSS: A Progressive Vehicle Search System for Video Surveillance Networks

Xin-Chen Liu, Member, CCF, IEEE, Wu Liu, Member, CCF, IEEE Hua-Dong Ma*, Fellow, CCF, Senior Member, IEEE, Member, ACM, Shuang-Qun Li   

  1. Beijing Key Laboratory of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing 100876, China
  • Received:2018-10-20 Revised:2019-03-22 Online:2019-05-05 Published:2019-05-06
  • Contact: Hua-Dong Ma E-mail:mhd@bupt.edu.cn
  • About author:Xin-Chen Liu received his Ph.D. degree at the Beijing Key Laboratory of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing, in 2018, and B.E. degree in computer science from Northwest Agricultural and Forestry University, Xi'an, in 2011. He received the Best Student Paper Award at ICME in 2016. His research interests include multimedia content analysis and computer vision.
  • Supported by:
    This work was supported by the Funds for International Cooperation and Exchange of the National Natural Science Foundation of China (NSFC) under Grant No. 61720106007, the NSFC-Guangdong Joint Fund under Grant No. U1501254, the National Key Research and Development Plan of China under Grant No. 2016YFC0801005, the NFSC under Grant No. 61602049, and the 111 Project under Grant No. B18008.

本文关注于车辆搜索,即搜索在监控网络出现的某一辆车辆的任务。现有的方法通常假定车辆图像从监控视频中完好地检测出来,然后使用视觉属性,如颜色和类型,或车牌号码匹配图像中目标车辆。然而,一个完整的车辆搜索系统应考虑的问题包括车辆检测、特征表示、索引、存储、匹配等等。此外,由于不同的相机角度和非常不确定的环境,基于属性的搜索通常不能准确找到相同的车辆。此外,车牌可能由于监控图像的低分辨率和噪音而被错误识别。本文中的渐进式车辆搜索系统,PVSS,旨在解决上述问题。PVSS由三个模块的构成:爬取器、索引器、搜索器。车辆爬取器用于在监控视频中检测和跟踪车辆,并将车辆的图像、元数据和上下文信息存储到服务器或云。然后车辆多粒度属性,比如视觉特征和车牌指纹,由索引器进行特征提取和构建索引。最后,一个查询三元组包含车辆图像、时间范围和空间范围,作为搜索者的输入。目标车辆将在数据库中进行渐进式地搜索。通过在真实监控数据集上的丰富实验表明了PVSS系统的有效性。

关键词: 多模态数据分析, 渐进式搜索系统, 车辆搜索, 视频监控网络

Abstract: This paper is focused on the task of searching for a specific vehicle that appears in the surveillance networks. Existing methods usually assume the vehicle images are well cropped from the surveillance videos, and then use visual attributes, like colors and types, or license plate numbers to match the target vehicle in the image set. However, a complete vehicle search system should consider the problems of vehicle detection, representation, indexing, storage, matching, and so on. Besides, it is very difficult for attribute-based search to accurately find the same vehicle due to intra-instance changes in different cameras and the extremely uncertain environment. Moreover, the license plates may be mis-recognized in surveillance scenes due to the low resolution and noise. In this paper, a progressive vehicle search system, named as PVSS, is designed to solve the above problems. PVSS is constituted of three modules:the crawler, the indexer, and the searcher. The vehicle crawler aims to detect and track vehicles in surveillance videos and transfer the captured vehicle images, metadata and contextual information to the server or cloud. Then multi-grained attributes, such as the visual features and license plate fingerprints, are extracted and indexed by the vehicle indexer. At last, a query triplet with an input vehicle image, the time range, and the spatial scope is taken as the input by the vehicle searcher. The target vehicle will be searched in the database by a progressive process. Extensive experiments on the public dataset from a real surveillance network validate the effectiveness of PVSS.

Key words: multi-modal data analysis, progressive search system, vehicle search, video surveillance network

[1] Ma H D. Internet of Things:Objectives and scientific challenges. Journal of Computer Science and Technology (JCST), 2011, 26(9):919-924.
[2] Brin S, Page L. The anatomy of a large-scale hypertextual Web search engine. Computer Networks, 1998, 30(1/2/3/4/5/6/7):107-117.
[3] Ma H D, Liu W. A progressive search paradigm for the Internet of things. IEEE MultiMedia, 2018, 25(1):76-86.
[4] Zhang J P, Wang F Y, Wang K F et al. Data-driven intelligent transportation systems:A survey. IEEE Trans. Intelligent Transportation Systems, 2011, 12(4):1624-1639.
[5] Yuan J Z, Chen H, Zhao B, Xu Y Y. Estimation of vehicle pose and position with monocular camera at urban road intersections. JCST, 2017, 32(6):1150-1161.
[6] Valera M, Velastin S A. Intelligent distributed surveillance systems:A review. IEE Proceedings-Vision, Image and Signal Processing, 2005, 152(2):192-204.
[7] Feris R, Siddiquie B, Zhai Y et al. Attribute-based vehicle search in crowded surveillance videos. In Proc. the 1st Int. Conf. Multimedia Retrieval, April 2011, Article No. 18.
[8] Feris R S, Siddiquie B, Petterson J et al. Large-scale vehicle detection, indexing, and search in urban surveillance videos. IEEE Transactions on Multimedia, 2012, 14(1):28-42.
[9] Wang Q, Ding Y D. A novel fine-grained method for vehicle type recognition based on the locally enhanced PCANet neural network. JCST, 2018, 33(2):335-350.
[10] Liu X C, Liu W, Ma H S, Fu H Y. Large-scale vehicle reidentification in urban surveillance videos. In Proc. the 2016 IEEE International Conference on Multimedia and Expo, July 2016, Article No. 145.
[11] Liu H, Tian Y, Yang Y, Pang L, Huang T. Deep relative distance learning:Tell the difference between similar vehicles. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2016, pp.2167- 2175.
[12] Liu X C, Liu W, Mei T, Ma H D. A deep learning based approach to progressive vehicle re-identification for urban surveillance. In Proc. the 14th European Conference on Computer Vision, October 2016, pp.869-884.
[13] Shen Y, Xiao T, Li H, Yi S, Wang X. Learning deep neural networks for vehicle re-ID with visual-spatio-temporal path proposals. In Proc. the 2017 IEEE International Conference on Computer Vision (ICCV), October 2017, pp.1918- 1927.
[14] Wang Z, Tang L, Liu X et al. Orientation invariant feature embedding and spatial temporal regularization for vehicle re-identification. In Proc. the 2017 ICCV, October 2017, pp.379-387.
[15] Liu X C, Liu W, Ma H D, Li S Q. A progressive vehicle search system for video surveillance networks. In Proc. the 4th IEEE Int. Conf. Multimedia Big Data, September 2018, Article No. 40.
[16] Sivic J, Zisserman A. Video Google:A text retrieval approach to object matching in videos. In Proc. the 9th ICCV, October 2003, pp.1470-1477.
[17] Lew M S, Sebe N, Djeraba C, Jain R. Content-based multimedia information retrieval:State of the art and challenges. ACM Transactions on Multimedia Computing, Communications, and Applications, 2006, 2(1):1-19.
[18] Hu W, Xie N, Li L et al. A survey on visual content-based video indexing and retrieval. IEEE Transactions on Systems, Man, and Cybernetics, Part C, 2011, 41(6):797-819.
[19] Mei T, Rui Y, Li S, Tian Q. Multimedia search reranking:A literature survey. ACM Computing Surveys, 2014, 46(3):Article No. 38.
[20] Liu W, Mei T, Zhang Y D. Instant mobile video search with layered audio-video indexing and progressive transmission. IEEE Transactions on Multimedia, 2014, 16(8):2242-2255.
[21] Zheng L, Yang Y, Tian Q. SIFT meets CNN:A decade survey of instance retrieval. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(5):1224-1244.
[22] Lin Y L, Tsai M K, Hsu W H, Chen C W. Investigating 3-D model and part information for improving content-based vehicle retrieval. IEEE Transactions on Circuits and Systems for Video Technology, 2013, 23(3):401-413.
[23] Farhadi A, Endres I, Hoiem D et al. Describing objects by their attributes. In Proc. the 2009 CVPR, June 2009, pp.1778-1785.
[24] Zheng L, Wang S, Liu Z, Tian Q. Fast image retrieval:Query pruning and early termination. IEEE Transactions on Multimedia, 2015, 17(5):648-659.
[25] Gong S, Cristani M, Yan S C, Loy C C. Person ReIdentification. Springer-Verlag London, 2014.
[26] Zheng L, Shen L, Tian L et al. Scalable person reidentification:A benchmark. In Proc. the 2015 ICCV, December 2015, pp.1116-1124.
[27] Zheng L, Yang Y, Hauptmann A G. Person reidentification:Past, present and future. arXiv:1610.02984, 2016. http://arxiv.org/abs/1610.02984, March 2019.
[28] Sun Y, Zheng L, Deng W, Wang S. SVDNet for pedestrian retrieval. In Proc. the 2017 ICCV, October 2017, pp.3820- 3828.
[29] Li W, Zhao R, Xiao T, Wang X. DeepReID:Deep filter pairing neural network for person re-identification. In Proc. the 2014 CVPR, June 2014, pp.152-159.
[30] Liao S, Hu Y, Zhu X, Li S Z. Person re-identification by local maximal occurrence representation and metric learning. In Proc. the 2015 CVPR, June 2015, pp.2197-2206.
[31] Zhang L, Xiang T, Gong S. Learning a discriminative null space for person re-identification. In Proc. the 2016 CVPR, June 2016, pp.1239-1248.
[32] Hermans A, Beyer L, Leibe B. In defense of the triplet loss for person re-identification. arXiv:1703.07737, 2017. http://arxiv.org/abs/1703.07737, March 2019.
[33] Feris R, Bobbitt R, Brown L, Pankanti S. Attribute-based people search:Lessons learnt from a practical surveillance system. In Proc. the 2014 Int. Conf. Multimedia Retrieval, April 2014, Article No. 153.
[34] Xu J, Jagadeesh V, Ni Z et al. Graph-based topic-focused retrieval in distributed camera network. IEEE Transactions on Multimedia, 2013, 15(8):2046-2057.
[35] Yang L, Luo P, Change Loy C, Tang X. A large-scale car dataset for fine-grained categorization and verification. In Proc. the 2015 CVPR, June 2015, pp.3973-3981.
[36] Liu X C, Ma H D, Fu H Y, Zhou M. Vehicle retrieval and trajectory inference in urban traffic surveillance scene. In Proc. the 2014 Int. Conf. Distributed Smart Cameras, November 2014, Article No. 26.
[37] Ren S, He K, Girshick R, Sun J. Faster R-CNN:Towards real-time object detection with region proposal networks. In Proc. the 2015 Annual Conference on Neural Information Processing Systems, December 2015, pp.91-99.
[38] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In Proc. the 2016 CVPR, June 2016, pp.770-778.
[39] Russakovsky O, Deng J, Su H et al. ImageNet large scale visual recognition challenge. International Journal of Computer Vision, 2015, 115(3):211-252.
[40] Chopra S, Hadsell R, LeCun Y. Learning a similarity metric discriminatively, with application to face verification. In Proc. the 2015 CVPR, June 2005, pp.539-546.
[41] Muja M, Lowe D G. Scalable nearest neighbor algorithms for high dimensional data. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(11):2227- 2240.
[42] Liu X C, Liu W, Mei T, Ma H D. PROVID:Progressive and multimodal vehicle reidentification for large-scale urban surveillance. IEEE Transactions on Multimedia, 2018, 20(3):645-658.
[43] Yan Y, Ni B, Song Z et al. Person re-identification via recurrent feature aggregation. In Proc. the 14th European Conference on Computer Vision, Part VI, October 2016, pp.701- 716.
[44] You J, Wu A, Li X, Zheng W S. Top-push video-based person re-identification. In Proc. the 2016 CVPR, June 2016, pp.1345-1353.
No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 孙钟秀; 商陆军;. DMODULA:A Distributed Programming Language[J]. , 1986, 1(2): 25 -31 .
[2] 周笛; 徐向文;. A Distributed Error Recovery Technique and Its Implementation and Application on UNIX[J]. , 1990, 5(2): 127 -138 .
[3] 韩建超; 史忠植;. Formalizing Default Reasoning[J]. , 1990, 5(4): 374 -378 .
[4] 徐洁盘; 王磊;. A New Approach to Database Auto-Design by Logic[J]. , 1991, 6(2): 201 -204 .
[5] Harald E. Otto;. UNDO, An Aid for Explorative Learning?[J]. , 1992, 7(3): 226 -236 .
[6] I.V.Vel bitsky; A.L.Kovalev; I.V.Kasatkina; 王镭;. R-Technology of Programming: Basic Notions and Implementation[J]. , 1992, 7(4): 345 -355 .
[7] 罗军舟; 顾冠群;. CIMS Network Protocol and Its Net Models[J]. , 1997, 12(5): 476 -481 .
[8] 陈阳军;. Counting and Topological Order[J]. , 1997, 12(6): 497 -509 .
[9] 迟学斌;. Parallel Implementation of Linear Algebra Problems on Dawning-1000[J]. , 1998, 13(2): 141 -146 .
[10] David de Frutos-Escrig; Luis Liana-Diaz; Manuel Nunez;. An invitation to Friendly Testing[J]. , 1998, 13(6): 531 -545 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持:support@magtech.com.cn
总访问量: