计算机科学技术学报 ›› 2020,Vol. 35 ›› Issue (3): 493-505.doi: 10.1007/s11390-020-0476-4

所属专题: 综述 Artificial Intelligence and Pattern Recognition Computer Graphics and Multimedia

• Special Section of CVM 2020 • 上一篇    下一篇

车道检测-新结果和调查研究

梁盾, 郭元晨, 张少魁, 穆太江, 黄晓蕾   

  1. 1 Department of Computer Science and Technology, BNRist, Tsinghua University, Beijing 100084, China;
    2 College of Information Sciences and Technology, Pennsylvania State University, University Park, PA 16802, U.S.A.
  • 收稿日期:2020-03-28 修回日期:2020-04-17 出版日期:2020-05-28 发布日期:2020-05-28
  • 通讯作者: Tai-Jiang Mu E-mail:taijiang@tsinghua.edu.cn
  • 作者简介:Dun Liang is a Ph.D. candidate in the Department of Computer Science and Technology at Tsinghua University, Beijing, where he received his B.S. degree in computer science and technology, in 2016. His research interests include computer graphics, visual media and high-performance computing.
  • 基金资助:
    This work was supported by the National Natural Science Foundation of China under Grant Nos. 61902210 and 61521002, a research grant from the Beijing Higher Institution Engineering Research Center, and the Tsinghua-Tencent Joint Laboratory for Internet Innovation Technology.

Lane Detection: A Survey with New Results

Dun Liang, Yuan-Chen Guo, Shao-Kui Zhang, Tai-Jiang Mu, Xiaolei Huang        

  1. 1 Department of Computer Science and Technology, BNRist, Tsinghua University, Beijing 100084, China;
    2 College of Information Sciences and Technology, Pennsylvania State University, University Park, PA 16802, U.S.A.
  • Received:2020-03-28 Revised:2020-04-17 Online:2020-05-28 Published:2020-05-28
  • Contact: Tai-Jiang Mu E-mail:taijiang@tsinghua.edu.cn
  • About author:Dun Liang is a Ph.D. candidate in the Department of Computer Science and Technology at Tsinghua University, Beijing, where he received his B.S. degree in computer science and technology, in 2016. His research interests include computer graphics, visual media and high-performance computing.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China under Grant Nos. 61902210 and 61521002, a research grant from the Beijing Higher Institution Engineering Research Center, and the Tsinghua-Tencent Joint Laboratory for Internet Innovation Technology.

1、研究背景(context):
车道检测是自动驾驶的重要组成部分,也是智能辅助驾驶系统的基础。在过去的几年内,自动驾驶获得了高度关注和快速发展,智能辅助驾驶系统也日渐成熟并被广泛应用。车道检测算法也逐渐从传统的检测方法转变为基于学习的方法,并得到了很大的提升。与此同时,在复杂路况下,如道路拥堵、光照天气变化等,车道检测还存在很大的挑战。
2、目的(Objective):
本文旨在对基于学习的车道检测进行调研,总结和比较已有的最先进的车道检测算法,并整理相关数据集。为车道检测的工业应用,如受到广泛关注的自动驾驶和被广泛应用的智能辅助驾驶系统,提供参考,同时也方便车道检测的学术研究。与此同时,本文提出了自己的车道检测数据集和算法,可应用于复杂路况下全道路的车道检测和高精度地图建模
3、方法(Method):
本文对车道检测的已有的数据集和算法进行了综述,主要针对基于学习的车道检测算法,并按照不同应用场景将车道检测分为三类进行总结和分析:当前车道检测,当前方向道路车道检测,所有道路车道检测。
4、结果(Result & Findings):
本文调研了KITTI,ELAS,Caltech Lanes等8个数据集,并按照不同数据集的标注类型,道路边界和道路拥堵情况,进行了总结和比较,并将车道检测分为三类:当前车道检测,同向车道检测和全路检测。我们提出的TTLane数据集包含了道路拥堵的数据,为真实复杂场景的车道线算法提供测试场景。
与此同时,本文调研了上述三类车道检测的算法和相应的应用场景。其中当前车道检测算法分为单任务(基于VGG,FPN,CNN或LSTM等的网络)和多任务算法(RBNet等),主要应用于车道偏离预警系统等。当前方向道路车道检测算法主要分为端到端的网络(SpinNet,Enet等)和分割与后处理结合的分析流程,主要应用于车道转换等场景。所有道路的车道检测算法是复杂道路,如交叉路口和车辆转弯的自动驾驶所必需的,它需要对道路的高精度地图进行建模,已有的算法需要综合利用前景图和俯视图。
基于上述调研结果,本文提出了车道检测新的运用方向:高精度地图建模,并且将建模误差减小到了0.31m。
5、结论(Conclusions):
在本次调查研究中,我们回顾了车道检测的数据集和应用领域的发展,汇总了车道检测的深度学习方法,讨论了车道检测方法的趋势,并且发现通过引入带有详细标注的数据,将会提高车道检测在复杂路况下的表现。同时,我们提出了车道检测新的应用方向:高精度地图建模,利用众包数据,无需特殊设备,即可准确建模HD地图。
我们希望这种新的数据集和方法能够为未来用于智能自动驾驶的高精度地图建模研究做出贡献。

关键词: 卷积神经网络, 数据集, 深度学习, 高精度地图, 车道检测

Abstract: Lane detection is essential for many aspects of autonomous driving, such as lane-based navigation and highdefinition (HD) map modeling. Although lane detection is challenging especially with complex road conditions, considerable progress has been witnessed in this area in the past several years. In this survey, we review recent visual-based lane detection datasets and methods. For datasets, we categorize them by annotations, provide detailed descriptions for each category, and show comparisons among them. For methods, we focus on methods based on deep learning and organize them in terms of their detection targets. Moreover, we introduce a new dataset with more detailed annotations for HD map modeling, a new direction for lane detection that is applicable to autonomous driving in complex road conditions, a deep neural network LineNet for lane detection, and show its application to HD map modeling.

Key words: convolutional neural network (CNN), dataset, deep learning, high-definition (HD) map, lane detection

[1] Zhu Z, Liang D, Zhang S, Huang X, Li B, Hu S. Trafficsign detection and classification in the wild. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.2110-2118.
[2] Lu Y, Lu J, Zhang S, Hall P. Traffic signal detection and classification in street views using an attention model. Computational Visual Media, 2018, 4(3):253-266.
[3] Song Y, Fan R, Huang S, Zhu Z, Tong R. A three-stage real-time detector for traffic signs in large panoramas. Computational Visual Media, 2019, 5(4):403-416.
[4] Máttyus G, Wang S, Fidler S, Urtasun R. HD maps:Finegrained road segmentation by parsing ground and aerial images. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.3611-3619.
[5] Bittel S, Rehfeld T, Weber M, Zöllner J M. Estimating high definition map parameters with convolutional neural networks. In Proc. the 2017 IEEE International Conference on Systems, Man, and Cybernetics, October 2017, pp.52-56.
[6] Zang A, Xu R, Li Z, Doria D. Lane boundary extraction from satellite imagery. In Proc. the 1st ACM SIGSPATIAL Workshop on High-Precision Maps and Intelligent Applications for Autonomous Vehicles, November 2017, Article No. 1.
[7] Yuan J Z, Chen H, Zhao B, Xu Y. Estimation of vehicle pose and position with monocular camera at urban road intersections. Journal of Computer Science and Technology, 2017, 32(6):1150-1161.
[8] Yenikaya S, Yenikaya G, Düven E. Keeping the vehicle on the road:A survey on on-road lane detection systems. ACM Computing Surveys, 2013, 46(1):Article No. 2.
[9] Bar-Hillel A, Lerner R, Levi D, Raz G. Recent progress in road and lane detection:A survey. Machine Vision and Applications, 2014, 25(3):727-745.
[10] Fritsch J, Kühnl T, Geiger A. A new performance measure and evaluation benchmark for road detection algorithms. In Proc. the 16th International IEEE Conference on Intelligent Transportation Systems, October 2013, pp.1693-1700.
[11] Berriel R F, de Aguiar E, de Souza A F, Oliveira-Santos T. Ego-Lane Analysis System (ELAS):Dataset and algorithms. Image and Vision Computing, 2017, 68:64-75.
[12] Aly M. Real time detection of lane markers in urban streets. In Proc. the 2008 IEEE Intelligent Vehicles Symposium, June 2008, pp.7-12.
[13] Yu F, Chen H, Wang X, Xian W, Chen Y, Liu F, Madhavan V, Darrell T. BDD100K:A diverse driving dataset for heterogeneous multitask learning. In Proc. the IEEE Conference on Computer Vision and Pattern Recognition, 2020.(Accepted)
[14] Lee S, Kim J, Yoon J S, Shin S, Bailo O, Kim N, Lee T, Hong H S, Han S, Kweon I S. VPGNet:Vanishing point guided network for lane and road marking detection and recognition. In Proc. the 2017 IEEE International Conference on Computer Vision, October 2017, pp.1965-1973.
[15] Pan X, Shi J, Luo P, Wang X, Tang X. Spatial as deep:Spatial CNN for traffic scene understanding. In Proc. the 32nd AAAI Conference on Artificial Intelligence, February 2018, pp.7276-7283.
[16] Liang X, Wei Y, Shen X, Jie Z, Feng J, Lin L, Yan S. Reversible recursive instance-level object segmentation. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.633-641.
[17] Oliveira G L, Burgard W, Brox T. Efficient deep models for monocular road segmentation. In Proc. the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems, October 2016, pp.4885-4891.
[18] Liu X, Deng Z, Yang G. Drivable road detection based on dilated FPN with feature aggregation. In Proc. the 29th IEEE International Conference on Tools with Artificial Intelligence, November 2017, pp.1128-1134.
[19] Kim J, Park C. End-to-end ego lane estimation based on sequential transfer learning for self-driving cars. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, July 2017, pp.1194-1202.
[20] Chen Z, Chen Z. RBNet:A deep neural network for unified road and road boundary detection. In Proc. the 24th International Conference on Neural Information Processing, November 2017, pp.677-687.
[21] Teichmann M, Weber M, Zöllner J M, Cipolla R, Urtasun R. MultiNet:Real-time joint semantic reasoning for autonomous driving. In Proc. the 2018 IEEE Intelligent Vehicles Symposium, June 2018, pp.1013-1020.
[22] Lyu Y, Bai L, Huang X. Road segmentation using CNN and distributed LSTM. In Proc. the IEEE International Symposium on Circuits and Systems, May 2019.
[23] Mamidala R S, Uthkota U, Shankar M B, Antony A J, Narasimhadhan A V. Dynamic approach for lane detection using Google street view and CNN. In Proc. the 2019 IEEE Region 10 Conference, October 2019, pp.2454-2459.
[24] Li J, Mei X, Prokhorov D V, Tao D. Deep neural network for structural prediction and lane detection in traffic scene. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(3):690-703.
[25] Hou Y, Ma Z, Liu C, Loy C C. Learning lightweight lane detection CNNs by self attention distillation. In Proc. the 2019 IEEE/CVF International Conference on Computer Vision, October 2019, pp.1013-1021.
[26] Fan R, Wang X, Hou Q, Liu H, Mu T J. SpinNet:Spinning convolutional network for lane boundary detection. Computational Visual Media, 2019, 5(4):417-428.
[27] Pizzati F, Allodi M, Barrera A, García F. Lane detection and classification using cascaded CNNs. arXiv:1907.01294, 2019. https://arxiv.org/abs/1907.01294, March 2020.
[28] Philion J. FastDraw:Addressing the long tail of lane detection by adapting a sequential prediction network. In Proc. the IEEE Conference on Computer Vision and Pattern Recognition, June 2019, pp.11582-11591.
[29] De Brabandere B D, Neven D, Gool L V. Semantic instance segmentation with a discriminative loss function. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, July 2017.
[30] Neven D, De Brabandere B, Georgoulis S, Proesmans M, Gool L V. Towards end-to-end lane detection:An instance segmentation approach. In Proc. the 2018 IEEE Intelligent Vehicles Symposium, June 2018, pp.286-291.
[31] Hsu Y, Xu Z, Kira Z, Huang J. Learning to cluster for proposal-free instance segmentation. In Proc. the 2018 International Joint Conference on Neural Networks, July 2018.
[32] Chen P, Lo S, Hang H, Chan S, Lin J. Efficient road lane marking detection with deep learning. In Proc. the 23rd IEEE International Conference on Digital Signal Processing, November 2018.
[33] Chang D, Chirakkal V V, Goswami S, Hasan M, Jung T, Kang J, Kee S, Lee D, Singh A P. Multi-lane detection using instance segmentation and attentive voting. In Proc. the 19th International Conference on Control, Automation and Systems, October 2019, pp. 1538-1542.
[34] Paszke A, Chaurasia A, Kim S, Culurciello E. ENet:A deep neural network architecture for realtime semantic segmentation. arXiv:1606.02147, 2016. http://arxiv.org/abs/1606.02147, March 2020.
[35] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.770-778.
[36] Jung H, Min J, Kim J. An efficient lane detection algorithm for lane departure detection. In Proc. the 2013 IEEE Intelligent Vehicles Symposium, June 2013, pp.976-981.
[37] Beyeler M, Mirus F, Verl A. Vision-based robust road lane detection in urban environments. In Proc. the 2014 IEEE International Conference on Robotics and Automation, May 2014, pp.4920-4925.
[38] He B, Ai R, Yan Y, Lang X. Accurate and robust lane detection based on dual-view convolutional neutral network. In Proc. the 2016 IEEE Intelligent Vehicles Symposium, June 2016, pp.1041-1046.
[39] Azimi S M, Fischer P, Körner M, Reinartz P. Aerial laneNet:Lane-marking semantic segmentation in aerial imagery using wavelet-enhanced cost-sensitive symmetric fully convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(5):2920-2938.
[40] Kurz F, Azimi S M, Sheu C, d'Angelo P. Deep learning segmentation and 3D reconstruction of road markings using multiview aerial imagery. ISPRS International Journal of Geo-Information, 2019, 8(1):Article No. 47.
[41] Bai M, Máttyus G, Homayounfar N, Wang S, Lakshmikanth S K, Urtasun R. Deep multi-sensor lane detection. In Proc. the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, October 2018, pp.3102-3109.
[42] Garnett N, Cohen R, Pe'er T, Lahav R, Levi D. 3D-laneNet:End-to-end 3D multiple lane detection. In Proc. the 2019 IEEE/CVF International Conference on Computer Vision, October 2019, pp.2921-2930.
[43] Chen L, Papandreou G, Kokkinos I, Murphy K, Yuille A L. DeepLab:Semantic image segmentation with deep convolutional nets, Atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):834-848.
[44] Ester M, Kriegel H, Sander J, Xu X. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proc. the 2nd International Conference on Knowledge Discovery and Data Mining, August 1996, pp. 226-231.
[45] He K, Gkioxari G, Dollár P, Girshick R B. Mask R-CNN. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2):386-397.
[46] Hur J, Kang S, Seo S. Multi-lane detection in urban driving environments using conditional random fields. In Proc. the 2013 IEEE Intelligent Vehicles Symposium, June 2013, pp.1297-1302.
[47] Chen P, Sun H, Fang Y, Huai J. Collusion-proof result inference in crowdsourcing. Journal of Computer Science and Technology, 2018, 33(2):351-365.
[48] Zhang A, Li J, Gao H, Chen Y, Ma H, Bah M J. CrowdOLA:Online aggregation on duplicate data powered by crowdsourcing. Journal of Computer Science and Technology, 2018, 33(2):366-379.
[49] Mendiboure L, Chalouf M A, Krief F. Edge computing based applications in vehicular environments:Comparative study and main issues. Journal of Computer Science and Technology, 2019, 34(4):869-886.
[1] 魏华鹏, 邓盈盈, 唐帆, 潘兴甲, 董未名. 基于卷积神经网络和Transformer的视觉风格迁移的比较研究[J]. 计算机科学技术学报, 2022, 37(3): 601-614.
[2] 张鑫, 陆思源, 王水花, 余翔, 王甦菁, 姚仑, 潘毅, 张煜东. 通过新型深度学习架构诊断COVID-19肺炎[J]. 计算机科学技术学报, 2022, 37(2): 330-343.
[3] Shao-Jie Qiao, Guo-Ping Yang, Nan Han, Hao Chen, Fa-Liang Huang, Kun Yue, Yu-Gen Yi, Chang-An Yuan. 基数估计器:利用垂直扫描卷积神经网络处理SQL[J]. 计算机科学技术学报, 2021, 36(4): 762-777.
[4] Songjie Niu, Shimin Chen. TransGPerf:利用迁移学习建模分布式图计算性能[J]. 计算机科学技术学报, 2021, 36(4): 778-791.
[5] Chen-Chen Sun, De-Rong Shen. 面向深度实体匹配的混合层次网络[J]. 计算机科学技术学报, 2021, 36(4): 822-838.
[6] Lan Chen, Juntao Ye, Xiaopeng Zhang. 基于多特征超分网络的布料褶皱合成[J]. 计算机科学技术学报, 2021, 36(3): 478-493.
[7] Yang Liu, Ruili He, Xiaoqian Lv, Wei Wang, Xin Sun, Shengping Zhang. 婴儿的年龄和性别容易被识别吗?[J]. 计算机科学技术学报, 2021, 36(3): 508-519.
[8] Yu-Jie Yuan, Yukun Lai, Tong Wu, Lin Gao, Li-Gang Liu. 回顾形状编辑技术:从几何角度到神经网络方法[J]. 计算机科学技术学报, 2021, 36(3): 520-554.
[9] Sheng-Luan Hou, Xi-Kun Huang, Chao-Qun Fei, Shu-Han Zhang, Yang-Yang Li, Qi-Lin Sun, Chuan-Qing Wang. 基于深度学习的文本摘要研究综述[J]. 计算机科学技术学报, 2021, 36(3): 633-663.
[10] Wei Du, Yu Sun, Hui-Min Bao, Liang Chen, Ying Li, Yan-Chun Liang. 基于迁移学习与深度学习的人类血液分泌蛋白预测框架[J]. 计算机科学技术学报, 2021, 36(2): 234-247.
[11] Jun Gao, Paul Liu, Guang-Di Liu, Le Zhang. 基于深度学习与波束偏转的穿刺针定位与增强算法[J]. 计算机科学技术学报, 2021, 36(2): 334-346.
[12] Hua Chen, Juan Liu, Qing-Man Wen, Zhi-Qun Zuo, Jia-Sheng Liu, Jing Feng, Bao-Chuan Pang, Di Xiao. CytoBrain:基于深度学习技术的宫颈癌筛查系统[J]. 计算机科学技术学报, 2021, 36(2): 347-360.
[13] Zhang-Jin Huang, Xiang-Xiang He, Fang-Jun Wang, Qing Shen. 基于卷积神经网络的实时多阶段斑马鱼头部姿态估计框架[J]. 计算机科学技术学报, 2021, 36(2): 434-444.
[14] Andrea Caroppo, Alessandro Leone, Pietro Siciliano. 用于老年人面部表情识别的深度学习模型和传统机器学习方法的对比研究[J]. 计算机科学技术学报, 2020, 35(5): 1127-1146.
[15] Zheng Zeng, Lu Wang, Bei-Bei Wang, Chun-Meng Kang, Yan-Ning Xu. 一种基于多重残差网络的随机渐进式光子映射的降噪方法[J]. 计算机科学技术学报, 2020, 35(3): 506-521.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 李万学;. Almost Optimal Dynamic 2-3 Trees[J]. , 1986, 1(2): 60 -71 .
[2] C.Y.Chung; 华宣仁;. A Chinese Information Processing System[J]. , 1986, 1(2): 15 -24 .
[3] 章萃; 赵沁平; 徐家福;. Kernel Language KLND[J]. , 1986, 1(3): 65 -79 .
[4] 王建潮; 魏道政;. An Effective Test Generation Algorithm for Combinational Circuits[J]. , 1986, 1(4): 1 -16 .
[5] 闵应骅; 韩智德;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[6] 黄学东; 蔡莲红; 方棣棠; 迟边进; 周立; 蒋力;. A Computer System for Chinese Character Speech Input[J]. , 1986, 1(4): 75 -83 .
[7] 史忠植;. Knowledge-Based Decision Support System[J]. , 1987, 2(1): 22 -29 .
[8] 龚振和;. On Conceptual Model Specification and Verification[J]. , 1987, 2(1): 35 -50 .
[9] 唐同诰; 招兆铿;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[10] 夏培肃; 方信我; 王玉祥; 严开明; 张廷军; 刘玉兰; 赵春英; 孙继忠;. Design of Array Processor Systems[J]. , 1987, 2(3): 163 -173 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持:support@magtech.com.cn
总访问量: