Journal of Computer Science and Technology ›› 2020, Vol. 35 ›› Issue (3): 493-505.doi: 10.1007/s11390-020-0476-4

Special Issue: Surveys; Artificial Intelligence and Pattern Recognition; Computer Graphics and Multimedia

• Special Section of CVM 2020 • Previous Articles     Next Articles

Lane Detection: A Survey with New Results

Dun Liang, Yuan-Chen Guo, Shao-Kui Zhang, Tai-Jiang Mu, Xiaolei Huang        

  1. 1 Department of Computer Science and Technology, BNRist, Tsinghua University, Beijing 100084, China;
    2 College of Information Sciences and Technology, Pennsylvania State University, University Park, PA 16802, U.S.A.
  • Received:2020-03-28 Revised:2020-04-17 Online:2020-05-28 Published:2020-05-28
  • Contact: Tai-Jiang Mu E-mail:taijiang@tsinghua.edu.cn
  • About author:Dun Liang is a Ph.D. candidate in the Department of Computer Science and Technology at Tsinghua University, Beijing, where he received his B.S. degree in computer science and technology, in 2016. His research interests include computer graphics, visual media and high-performance computing.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China under Grant Nos. 61902210 and 61521002, a research grant from the Beijing Higher Institution Engineering Research Center, and the Tsinghua-Tencent Joint Laboratory for Internet Innovation Technology.

PDF (PC)

79

Lane detection is essential for many aspects of autonomous driving, such as lane-based navigation and highdefinition (HD) map modeling. Although lane detection is challenging especially with complex road conditions, considerable progress has been witnessed in this area in the past several years. In this survey, we review recent visual-based lane detection datasets and methods. For datasets, we categorize them by annotations, provide detailed descriptions for each category, and show comparisons among them. For methods, we focus on methods based on deep learning and organize them in terms of their detection targets. Moreover, we introduce a new dataset with more detailed annotations for HD map modeling, a new direction for lane detection that is applicable to autonomous driving in complex road conditions, a deep neural network LineNet for lane detection, and show its application to HD map modeling.

Key words: convolutional neural network (CNN); dataset; deep learning; high-definition (HD) map; lane detection;

[1] Zhu Z, Liang D, Zhang S, Huang X, Li B, Hu S. Trafficsign detection and classification in the wild. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.2110-2118.
[2] Lu Y, Lu J, Zhang S, Hall P. Traffic signal detection and classification in street views using an attention model. Computational Visual Media, 2018, 4(3):253-266.
[3] Song Y, Fan R, Huang S, Zhu Z, Tong R. A three-stage real-time detector for traffic signs in large panoramas. Computational Visual Media, 2019, 5(4):403-416.
[4] Máttyus G, Wang S, Fidler S, Urtasun R. HD maps:Finegrained road segmentation by parsing ground and aerial images. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.3611-3619.
[5] Bittel S, Rehfeld T, Weber M, Zöllner J M. Estimating high definition map parameters with convolutional neural networks. In Proc. the 2017 IEEE International Conference on Systems, Man, and Cybernetics, October 2017, pp.52-56.
[6] Zang A, Xu R, Li Z, Doria D. Lane boundary extraction from satellite imagery. In Proc. the 1st ACM SIGSPATIAL Workshop on High-Precision Maps and Intelligent Applications for Autonomous Vehicles, November 2017, Article No. 1.
[7] Yuan J Z, Chen H, Zhao B, Xu Y. Estimation of vehicle pose and position with monocular camera at urban road intersections. Journal of Computer Science and Technology, 2017, 32(6):1150-1161.
[8] Yenikaya S, Yenikaya G, Düven E. Keeping the vehicle on the road:A survey on on-road lane detection systems. ACM Computing Surveys, 2013, 46(1):Article No. 2.
[9] Bar-Hillel A, Lerner R, Levi D, Raz G. Recent progress in road and lane detection:A survey. Machine Vision and Applications, 2014, 25(3):727-745.
[10] Fritsch J, Kühnl T, Geiger A. A new performance measure and evaluation benchmark for road detection algorithms. In Proc. the 16th International IEEE Conference on Intelligent Transportation Systems, October 2013, pp.1693-1700.
[11] Berriel R F, de Aguiar E, de Souza A F, Oliveira-Santos T. Ego-Lane Analysis System (ELAS):Dataset and algorithms. Image and Vision Computing, 2017, 68:64-75.
[12] Aly M. Real time detection of lane markers in urban streets. In Proc. the 2008 IEEE Intelligent Vehicles Symposium, June 2008, pp.7-12.
[13] Yu F, Chen H, Wang X, Xian W, Chen Y, Liu F, Madhavan V, Darrell T. BDD100K:A diverse driving dataset for heterogeneous multitask learning. In Proc. the IEEE Conference on Computer Vision and Pattern Recognition, 2020.(Accepted)
[14] Lee S, Kim J, Yoon J S, Shin S, Bailo O, Kim N, Lee T, Hong H S, Han S, Kweon I S. VPGNet:Vanishing point guided network for lane and road marking detection and recognition. In Proc. the 2017 IEEE International Conference on Computer Vision, October 2017, pp.1965-1973.
[15] Pan X, Shi J, Luo P, Wang X, Tang X. Spatial as deep:Spatial CNN for traffic scene understanding. In Proc. the 32nd AAAI Conference on Artificial Intelligence, February 2018, pp.7276-7283.
[16] Liang X, Wei Y, Shen X, Jie Z, Feng J, Lin L, Yan S. Reversible recursive instance-level object segmentation. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.633-641.
[17] Oliveira G L, Burgard W, Brox T. Efficient deep models for monocular road segmentation. In Proc. the 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems, October 2016, pp.4885-4891.
[18] Liu X, Deng Z, Yang G. Drivable road detection based on dilated FPN with feature aggregation. In Proc. the 29th IEEE International Conference on Tools with Artificial Intelligence, November 2017, pp.1128-1134.
[19] Kim J, Park C. End-to-end ego lane estimation based on sequential transfer learning for self-driving cars. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, July 2017, pp.1194-1202.
[20] Chen Z, Chen Z. RBNet:A deep neural network for unified road and road boundary detection. In Proc. the 24th International Conference on Neural Information Processing, November 2017, pp.677-687.
[21] Teichmann M, Weber M, Zöllner J M, Cipolla R, Urtasun R. MultiNet:Real-time joint semantic reasoning for autonomous driving. In Proc. the 2018 IEEE Intelligent Vehicles Symposium, June 2018, pp.1013-1020.
[22] Lyu Y, Bai L, Huang X. Road segmentation using CNN and distributed LSTM. In Proc. the IEEE International Symposium on Circuits and Systems, May 2019.
[23] Mamidala R S, Uthkota U, Shankar M B, Antony A J, Narasimhadhan A V. Dynamic approach for lane detection using Google street view and CNN. In Proc. the 2019 IEEE Region 10 Conference, October 2019, pp.2454-2459.
[24] Li J, Mei X, Prokhorov D V, Tao D. Deep neural network for structural prediction and lane detection in traffic scene. IEEE Transactions on Neural Networks and Learning Systems, 2017, 28(3):690-703.
[25] Hou Y, Ma Z, Liu C, Loy C C. Learning lightweight lane detection CNNs by self attention distillation. In Proc. the 2019 IEEE/CVF International Conference on Computer Vision, October 2019, pp.1013-1021.
[26] Fan R, Wang X, Hou Q, Liu H, Mu T J. SpinNet:Spinning convolutional network for lane boundary detection. Computational Visual Media, 2019, 5(4):417-428.
[27] Pizzati F, Allodi M, Barrera A, García F. Lane detection and classification using cascaded CNNs. arXiv:1907.01294, 2019. https://arxiv.org/abs/1907.01294, March 2020.
[28] Philion J. FastDraw:Addressing the long tail of lane detection by adapting a sequential prediction network. In Proc. the IEEE Conference on Computer Vision and Pattern Recognition, June 2019, pp.11582-11591.
[29] De Brabandere B D, Neven D, Gool L V. Semantic instance segmentation with a discriminative loss function. In Proc. the 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops, July 2017.
[30] Neven D, De Brabandere B, Georgoulis S, Proesmans M, Gool L V. Towards end-to-end lane detection:An instance segmentation approach. In Proc. the 2018 IEEE Intelligent Vehicles Symposium, June 2018, pp.286-291.
[31] Hsu Y, Xu Z, Kira Z, Huang J. Learning to cluster for proposal-free instance segmentation. In Proc. the 2018 International Joint Conference on Neural Networks, July 2018.
[32] Chen P, Lo S, Hang H, Chan S, Lin J. Efficient road lane marking detection with deep learning. In Proc. the 23rd IEEE International Conference on Digital Signal Processing, November 2018.
[33] Chang D, Chirakkal V V, Goswami S, Hasan M, Jung T, Kang J, Kee S, Lee D, Singh A P. Multi-lane detection using instance segmentation and attentive voting. In Proc. the 19th International Conference on Control, Automation and Systems, October 2019, pp. 1538-1542.
[34] Paszke A, Chaurasia A, Kim S, Culurciello E. ENet:A deep neural network architecture for realtime semantic segmentation. arXiv:1606.02147, 2016. http://arxiv.org/abs/1606.02147, March 2020.
[35] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.770-778.
[36] Jung H, Min J, Kim J. An efficient lane detection algorithm for lane departure detection. In Proc. the 2013 IEEE Intelligent Vehicles Symposium, June 2013, pp.976-981.
[37] Beyeler M, Mirus F, Verl A. Vision-based robust road lane detection in urban environments. In Proc. the 2014 IEEE International Conference on Robotics and Automation, May 2014, pp.4920-4925.
[38] He B, Ai R, Yan Y, Lang X. Accurate and robust lane detection based on dual-view convolutional neutral network. In Proc. the 2016 IEEE Intelligent Vehicles Symposium, June 2016, pp.1041-1046.
[39] Azimi S M, Fischer P, Körner M, Reinartz P. Aerial laneNet:Lane-marking semantic segmentation in aerial imagery using wavelet-enhanced cost-sensitive symmetric fully convolutional neural networks. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(5):2920-2938.
[40] Kurz F, Azimi S M, Sheu C, d'Angelo P. Deep learning segmentation and 3D reconstruction of road markings using multiview aerial imagery. ISPRS International Journal of Geo-Information, 2019, 8(1):Article No. 47.
[41] Bai M, Máttyus G, Homayounfar N, Wang S, Lakshmikanth S K, Urtasun R. Deep multi-sensor lane detection. In Proc. the 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, October 2018, pp.3102-3109.
[42] Garnett N, Cohen R, Pe'er T, Lahav R, Levi D. 3D-laneNet:End-to-end 3D multiple lane detection. In Proc. the 2019 IEEE/CVF International Conference on Computer Vision, October 2019, pp.2921-2930.
[43] Chen L, Papandreou G, Kokkinos I, Murphy K, Yuille A L. DeepLab:Semantic image segmentation with deep convolutional nets, Atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):834-848.
[44] Ester M, Kriegel H, Sander J, Xu X. A density-based algorithm for discovering clusters in large spatial databases with noise. In Proc. the 2nd International Conference on Knowledge Discovery and Data Mining, August 1996, pp. 226-231.
[45] He K, Gkioxari G, Dollár P, Girshick R B. Mask R-CNN. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(2):386-397.
[46] Hur J, Kang S, Seo S. Multi-lane detection in urban driving environments using conditional random fields. In Proc. the 2013 IEEE Intelligent Vehicles Symposium, June 2013, pp.1297-1302.
[47] Chen P, Sun H, Fang Y, Huai J. Collusion-proof result inference in crowdsourcing. Journal of Computer Science and Technology, 2018, 33(2):351-365.
[48] Zhang A, Li J, Gao H, Chen Y, Ma H, Bah M J. CrowdOLA:Online aggregation on duplicate data powered by crowdsourcing. Journal of Computer Science and Technology, 2018, 33(2):366-379.
[49] Mendiboure L, Chalouf M A, Krief F. Edge computing based applications in vehicular environments:Comparative study and main issues. Journal of Computer Science and Technology, 2019, 34(4):869-886.
[1] Xin Zhang, Siyuan Lu, Shui-Hua Wang, Xiang Yu, Su-Jing Wang, Lun Yao, Yi Pan, and Yu-Dong Zhang. Diagnosis of COVID-19 Pneumonia via a Novel Deep Learning Architecture [J]. Journal of Computer Science and Technology, 2022, 37(2): 330-343.
[2] Songjie Niu, Shimin Chen. TransGPerf: Exploiting Transfer Learning for Modeling Distributed Graph Computation Performance [J]. Journal of Computer Science and Technology, 2021, 36(4): 778-791.
[3] Lan Chen, Juntao Ye, Xiaopeng Zhang. Multi-Feature Super-Resolution Network for Cloth Wrinkle Synthesis [J]. Journal of Computer Science and Technology, 2021, 36(3): 478-493.
[4] Yang Liu, Ruili He, Xiaoqian Lv, Wei Wang, Xin Sun, Shengping Zhang. Is It Easy to Recognize Baby's Age and Gender? [J]. Journal of Computer Science and Technology, 2021, 36(3): 508-519.
[5] Yu-Jie Yuan, Yukun Lai, Tong Wu, Lin Gao, Li-Gang Liu. A Revisit of Shape Editing Techniques: From the Geometric to the Neural Viewpoint [J]. Journal of Computer Science and Technology, 2021, 36(3): 520-554.
[6] Sheng-Luan Hou, Xi-Kun Huang, Chao-Qun Fei, Shu-Han Zhang, Yang-Yang Li, Qi-Lin Sun, Chuan-Qing Wang. A Survey of Text Summarization Approaches Based on Deep Learning [J]. Journal of Computer Science and Technology, 2021, 36(3): 633-663.
[7] Wei Du, Yu Sun, Hui-Min Bao, Liang Chen, Ying Li, Yan-Chun Liang. DeepHBSP: A Deep Learning Framework for Predicting Human Blood-Secretory Proteins Using Transfer Learning [J]. Journal of Computer Science and Technology, 2021, 36(2): 234-247.
[8] Jun Gao, Paul Liu, Guang-Di Liu, Le Zhang. Robust Needle Localization and Enhancement Algorithm for Ultrasound by Deep Learning and Beam Steering Methods [J]. Journal of Computer Science and Technology, 2021, 36(2): 334-346.
[9] Hua Chen, Juan Liu, Qing-Man Wen, Zhi-Qun Zuo, Jia-Sheng Liu, Jing Feng, Bao-Chuan Pang, Di Xiao. CytoBrain: Cervical Cancer Screening System Based on Deep Learning Technology [J]. Journal of Computer Science and Technology, 2021, 36(2): 347-360.
[10] Nuo Qun, Hang Yan, Xi-Peng Qiu, Xuan-Jing Huang. Chinese Word Segmentation via BiLSTM+Semi-CRF with Relay Node [J]. Journal of Computer Science and Technology, 2020, 35(5): 1115-1126.
[11] Andrea Caroppo, Alessandro Leone, Pietro Siciliano. Comparison Between Deep Learning Models and Traditional Machine Learning Approaches for Facial Expression Recognition in Ageing Adults [J]. Journal of Computer Science and Technology, 2020, 35(5): 1127-1146.
[12] Zheng Zeng, Lu Wang, Bei-Bei Wang, Chun-Meng Kang, Yan-Ning Xu. Denoising Stochastic Progressive Photon Mapping Renderings Using a Multi-Residual Network [J]. Journal of Computer Science and Technology, 2020, 35(3): 506-521.
[13] Shuai Li, Zheng Fang, Wen-Feng Song, Ai-Min Hao, Hong Qin. Bidirectional Optimization Coupled Lightweight Networks for Efficient and Robust Multi-Person 2D Pose Estimation [J]. Journal of Computer Science and Technology, 2019, 34(3): 522-536.
[14] Ri-Sheng Liu, Cai-Sheng Mao, Zhi-Hui Wang, Hao-Jie Li. Blind Image Deblurring via Adaptive Optimization with Flexible Sparse Structure Control [J]. Journal of Computer Science and Technology, 2019, 34(3): 609-621.
[15] Tai-Ling Yuan, Zhe Zhu, Kun Xu, Cheng-Jun Li, Tai-Jiang Mu, Shi-Min Hu. A Large Chinese Text Dataset in the Wild [J]. Journal of Computer Science and Technology, 2019, 34(3): 509-521.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] Li Wanxue;. Almost Optimal Dynamic 2-3 Trees[J]. , 1986, 1(2): 60 -71 .
[2] C.Y.Chung; H.R.Hwa;. A Chinese Information Processing System[J]. , 1986, 1(2): 15 -24 .
[3] Zhang Cui; Zhao Qinping; Xu Jiafu;. Kernel Language KLND[J]. , 1986, 1(3): 65 -79 .
[4] Wang Jianchao; Wei Daozheng;. An Effective Test Generation Algorithm for Combinational Circuits[J]. , 1986, 1(4): 1 -16 .
[5] Min Yinghua; Han Zhide;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[6] Huang Xuedong; Cai Lianhong; Fang Ditang; Chi Bianjin; Zhou Li; Jiang Li;. A Computer System for Chinese Character Speech Input[J]. , 1986, 1(4): 75 -83 .
[7] Shi Zhongzhi;. Knowledge-Based Decision Support System[J]. , 1987, 2(1): 22 -29 .
[8] Gong Zhenhe;. On Conceptual Model Specification and Verification[J]. , 1987, 2(1): 35 -50 .
[9] Tang Tonggao; Zhao Zhaokeng;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[10] Xia Peisu; Fang Xinwo; Wang Yuxiang; Yan Kaiming; Zhang Tingjun; Liu Yulan; Zhao Chunying; Sun Jizhong;. Design of Array Processor Systems[J]. , 1987, 2(3): 163 -173 .

ISSN 1000-9000(Print)

         1860-4749(Online)
CN 11-2296/TP

 Home
Editorial Board
Author Guidelines
Subscription		 Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
Tel.:86-10-62610746
E-mail: jcst@ict.ac.cn
 
  Copyright ©2015 JCST, All Rights Reserved