Journal of Computer Science and Technology

   

Real-time Semantic Segmentation via An Efficient Multi-column Network

Cheng-Li Peng (彭程里) and Jia-Yi Ma* (马佳义), Member, CCF, Senior Member, IEEE   

  1. Electronic Information School, Wuhan University, Wuhan, 430072, China

Existing semantic segmentation networks based on the multi-column structure can hardly satisfy the efficiency and precision requirements simultaneously due to their shallow spatial branches. In this paper, we propose a new efficient multi-column network termed as LadderNet to address this problem. Our LadderNet includes two branches where the spatial branch generates high-resolution output feature map and the context branch encodes accurate semantic information. In particular, we first propose a channel attention fusion block and a global context module to enhance the information encoding ability of the context branch. Subsequently, a new branch fusion method, i.e., fusing some middle feature maps of context branch into the spatial branch, is developed to improve the depth of the spatial branch. Meanwhile, we design a feature fusing module to enhance the fusion quality of these two branches, leading to a more efficient network. We compare our model with other state-of-the-arts on PASCAL VOC 2012 and Cityscapes benchmarks. Experimental results demonstrate that, compared with other state-of-the-art methods, our LadderNet can achieve average 1.25% mIoU improvement with comparable or less computation.


中文摘要

图像语义分割是一个十分重要的高维计算机视觉任务。当前基于深度学习的方法在图像语义分割任务上取得了十分巨大的突破,国内外学者已提出了大量基于深度网络的图像语义分割方法。然而,当前主流的深度语义分割网络的网络结构较为低效,导致它们难以同时满足实际应用的速度和精度要求,即难以完成实时语义分割任务。其中,多分支网络由于其低效的空间分支存在语义信息捕获能力与信息捕获速度的矛盾,导致其难以实现实时语义分割。针对多分支网络在实时语义分割任务上的缺陷,本文提出一个更高效的多分支网络,在不引入过多计算量的情况下提升多分支网络中空间分支的语义信息捕获能力,从而提升网络的实时性。
目的
本文通过设计高效的结构来优化多分支网络的空间分支。当空间分支过于简单时,网络虽然速度快,但其语义信息捕获能力不足;当空间分支过于复杂时,网络虽然具有较强的信息捕获能力,但运行时间会显著增加。为解决这一矛盾,本文采用了比较简单的空间分支以确保网络速度,同时在空间分支中引入语义分支的信息以增大空间分支的感受野,从而提升空间分支语义信息捕获能力。最终,空间分支能在保持简单结构的同时具备较强的语义信息捕获能力,缓和了多分支网络速度和精度的矛盾。
方法
本文提出了一种新的多分支网络,所提网络包含一条可生成高分辨率输出的空间分支和一条可编码高精度语义信息的语义分支。具体来说,本文首先提出了一个通道注意力融合模块和全局上下文模块来提升语义分支的信息编码能力。在此基础上,本文通过融合空间分支和语义分支中间层特征的方式来提升空间分支的感受野,从而提升空间分支捕获语义信息的能力,同时本文还提出了一个有效的融合模块来提升不同分支的融合质量。
结果
本文在当前主流的图像语义分割数据集Pascal VOC 2012和Cityscapes上验证了所提方法的先进性。在Pascal VOC 2012的实验结果表明,相比于当前先进的方法,本文所提方法能在提升0.4%的mIoU的情况下,将计算量降低至原来的1/10。在Cityscapes数据集上的结果表明,相比于当前先进的方法,本文所提方法可在计算量相当的情况下,将mIoU提升1%。这说明本文所提方法在运算量和精度上都具备一定优势。
结论
实验结果证明本文所提的各个模块和网络设计方法均能有效提升网络性能。特别的,融合各分支不同层次特征图的方式可显著增大空间分支的有效感受野,提供了突破空间分支语义信息捕获瓶颈的可能性,这对后续多分支网络的设计具有一定的启发意义。但本文所提方法都是基于现有基准网络的,后续工作中可通过优化基准网络的方式进一步提升网络的实时性。总的来说,本文提出了一种新的多分支网络设计思路,有助于多分支网络实现实时语义分割任务。


Key words: semantic segmentation, real time, multi-column network, deep learning

;

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] Zhou Di;. A Recovery Technique for Distributed Communicating Process Systems[J]. , 1986, 1(2): 34 -43 .
[2] Feng Yulin;. Recursive Implementation of VLSI Circuits[J]. , 1986, 1(2): 72 -82 .
[3] Jin Lan; Yang Yuanyuan;. A Modified Version of Chordal Ring[J]. , 1986, 1(3): 15 -32 .
[4] Wang Jianchao; Wei Daozheng;. An Effective Test Generation Algorithm for Combinational Circuits[J]. , 1986, 1(4): 1 -16 .
[5] Min Yinghua; Han Zhide;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[6] Wu Yunzeng;. On the Development of Applications of Logic in Programming[J]. , 1987, 2(1): 30 -34 .
[7] Zhong Renbao; Xing Lin; Ren Zhaoyang;. An Interactive System SDI on Microcomputer[J]. , 1987, 2(1): 64 -71 .
[8] Xia Peisu; Fang Xinwo; Wang Yuxiang; Yan Kaiming; Zhang Tingjun; Liu Yulan; Zhao Chunying; Sun Jizhong;. Design of Array Processor Systems[J]. , 1987, 2(3): 163 -173 .
[9] S. T. Chanson; L. Liang; A. Kumar. Throughput Models of CSMA Network with Stations Uniformly Distributed along the Bus[J]. , 1987, 2(4): 243 -264 .
[10] Qi Yulu;. A Systolic Approach for an Improvement of a Finite Field Multiplier[J]. , 1987, 2(4): 303 -309 .

ISSN 1000-9000(Print)

         1860-4749(Online)
CN 11-2296/TP

Home
Editorial Board
Author Guidelines
Subscription
Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
Tel.:86-10-62610746
E-mail: jcst@ict.ac.cn
 
  Copyright ©2015 JCST, All Rights Reserved