? 基于深度卷积网络的多重曝光运动估计
Journal of Computer Science and Technology
Quick Search in JCST
 Advanced Search 
      Home | PrePrint | SiteMap | Contact Us | Help
 
Indexed by   SCIE, EI ...
Bimonthly    Since 1986
Journal of Computer Science and Technology 2018, Vol. 33 Issue (3) :487-501    DOI: 10.1007/s11390-018-1833-4
Special Section of CVM 2018 << Previous Articles | Next Articles >>
基于深度卷积网络的多重曝光运动估计
Zhi-Feng Xie1,2, Yu-Chen Guo1, Shu-Han Zhang1, Wen-Jun Zhang1, Li-Zhuang Ma2,3, Member, CCF
1 Department of Film and Television Engineering, Shanghai University, Shanghai 200072, China;
2 Shanghai Engineering Research Center of Motion Picture Special Effects, Shanghai 200072, China;
3 Department of Software Science and Technology, East China Normal University, Shanghai 200062, China
Multi-exposure Motion Estimation based on Deep Convolutional Networks
Zhi-Feng Xie1,2, Yu-Chen Guo1, Shu-Han Zhang1, Wen-Jun Zhang1, Li-Zhuang Ma2,3, Member, CCF
1 Department of Film and Television Engineering, Shanghai University, Shanghai 200072, China;
2 Shanghai Engineering Research Center of Motion Picture Special Effects, Shanghai 200072, China;
3 Department of Software Science and Technology, East China Normal University, Shanghai 200062, China

摘要
参考文献
相关文章
Download: [PDF 5259KB]  
摘要 在运动估计中,光照变化一直是一个非常棘手的问题,经常导致光流计算质量的急剧下降。其关键原因在于绝大多数的估计方法不能对于各种各样的环境变化,在颜色或梯度域上形式化一个统一完整的定义。在这篇论文中,我们提出一种新的基于深度卷积网络的解决方案来解决这个问题。我们的思路是训练深度卷积网络来表示光照变化下的复杂运动特征,进而预测最终的光流场。为此,我们通过对传统光流计算训练集执行一系列的非线性调整,从而构建一个多重曝光图片对的训练集。我们端到端的网络模型包括三个主要组件:低层特征网络、混合特征网络和运动估计网络。前两者属于我们模型的收缩部分,主要是为抽取和表示多重曝光运动特征;第三个组件是我们模型的扩张部分,主要是为学习和预测高质量光流场。对比于许多传统的方法,我们基于深度卷积网络的运动估计方法能够消除光照变化的影响,并生成高精度高效率的光流结果。此外,我们的模型在一些多曝光视频应用中也有很出色的表现,例如HDR(高动态范围)合成和闪烁消除。
关键词运动估计   光流   卷积神经网络   多重曝光     
Abstract: In motion estimation, illumination change is always a troublesome obstacle, which often causes severely performance reduction of optical flow computation. The essential reason is that most of estimation methods fail to formalize a unified definition in color or gradient domain for diverse environmental changes. In this paper, we propose a new solution based on deep convolutional networks to solve the key issue. Our idea is to train deep convolutional networks to represent the complex motion features under illumination change, and further predict the final optical flow fields. To this end, we construct a training dataset of multi-exposure image pairs by performing a series of non-linear adjustments in the traditional datasets of optical flow estimation. Our end-to-end network model consists of three main components:low-level feature network, fusion feature network, and motion estimation network. The former two components belong to the contracting part of our model in order to extract and represent the multi-exposure motion features; the third component is the expanding part of our model in order to learn and predict the high-quality optical flow. Compared with many state-of-the-art methods, our motion estimation based on deep convolutional networks can eliminate the obstacle of illumination change and yield optical flow results with competitive accuracy and time efficiency. Moreover, the good performance of our model is also demonstrated in some multi-exposure video applications, like HDR (High Dynamic Range) composition and flicker removal.
Keywordsmotion estimation   optical flow   CNN   multi-exposure     
Received 2017-12-27;
本文基金:

This work was supported by the National Natural Science Foundation of China under Grant Nos. 61303093, 61472245, and 61402278, the Innovation Program of the Science and Technology Commission of Shanghai Municipality of China under Grant No. 16511101300, and the Gaofeng Film Discipline Grant of Shanghai Municipal Education Commission of China.

About author: Zhi-Feng Xie received his Ph.D. degree in computer application technology from Shanghai Jiao Tong University, Shanghai, in 2013. He was a research assistant at the Department of Computer Science, City University of Hong Kong, Hong Kong, in 2011. He is now an assistant professor with Shanghai University, Shanghai. His research interests include image/video editing, computer graphics, and digital media technology.
引用本文:   
Zhi-Feng Xie, Yu-Chen Guo, Shu-Han Zhang, Wen-Jun Zhang, Li-Zhuang Ma.基于深度卷积网络的多重曝光运动估计[J]  Journal of Computer Science and Technology , 2018,V33(3): 487-501
Zhi-Feng Xie, Yu-Chen Guo, Shu-Han Zhang, Wen-Jun Zhang, Li-Zhuang Ma.Multi-exposure Motion Estimation based on Deep Convolutional Networks[J]  Journal of Computer Science and Technology, 2018,V33(3): 487-501
链接本文:  
http://jcst.ict.ac.cn:8080/jcst/CN/10.1007/s11390-018-1833-4
Copyright 2010 by Journal of Computer Science and Technology