We use cookies to improve your experience with our site.

基于深度卷积网络的多重曝光运动估计

Multi-exposure Motion Estimation based on Deep Convolutional Networks

  • 摘要: 在运动估计中,光照变化一直是一个非常棘手的问题,经常导致光流计算质量的急剧下降。其关键原因在于绝大多数的估计方法不能对于各种各样的环境变化,在颜色或梯度域上形式化一个统一完整的定义。在这篇论文中,我们提出一种新的基于深度卷积网络的解决方案来解决这个问题。我们的思路是训练深度卷积网络来表示光照变化下的复杂运动特征,进而预测最终的光流场。为此,我们通过对传统光流计算训练集执行一系列的非线性调整,从而构建一个多重曝光图片对的训练集。我们端到端的网络模型包括三个主要组件:低层特征网络、混合特征网络和运动估计网络。前两者属于我们模型的收缩部分,主要是为抽取和表示多重曝光运动特征;第三个组件是我们模型的扩张部分,主要是为学习和预测高质量光流场。对比于许多传统的方法,我们基于深度卷积网络的运动估计方法能够消除光照变化的影响,并生成高精度高效率的光流结果。此外,我们的模型在一些多曝光视频应用中也有很出色的表现,例如HDR(高动态范围)合成和闪烁消除。

     

    Abstract: In motion estimation, illumination change is always a troublesome obstacle, which often causes severely performance reduction of optical flow computation. The essential reason is that most of estimation methods fail to formalize a unified definition in color or gradient domain for diverse environmental changes. In this paper, we propose a new solution based on deep convolutional networks to solve the key issue. Our idea is to train deep convolutional networks to represent the complex motion features under illumination change, and further predict the final optical flow fields. To this end, we construct a training dataset of multi-exposure image pairs by performing a series of non-linear adjustments in the traditional datasets of optical flow estimation. Our end-to-end network model consists of three main components:low-level feature network, fusion feature network, and motion estimation network. The former two components belong to the contracting part of our model in order to extract and represent the multi-exposure motion features; the third component is the expanding part of our model in order to learn and predict the high-quality optical flow. Compared with many state-of-the-art methods, our motion estimation based on deep convolutional networks can eliminate the obstacle of illumination change and yield optical flow results with competitive accuracy and time efficiency. Moreover, the good performance of our model is also demonstrated in some multi-exposure video applications, like HDR (High Dynamic Range) composition and flicker removal.

     

/

返回文章
返回