基于不确定性度量的自监督单目深度估计
Self-Supervised Monocular Depth Estimation by Digging into Uncertainty Quantification
-
摘要:研究背景 随着深度学习的快速发展,单张图像深度估计得到了大量的研究和应用。单张图像深度估计可应用于AR/VR,3D重构, 医学图像处理等方面。与有监督的单目深度估计相比,自监督的方法更容易获取训练数据,丰富的训练数据使得模型的泛化性强。为了提高深度估计模型的准确性,现有方法通过修改网络模块,增加损失函数,多任务学习指导深度估计网络学习。由于训练数据存在遮挡,运动物体,弱纹理区域等因素,单目深度估计在尖细物体、运动物体、物体边界、弱纹理区域存在深度估计不准确现象。目的 本篇论文的研究目标是通过设计一种不确定性度量策略,在训练网络模型时检测单张图像深度估计中不准确区域,引导网络学习这些区域的深度,得到一个高精度的深度估计网络模型。方法 本文的不确定性量化策略包含三个部分:不确定性检测,不确定性引导,和不确定性后处理,整个框架如图1所示。本文基于快照(Snapshot)和暹罗(Siam)的网络结构,探究深度图的不确定性区域并引导网络学习这些区域深度。Snapshot的结构是通过计算连续迭代周期(epoch)之间的模型方差估计不确定性区域。Siam的结构是通过计算双胞胎网络在同一个epoch的方差来度量不确定性。然后把不确定性约束添加到损失函数中,加强基线模型对场景中不确定性区域的学习。最后,本文提出了一种基于集成的不确定性后处理策略,对深度图进行修正,以提升最终深度估计的精度和鲁棒性。结果 通过大量的比较实验和消融实验验证本文方法的有效性。如表1所示,在深度图的七个量化指标上,本文的量化结果高于基线模型和现有的基于不确定性的方法。如表2所示,在两个不确定性的量化指标上,本文的结果也高于现有不确定性的方法。如图2所示,在深度图视觉效果比较上,本文的方法可以提高尖锐物体深度估计的准确性。在消融实验中,验证不确定性引导和不确定性后处理的有效性。在两个基线上的消融结果如表3和表4所示。从消融实验结果来看,本文的不确定性引导和不确定性后处理都能提升深度估计的准确性。结论 通过大量的实验验证了本文方法的有效性。基于我们提出的不确定性测量策略来训练单目深度估计模型,可以提高模型的准确性。本文提出的不确定性度量方法可以推广到其他深度学习工作中。Abstract: Based on well-designed network architectures and objective functions, self-supervised monocular depth estimation has made great progress. However, lacking a specific mechanism to make the network learn more about the regions containing moving objects or occlusion scenarios, existing depth estimation methods likely produce poor results for them. Therefore, we propose an uncertainty quantification method to improve the performance of existing depth estimation networks without changing their architectures. Our uncertainty quantification method consists of uncertainty measurement, the learning guidance by uncertainty, and the ultimate adaptive determination. Firstly, with Snapshot and Siam learning strategies, we measure the uncertainty degree by calculating the variance of pre-converged epochs or twins during training. Secondly, we use the uncertainty to guide the network to strengthen learning about those regions with more uncertainty. Finally, we use the uncertainty to adaptively produce the final depth estimation results with a balance of accuracy and robustness. To demonstrate the effectiveness of our uncertainty quantification method, we apply it to two state-of-the-art models, Monodepth2 and Hints. Experimental results show that our method has improved the depth estimation performance in seven evaluation metrics compared with two baseline models and exceeded the existing uncertainty method.