We use cookies to improve your experience with our site.

初始估计引导的非局部PatchMatch多视图立体重建

CNLPA-MVS: Coarse-Hypotheses Guided Non-Local PAtchMatch Multi-View Stereo

  • 摘要: 研究背景
    在多视图立体(Multi-View Stereo, MVS)重建中,由于弱纹理区域完整准确的深度信息难以估计,使得高质量和完整的三维模型获取仍然是一个挑战。对于弱纹理区域的深度估计,其主要困难是局部窗口下像素颜色相似而导致的匹配不确定。基于马尔可夫随机场(Markov Random Field, MRF)模型的方法通过采用非局部信息,可以减轻匹配的不确定性,从而提高重建完整度。但是,这种方法在连续的深度空间中具有较高的计算复杂度。近些年来,基于PatchMatch的多视图立体重建方法以其重建的高准确度和在连续空间中的高计算效率的优势逐渐成为多视图立体重建的主流方法。然而,由于原始PatchMatch方法计算过程中只存在一个数据项,在大块弱纹理区域中仍然无法充分考虑全局信息,难以解决弱纹理区域匹配不确定性的问题。
    目的
    本文旨在充分考虑非局部信息,设计一种基于PatchMatch的多视图立体重建方法,提高弱纹理区域的重建完整度。
    方法
    本文通过结合两种方法(基于MRF模型的非局部方法和基于PatchMatch MVS的方法)的优势,并对其劣势进行相互补偿,提出了一种新颖的重建方法,即初始估计引导的非局部PatchMatch的多视图立体重建(CNLPA-MVS)。首先,结合动态规划算法和沿扫描线顺序传播的思想,利用非局部信息实现并行计算,从而使每个像素获得最优的深度值和法向值。其次,本文引入低分辨率下赢者通吃的策略得到初始深度值和法向值,并将其作为备选最优值,以此来减轻动态规划导致的条纹现象,提高重建完整度。最后,本文基于相似颜色的像素具有相似深度的假设,将局部一致性策略结合到CNLPA-MVS中,进一步提高重建完整度。
    结果
    在公开数据集上的实验表明,即使在室内外强遮挡环境下,CNLPA-MVS得到的重建模型在弱纹理区域仍然具有高完整度。定量和定性对比结果表明,CNLPA-MVS可以在完整度和整体质量方面达到最优性能。
    结论
    本文提出了一种能够并行处理的非局部PatchMatch MVS方法,该方法可以有效缓解弱纹理区域的匹配不确定性问题。为了进一步提高弱纹理区域重建的完整度,本文同时考虑了初始估计引导和局部一致性策略。实验表明,本文方法能够重建高完整度三维模型,并且可以广泛应用于强遮挡下的弱纹理区域重建。此外,本文对基于MRF模型的全局方法的优化效率进行了提升,提供了一种新思路。未来工作中,我们将结合扩散式传播方法和全局优化方法,进一步提高计算效率和模型的重建质量。

     

    Abstract: In multi-view stereo, unreliable matching in low-textured regions has a negative impact on the completeness of reconstructed models. Since the photometric consistency of low-textured regions is not discriminative under a local window, non-local information provided by the Markov Random Field (MRF) model can alleviate the matching ambiguity but is limited in continuous space with high computational complexity. Owing to its sampling and propagation strategy, PatchMatch multi-view stereo methods have advantages in terms of optimizing the continuous labeling problem. In this paper, we propose a novel method to address this problem, namely the Coarse-Hypotheses Guided Non-Local PAtchMatch Multi-View Stereo (CNLPA-MVS), which takes the advantages of both MRF-based non-local methods and PatchMatch multi-view stereo and compensates for their defects mutually. First, we combine dynamic programing (DP) and sequential propagation along scanlines in parallel to perform CNLPA-MVS, thereby obtaining the optimal depth and normal hypotheses. Second, we introduce coarse inference within a universal window provided by winner-takes-all to eliminate the stripe artifacts caused by DP and improve completeness. Third, we add a local consistency strategy based on the hypotheses of similar color pixels sharing approximate values into CNLPA-MVS for further improving completeness. CNLPA-MVS was validated on public benchmarks and achieved state-of-the-art performance with high completeness.

     

/

返回文章
返回