PuzzleNet: 基于边界感知与特征匹配的无重叠三维点云拼接方法
PuzzleNet: Boundary-Aware Feature Matching for Non-Overlapping 3D Point Clouds Assembly
-
摘要:研究背景 三维形状拼接与装配是创建复杂三维场景的重要手段,也是计算机图形学研究的基本问题之一。基于无重叠点云的形状拼接是许多实际应用领域的基础,诸如三维模型设计、考古以及机器人等学科。当瓷器类物体被打碎成多块时,我们需要分别对这些碎片进行扫描,然后自动拼接成一个完整的三维数字化模型。然而如何将现有的部分重叠的三维形状拼接方法扩展到无重叠点云,仍然是一个开放性问题。目的 我们的研究目标是提出一种仅通过几何特征将多个完全无重叠的点云拼接为一个完整点云的方法。受拼图游戏的启发,点云之间的几何边界信息可以作为拼接两个无重叠点云的线索。方法 我们提出一种称为 PuzzleNet的人工神经网络模型,在该模型中,同时对拼接两个点云的空间变换矩阵和边界点进行学习。设计了一个基于注意力机制的点云特征编码器分别学习点云的全局形状特征和局部逐点特征,分别作为两个并行的解码器分支的输入,学习两个点云拼接的空间变换矩阵和边界点集合。与单任务网络相比,额外的边界点提取分支不仅可以作为微调变换矩阵的依据,进而得到更好的拼接结果,还可以用于设计迭代式贪心方法,通过点云之间边界点进行匹配将两个点云的拼接方法扩展成将多个点云拼接为一个完整形状的方法。结果 我们在真实城市场景的扫描数据集DublinCity和CAD合成数据集 ModelNet40上训练和测试了本文提出的PuzzleNet。实验证明该方法在解决任意几何形状和语义标签不一致的多部件3D 形状组装方面的有效性。与相关的先进形状匹配和三维点云配准方法相比,我们的方法具有更好的拼接效果。结论 我们通过引入名为PuzzleNet的深度神经网络,利用纯几何信息学习组装非重叠点云,进一步提升了3D形状装配的水平。PuzzleNet受拼图解决方案的启发,通过同时推断刚性变换和边界特征,学习准确的点云对齐,其中两个解码器被精心设计为在一个统一的网络中不受彼此影响。借助于边界特征的学习和匹配,多块组装可以通过迭代贪心算法实现。我们通过与最先进的方法在合成和实际扫描数据集上的比较,证明了我们方法的有效性和优势。Abstract: We address the 3D shape assembly of multiple geometric pieces without overlaps, a scenario often encountered in 3D shape design, field archeology, and robotics. Existing methods depend on strong assumptions on the number of shape pieces and coherent geometry or semantics of shape pieces. Despite raising attention to 3D registration with complex or low overlapping patterns, few methods consider shape assembly with rare overlaps. To address this problem, we present a novel framework inspired by solving puzzles, named PuzzleNet, which conducts multi-task learning by leveraging both 3D alignment and boundary information. Specifically, we design an end-to-end neural network based on a point cloud transformer with two-way branches for estimating rigid transformation and predicting boundaries simultaneously. The framework is then naturally extended to reassemble multiple pieces into a full shape by using an iterative greedy approach based on the distance between each pair of candidate-matched pieces. To train and evaluate PuzzleNet, we construct two datasets, named DublinPuzzle and ModelPuzzle, based on a real-world urban scan dataset (DublinCity) and a synthetic CAD dataset (ModelNet40) respectively. Experiments demonstrate our effectiveness in solving 3D shape assembly for multiple pieces with arbitrary geometry and inconsistent semantics. Our method surpasses state-of-the-art algorithms by more than 10 times in rotation metrics and four times in translation metrics.