由流形约束实现人脸知识迁移的三维卡通重建方法
Manifold Constrained Transfer of Facial Geometric Knowledge for 3D Caricature Reconstruction
-
摘要: 三维卡通人脸是一种重要的资源,可以广泛应用于在线游戏、网络虚拟社区、卡通动画影视等领域。目前三维模型的制作可以基于专用软件,如MAYA,3DS MAX等,但这是一项非常繁琐的手工劳动,同时,三维卡通人脸创作要求制作者具有足够的艺术背景,以确保创作的质量效果。但漫画艺术家们在进行创作时,主要以二维的卡通为主,现实中存在的三维卡通人脸资源十分有限。目前已有一些研究工作对相关问题进行了研究以实现自动化生成三维卡通人脸,如采用PCA方法等。但是这些工作在选择统计模型时采用线性模型,对本文复杂的三维卡通人脸数据尤其是人脸侧面细节方面进行模拟时,尚存在明显的不足。本文提出一种自动化生成方法,其核心思想为:既然二维卡通人脸是三维卡通人脸在二维平面的投影,可以认为二者是同一种特征的不同“视图(View)”,同时又由于二者均具有显著的流形特征,则可以认为二者来源于同一潜在流形分布。基于这一思想,本文提出一种三维重建方法,基于一批真实的二维卡通人脸,重建相应的一批三维卡通人脸。首先,根据100个在MAYA环境下手工制作的三维卡通初始化全体三维数据集。其次,对二维卡通与三维卡通进行约束降维,使二者在某一低维流形嵌入上具有相同流形特征,即二者共享此流形。基于此流形进行流形升维计算,对三维卡通数据集进行重建,因为该流形已包含二维数据信息,因此对三维卡通数据集的重建是对数据集的进一步修正和知识迁移。然后再次对二维数据集与三维数据集进行约束流形降维计算和流形升维计算,进入迭代步骤,直至算法收敛。对比试验表明,该方法能够有效地重建三维卡通人脸,与主成分分析方法相比更能体现出原始二维卡通人脸的风格。Abstract: 3D caricatures are important attractive elements of the interface in virtual environment such as online game. However, very limited 3D caricatures exist in the real world. Meanwhile, creating 3D caricatures manually is rather costly, and even professional skills are needed. This paper proposes a novel and effective manifold transfer algorithm to reconstruct 3D caricatures according to their original 2D caricatures. We first manually create a small dataset with only 100 3D caricature models and use them to initialize the whole 3D dataset. After that, manifold transfer algorithm is carried out to refine the dataset. The algorithm comprises of two steps. The first is to perform manifold alignment between 2D and 3D caricatures to get a "standard" manifold map; the second is to reconstruct all the 3D caricatures based on the manifold map. The proposed approach utilizes and transfers knowledge of 2D caricatures to the target 3D caricatures well. Comparative experiments show that the approach reconstructs 3D caricatures more effectively and the results conform more to the styles of the original 2D caricatures than the Principal Components Analysis (PCA) based method.