计算机科学技术学报 ›› 2018,Vol. 33 ›› Issue (6): 1164-1177.doi: 10.1007/s11390-018-1879-3

所属专题: Computer Graphics and Multimedia

• Computer Graphics and Multimedia • 上一篇    下一篇

一种针对移动增强现实系统设计的基于几何信息的点云约减方法

Hao-Ren Wang, Juan Lei, Ao Li, Yi-Hong Wu*, Member, CCF   

  1. National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China University of Chinese Academy of Sciences, Beijing 100049, China
  • 收稿日期:2017-10-27 修回日期:2018-09-10 出版日期:2018-11-15 发布日期:2018-11-15
  • 通讯作者: Yi-Hong Wu,E-mail:yhwu@nlpr.ia.ac.cn E-mail:yhwu@nlpr.ia.ac.cn
  • 作者简介:Hao-Ren Wang is now a Ph.D. candidate in the Robert Vision Group of National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, and University of Chinese Academy of Sciences, Beijing. He received his B.S. degree in automation from Beijing Institute of Technology, Beijing, in 2004. He received his M.S. degree in pattern recognition from Chinese Aviation University of China, Tianjin, in 2010. His research focuses on computer vision, camera localization, and vision geometry.
  • 基金资助:
    This work was supported by the National Natural Science Foundation of China under Grant Nos. 61836015, 61572499, and 61421004.

A Geometry-Based Point Cloud Reduction Method for Mobile Augmented Reality System

Hao-Ren Wang, Juan Lei, Ao Li, Yi-Hong Wu*, Member, CCF   

  1. National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China University of Chinese Academy of Sciences, Beijing 100049, China
  • Received:2017-10-27 Revised:2018-09-10 Online:2018-11-15 Published:2018-11-15
  • Contact: Yi-Hong Wu,E-mail:yhwu@nlpr.ia.ac.cn E-mail:yhwu@nlpr.ia.ac.cn
  • About author:Hao-Ren Wang is now a Ph.D. candidate in the Robert Vision Group of National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing, and University of Chinese Academy of Sciences, Beijing. He received his B.S. degree in automation from Beijing Institute of Technology, Beijing, in 2004. He received his M.S. degree in pattern recognition from Chinese Aviation University of China, Tianjin, in 2010. His research focuses on computer vision, camera localization, and vision geometry.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China under Grant Nos. 61836015, 61572499, and 61421004.

本文中,我们提出了一个基于几何信息的点云约减算法,进而针对城市场景研究了一个实时的移动增强现实系统。我们通过结合空间点的重建误差和约束其空间几何分布的方式,构建了一个新的目标函数。在此目标函数基础上,用混合整数规划求解点云约减问题。而本文中研究的移动增强现实系统右离线和在线两部分组成。在离线阶段,我们用运动恢复结构的方法建立了定位数据库,还用前文提到的点云约减算法压缩了点云数据。而在在线阶段,我们采用结合基于图像的定位算法和连续姿态跟踪算法来实时地计算相机姿态。在基准数据和实际数据上的实验结果表明,与已有算法相比,本文提出的基于几何信息的点云约减方法选出来的点云子集可以使基于图像的定位方法有更高的定位成功率。在移动平台上的实验也表明提出的点云约减算法不仅减少了初始化和再初始化的时间,而且可以节省内存占用,还可用于一个可扩展的、实时的移动增强现实系统。

关键词: 移动平台, 增强现实, 点云约减, 运动恢复结构

Abstract: In this paper, a geometry-based point cloud reduction method is proposed, and a real-time mobile augmented reality system is explored for applications in urban environments. We formulate a new objective function which combines the point reconstruction errors and constraints on spatial point distribution. Based on this formulation, a mixed integer programming scheme is utilized to solve the points reduction problem. The mobile augmented reality system explored in this paper is composed of the offline and online stages. At the offline stage, we build up the localization database using structure from motion and compress the point cloud by the proposed point cloud reduction method. While at the online stage, we compute the camera pose in real time by combining an image-based localization algorithm and a continuous pose tracking algorithm. Experimental results on benchmark and real data show that compared with the existing methods, this geometry-based point cloud reduction method selects a point cloud subset which helps the image-based localization method to achieve higher success rate. Also, the experiments conducted on a mobile platform show that the reduced point cloud not only reduces the time consuming for initialization and re-initialization, but also makes the memory footprint small, resulting a scalable and real-time mobile augmented reality system.

Key words: mobile platform, augmented reality, point cloud reduction, structure from motion

[1] Agarwal S, Snavely N, Simon I, Seitz S M, Szeliski R. Building Rome in a day. In Proc. the 12th IEEE International Conference on Computer Vision, September 2009, pp.72-79.
[2] Arth C, Wagner D, Klopschitz M, Irschara A, Schmalstieg D. Wide area localization on mobile phones. In Proc. the 8th IEEE International Symposium on Mixed and Augmented Reality, October 2009, pp.73-82.
[3] Arth C, Mulloni A, Schmalstieg D. Exploiting sensors on mobile phones to improve wide-area localization. In Proc. the 21st International Conference on Pattern Recognition, November 2012, pp.2152-2156.
[4] Middelberg S, Sattler T, Untzelmann O, Kobbelt L. Scalable 6-DOF localization on mobile devices. In Proc. the 13th European Conference on Computer Vision, September 2014, pp.268-283.
[5] Jeong Y, Bok Y, Kim J S, Kweon I S. Complementation of cameras and lasers for accurate 6D SLAM:From correspondences to bundle adjustment. In Proc. the 2011 IEEE International Conference on Robotics and Automation, May 2011, pp.3581-3588.
[6] Miraldo P, Araujo H. Pose estimation for non-central cameras using planes. Journal of Intelligent & Robotic Systems, 2015, 80(3/4):595-608.
[7] Miraldo P, Araujo H, Gonçalves N. Pose estimation for general cameras using lines. IEEE Transactions on Cybernetics, 2015, 45(10):2156-2164.
[8] Liu H, Zhang G, Bao H. Robust keyframe-based monocular SLAM for augmented reality. In Proc. the 2016 IEEE International Symposium on Mixed and Augmented Reality, September 2016.
[9] Sattler T, Leibe B, Kobbelt L. Fast image-based localization using direct 2D-to-3D matching. In Proc. the 2011 IEEE International Conference on Computer Vision, November 2011, pp.667-674.
[10] Sattler T, Weyand T, Leibe B, Kobbelt L. Image retrieval for image-based localization revisited. In Proc. the 2012 British Machine Vision Conference, Sept. 2012.
[11] Choudhary S, Indelman V, Christensen H I, Dellaert F. Information-based reduced landmark SLAM. In Proc. the 2015 IEEE International Conference on Robotics and Automation, May 2015, pp.4620-4627.
[12] Torii A, Arandjelović R, Sivic J, Okutomi M, Pajdla T. 24/7 place recognition by view synthesis. In Proc. the 2015 IEEE Conference on Computer Vision and Pattern Recognition, June 2015, pp.1808-1817.
[13] Zeisl B, Sattler T, Pollefeys M. Camera pose voting for large-scale image-based localization. In Proc. the 2015 IEEE International Conference on Computer Vision, December 2015, pp.2704-2712.
[14] Feng Y, Fan L, Wu Y. Fast localization in large-scale environments using supervised indexing of binary features. IEEE Transactions on Image Processing, 2016, 25(1):343-358.
[15] Campbell D, Petersson L, Kneip L, Li H. Globally-optimal inlier set maximisation for simultaneous camera pose and feature correspondence. In Proc. the 2017 IEEE International Conference on Computer Vision, October 2017.
[16] Yang J, Li H, Campbell D, Jia Y. Go-ICP:A globally optimal solution to 3D ICP point-set registration. IEEE Transactions on Pattern Analysis and Machine Intelligence, November 2016, 38(11):2241-2254.
[17] Li Y, Snavely N, Huttenlocher D P. Location recognition using prioritized feature matching. In Proc. the 11th European Conference on Computer Vision, September 2010, pp.791-804.
[18] Irschara A, Zach C, Frahm J M, Bischof H. From structurefrom-motion point clouds to fast location recognition. In Proc. the 2009 IEEE Conference on Computer Vision and Pattern Recognition, June 2009, pp.2599-2606.
[19] Dong Z, Zhang G, Jia J, Bao H. Efficient keyframe-based real-time camera tracking. Computer Vision and Image Understanding, 2014, 118:97-110.
[20] Cao S, Snavely N. Minimal scene descriptions from structure from motion models. In Proc. the 2014 IEEE Conference on Computer Vision and Pattern Recognition, June 2014, pp.461-468.
[21] Park H S, Wang Y, Nurvitadhi E, Hoe J C, Sheikh Y, Chen M. 3D point cloud reduction using mixed-integer quadratic programming. In Proc. the 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops, June 2013, pp.229-236.
[22] Leutenegger S, Chli M, Siegwart R. BRISK:Binary robust invariant scalable keypoints. In Proc. the 2011 IEEE International Conference on Computer Vision, November 2011, pp.2548-2555.
[23] Lowe D G. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 2004, 60(2):91-110.
[24] Rosten E, Drummond T. Machine learning for high-speed corner detection. In Proc. the 9th European Conference on Computer Vision, May 2006, pp.430-443.
[25] Gionis A, Indyk P, Motwani R. Similarity search in high dimensions via hashing. In Proc. the 25th International Conference on Very Large Data Bases, September 1999, pp.518-529.
[26] Chum O, Matas J. Matching with PROSAC-Progressive sample consensus. In Proc. the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Volume 1, June 2005, pp.220-226.
[27] Lepetit V, Moreno-Noguer F, Fua P. EPnP:An accurate O(n) solution to the PnP problem. International Journal of Computer Vision, 2009, 81(2):155-166.
[28] Takacs G, Chandrasekhar V, Tsai S, Chen D, Grzeszczuk R, Girod B. Unified real-time tracking and recognition with rotation-invariant fast features. In Proc. the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, June 2010, pp.934-941.
[29] Stewart C V. Robust parameter estimation in computer vision. SIAM Review, 1999, 41(3):513-537.
[30] Hartley R, Zisserman A. Multiple View Geometry in Computer Vision (2nd edition). Cambridge University Press, 2004.
[31] Fischler M A, Bolles R C. Random sample consensus:A paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 1981, 24(6):381-395.
[1] Mo-Han Zhang, Jin-Hui Yu, Kang Zhang, Jun-Song Zhang. 基于水珠的照片艺术增强[J]. 计算机科学技术学报, 2019, 34(6): 1294-1306.
[2] Bin Liu, Kun Xu, Ralph R. Martin. 视频中静态场景光照估计及应用[J]. , 2017, 32(3): 430-442.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 冯玉琳;. Hierarchical Protocol Analysis by Temporal Logic[J]. , 1988, 3(1): 56 -69 .
[2] 张福炎; 蔡士杰; 王曙; 葛如顶;. The Human-Computer Dialogue Management of FCAD System[J]. , 1988, 3(3): 221 -227 .
[3] 郑崇勋; 张克农;. Orthogonal Algorithm of Logic Probability and Syndrome-Testable Analysis[J]. , 1990, 5(2): 203 -209 .
[4] 徐美瑞; 刘小林;. A VLSI Algorithm for Calculating the Tree to Tree Distance[J]. , 1993, 8(1): 68 -76 .
[5] 向东; 魏道政;. GLOBAL: A Design for Random Testability Algorithm[J]. , 1994, 9(2): 182 -192 .
[6] 谭建荣; 郑建民; 彭群生;. A Unified Algorithm for Finding the Intersection Curve of Surfaces[J]. , 1994, 9(2): 107 -116 .
[7] 廖先湜; 金兰;. Rendezvous Facilities in a Distributed Computer System[J]. , 1995, 10(2): 188 -192 .
[8] 高庆狮; 刘志勇;. K-Dimensional Optimal Parallel Algorithm for the Solution of a General Class of Recurrence Equations[J]. , 1995, 10(5): 417 -424 .
[9] 陶雪红; 孙伟; 马绍汉;. A Practical Propositional Knowledge Base Revision Algorithm[J]. , 1997, 12(2): 154 -159 .
[10] 张永越; 彭振云; 游素亚; 徐光佑;. A Multi-View Face Recognition System[J]. , 1997, 12(5): 400 -407 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持:support@magtech.com.cn
总访问量: