计算机科学技术学报 ›› 2019,Vol. 34 ›› Issue (5): 1123-1135.doi: 10.1007/s11390-019-1964-2

所属专题： Computer Graphics and Multimedia

• Computer Graphics and Multimedia • 上一篇    下一篇

DEMC:用于蒙特卡洛渲染去噪声的深度双编码器网络

Xin Yang1, Dawei Wang2, Wenbo Hu3, Li-Jing Zhao1, Bao-Cai Yin1, Qiang Zhang1, Xiao-Peng Wei1,*, Hongbo Fu4   

  1. 1 Department of Computer Science and Technology, Dalian University of Technology, Dalian 116024, China;
    2 Department of Computer Science, The University of Hong Kong, Pokfulam, Hong Kong, China;
    3 Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong, China;
    4 School of Creative Media, City University of Hong Kong, Kowloon, Hong Kong, China
  • 收稿日期:2019-01-15 修回日期:2019-05-28 出版日期:2019-08-31 发布日期:2019-08-31
  • 通讯作者: Xiao-Peng Wei E-mail:xpwei@dlut.edu.cn
  • 作者简介:Xin Yang is an associate professor in the Department of Computer Science and Technology at Dalian University of Technology, Dalian. Xin received his B.S. degree in computer science from Jilin University, Changchun, in 2007. From 2007 to June 2012, he was a joint Ph.D. student in Zhejiang University, Hangzhou, and Davis University of California for Graphics, and received his Ph.D. degree in computer science from Zhejiang University, Hangzhou, in 2012. His research interests include computer graphics and robotic vision.
  • 基金资助:
    This work was supported in part by the National Natural Science Foundation of China under Grant Nos. 91748104, U1811463, 61632006, 61425002, and 61751203, the National Key Research and Development Program of China under Grant No. 2018YFC0910506, the Open Project Program of the State Key Laboratory of CAD&CG of Zhejiang University of China under Grant No. A1901, and the Open Research Fund of Beijing Key Laboratory of Big Data Technology for Food Safety Project under Grant No. BTBD-2018KF.

DEMC: A Deep Dual-Encoder Network for Denoising Monte Carlo Rendering

Xin Yang1, Dawei Wang2, Wenbo Hu3, Li-Jing Zhao1, Bao-Cai Yin1, Qiang Zhang1, Xiao-Peng Wei1,*, Hongbo Fu4   

  1. 1 Department of Computer Science and Technology, Dalian University of Technology, Dalian 116024, China;
    2 Department of Computer Science, The University of Hong Kong, Pokfulam, Hong Kong, China;
    3 Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, Hong Kong, China;
    4 School of Creative Media, City University of Hong Kong, Kowloon, Hong Kong, China
  • Received:2019-01-15 Revised:2019-05-28 Online:2019-08-31 Published:2019-08-31
  • Contact: Xiao-Peng Wei E-mail:xpwei@dlut.edu.cn
  • About author:Xin Yang is an associate professor in the Department of Computer Science and Technology at Dalian University of Technology, Dalian. Xin received his B.S. degree in computer science from Jilin University, Changchun, in 2007. From 2007 to June 2012, he was a joint Ph.D. student in Zhejiang University, Hangzhou, and Davis University of California for Graphics, and received his Ph.D. degree in computer science from Zhejiang University, Hangzhou, in 2012. His research interests include computer graphics and robotic vision.
  • Supported by:
    This work was supported in part by the National Natural Science Foundation of China under Grant Nos. 91748104, U1811463, 61632006, 61425002, and 61751203, the National Key Research and Development Program of China under Grant No. 2018YFC0910506, the Open Project Program of the State Key Laboratory of CAD&CG of Zhejiang University of China under Grant No. A1901, and the Open Research Fund of Beijing Key Laboratory of Big Data Technology for Food Safety Project under Grant No. BTBD-2018KF.

PDF (PC)

156

补充材料

1

本文提出了一种双编码器网络（DEMC）用于去除蒙特卡洛渲染噪声并保留更多细节信息。去除蒙特卡洛渲染噪声和图片噪声的差别主要在于蒙特卡洛渲染去噪可以利用在渲染阶段低代价地提取的副产品（特征图）。这些特征图大多是无噪声的并且可以为图片的重建提供丰富的细节信息。但是，这些特征图也包含很多冗余信息。因此，蒙特卡洛渲染去噪主要的挑战在于如何从特征图中提取有效信息并更好的帮助重建。为了解决这一问题，我们提出了一种新颖的网络架构，双编码器网络和一个特征融合子网络，首先先将特征图融合并同时编码融合过的特征图和有噪声的图像，最终通过解码网络重建清晰无噪声的图片。和最先进的几种方法比较，我们的模型在更大范围场景测试中更加鲁邦，并且能够在很快的时间内产生令人满意的结果。

关键词: 蒙特卡洛渲染, 蒙特卡洛去噪, 神经网络

Abstract: In this paper, we present DEMC, a deep dual-encoder network to remove Monte Carlo noise efficiently while preserving details. Denoising Monte Carlo rendering is different from natural image denoising since inexpensive by-products (feature buffers) can be extracted in the rendering stage. Most of them are noise-free and can provide sufficient details for image reconstruction. However, these feature buffers also contain redundant information. Hence, the main challenge of this topic is how to extract useful information and reconstruct clean images. To address this problem, we propose a novel network structure, dual-encoder network with a feature fusion sub-network, to fuse feature buffers firstly, then encode the fused feature buffers and a noisy image simultaneously, and finally reconstruct a clean image by a decoder network. Compared with the state-of-the-art methods, our model is more robust on a wide range of scenes, and is able to generate satisfactory results in a significantly faster way.

Key words: Monte Carlo rendering, Monte Carlo denoising, neural network

[1] Kajiya J T. The rendering equation. In Proc. the 13th Annual Conference on Computer Graphics and Interactive Techniques, August 1986, pp.143-150.
[2] Rousselle F, Manzi M, Zwicker M. Robust denoising using feature and color information. Computer Graphics Forum, 2013, 32(7):121-130.
[3] Kalantari N K, Bako S, Sen P. A machine learning approach for filtering Monte Carlo noise. ACM Trans. Graph., 2015, 34(4):Article No. 122.
[4] Bitterli B, Rousselle F, Moon B, Guitián J A, Adler D, Mitchell K, Jarosz W, Novák J. Nonlinearly weighted first-order regression for denoising Monte Carlo renderings. Computer Graphics Forum, 2016, 35(4):107-117.
[5] Bako S, Vogels T, McWilliams B, Meyer M, Novák J, Harvill A, Sen P, Derose T, Rousselle F. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. ACM Trans. Graph., 2017, 36(4):Article No. 97.
[6] Moon B, Carr N, Yoon S E. Adaptive rendering based on weighted local regression. ACM Transactions on Graphics, 2017, 33(5):Article No. 170.
[7] Cook R L, Porter T, Carpenter L. Distributed ray tracing. In Proc. the 11th Annual Conference on Computer Graphics and Interactive Techniques, July 1984, pp.137-145.
[8] Zwicker M, Jarosz W, Lehtinen J, Moon B, Ramamoorthi R, Rousselle F, Sen P, Soler C, Yoon S E. Recent advances in adaptive sampling and reconstruction for Monte Carlo rendering. Computer Graphics Forum, 2015, 34(2):667-681.
[9] Ramamoorthi R, Mahajan D, Belhumeur P. A first-order analysis of lighting, shading, and shadows. ACM Transactions on Graphics, 2007, 26(1):Article No. 2.
[10] Jarosz W, Schönefeld V, Kobbelt L, Jensen H W. Theory, analysis and applications of 2D global illumination. ACM Transactions on Graphics, 2012, 31(5):Article No. 125.
[11] Bauszat P, Eisemann M, Magnor M. Guided image filtering for interactive high-quality global illumination. Computer Graphics Forum, 2011, 30(4):1361-1368.
[12] Bauszat P, Eisemann M, Eisemann E, Magnor M. General and robust error estimation and reconstruction for Monte Carlo rendering. Computer Graphics Forum, 2015, 34(2):597-608.
[13] Yang X, Wang D, Hu W, Zhao L, Piao X, Zhou D, Zhang Q, Yin B, Cai Q, Wei X. Fast reconstruction for Monte Carlo rendering using deep convolutional networks. IEEE Access, 2019, 7:21177-21187.
[14] Chaitanya C R A, Kaplanyan A S, Schied C, Salvi M, Lefohn A, Nowrouzezahrai D, Aila T. Interactive reconstruction of Monte Carlo image sequences using a recurrent denoising autoencoder. ACM Transactions on Graphics, 2017, 36(4):Article No. 98.
[15] Zhang K, Zuo W, Chen Y, Meng D, Zhang L. Beyond a Gaussian denoiser:Residual learning of deep CNN for image denoising. IEEE Transactions on Image Processing, 2017, 26(7):3142-3155.
[16] Gharbi M, Chaurasia G, Paris S, Durand F. Deep joint demosaicking and denoising. ACM Transactions on Graphics, 2016, 35(6):Article No. 191.
[17] Mao X, Shen C, Yang Y B. Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In Proc. the 2016 Annual Conference on Neural Information Processing Systems, December 2016, pp.2802-2810.
[18] Hinton G E, Salakhutdinov R R. Reducing the dimensionality of data with neural networks. Science, 2016, 313(5786):504-507.
[19] Balduzzi D, McWilliams B, Butler-Yeoman T. Neural Taylor approximations:Convergence and exploration in rectifier networks. In Proc. the 34th International Conference on Machine Learning, August 2017, pp.351-360.
[20] He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.770-778.
[21] Wang Z, Bovik A C, Sheikh H R, Simoncelli E P. Image quality assessment:From error visibility to structural similarity. IEEE Transactions on Image Processing, 2004, 13(4):600-612.
[22] Pharr M, Jakob W, Humphreys G. Physically Based Rendering:From Theory to Implementation (3rd edition). Morgan Kaufmann, 2016.
[23] Eilertsen G, Kronander J, Denes G, Mantiuk R K, Unger J. HDR image reconstruction from a single exposure using deep CNNs. ACM Transactions on Graphics, 2107, 36(6):Article No. 178.
[24] Rousselle F, Knaus C, Zwicker. Adaptive sampling and reconstruction using greedy error minimization. ACM Transactions on Graphics, 2011, 30(6):Article No. 159.
[25] Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks. In Proc. the 13th International Conference on Artificial Intelligence and Statistics, May 2010, pp.249-256
[26] Abadi M, Barham P, Chen J et al. Tensorflow:A system for large-scale machine learning. In Proc. the 12th USENIX Symposium on Operating Systems Design and Implementation, November 2016, pp.265-283.
[27] Kingma D P, Ba J. Adam:A method for stochastic optimization. arXiv:1412.6980, 2014. https://arxiv.org/abs/1412.6980, May 2019.
[28] Boughida M, Boubekeur T. Bayesian collaborative denoising for Monte Carlo rendering. Computer Graphics Forum, 2017, 36(4):137-153.
[29] Lang M, Wang O, Aydin T, Smolic A, Gross M. Practical temporal consistency for image-based graphics applications. ACM Transactions on Graphics, 2012, 31(4):Article No. 34.
[30] Bonneel N, Tompkin J, Sunkavalli K, Sun D, Paris S, Pfister H. Blind video temporal consistency. ACM Transactions on Graphics, 2015, 34(6):Article No. 196.
[31] Lai W S, Huang J B, Wang O, Shechtman E, Yumer E, Yang M H. Learning blind video temporal consistency. In Proc. the 15th European Conference on Computer Vision, September 2018, pp.179-185.
[1] 解晓政, 牛建伟, 刘雪峰, 李青锋, 王勇, 韩洁, 唐少杰. 基于卷积神经网络并融合边界信息的乳腺癌超声图像诊断[J]. 计算机科学技术学报, 2022, 37(2): 277-294.
[2] 王新峰、周翔、饶家华、张柱金、杨跃东. 基于迁移学习的DNA甲基化缺失数据补齐[J]. 计算机科学技术学报, 2022, 37(2): 320-329.
[3] 张鑫, 陆思源, 王水花, 余翔, 王甦菁, 姚仑, 潘毅, 张煜东. 通过新型深度学习架构诊断COVID-19肺炎[J]. 计算机科学技术学报, 2022, 37(2): 330-343.
[4] Dan-Hao Zhu, Xin-Yu Dai, Jia-Jun Chen. 预训练和学习：在图神经网络中保留全局信息[J]. 计算机科学技术学报, 2021, 36(6): 1420-1430.
[5] Yi Zhong, Jian-Hua Feng, Xiao-Xin Cui, Xiao-Le Cui. 机器学习辅助的抗逻辑块加密密钥猜测攻击范式[J]. 计算机科学技术学报, 2021, 36(5): 1102-1117.
[6] Feng Wang, Guo-Jie Luo, Guang-Yu Sun, Yu-Hao Wang, Di-Min Niu, Hong-Zhong Zheng. 在忆阻器中基于模式表示法的二值神经网络权重映射法[J]. 计算机科学技术学报, 2021, 36(5): 1155-1166.
[7] Shao-Jie Qiao, Guo-Ping Yang, Nan Han, Hao Chen, Fa-Liang Huang, Kun Yue, Yu-Gen Yi, Chang-An Yuan. 基数估计器：利用垂直扫描卷积神经网络处理SQL[J]. 计算机科学技术学报, 2021, 36(4): 762-777.
[8] Chen-Chen Sun, De-Rong Shen. 面向深度实体匹配的混合层次网络[J]. 计算机科学技术学报, 2021, 36(4): 822-838.
[9] Yang Liu, Ruili He, Xiaoqian Lv, Wei Wang, Xin Sun, Shengping Zhang. 婴儿的年龄和性别容易被识别吗？[J]. 计算机科学技术学报, 2021, 36(3): 508-519.
[10] Zhang-Jin Huang, Xiang-Xiang He, Fang-Jun Wang, Qing Shen. 基于卷积神经网络的实时多阶段斑马鱼头部姿态估计框架[J]. 计算机科学技术学报, 2021, 36(2): 434-444.
[11] Bo-Wei Zou, Rong-Tao Huang, Zeng-Zhuang Xu, Yu Hong, Guo-Dong Zhou. 基于对抗神经网络的跨语言实体关系分类[J]. 计算机科学技术学报, 2021, 36(1): 207-220.
[12] Wan-Wei Liu, Fu Song, Tang-Hao-Ran Zhang, Ji Wang. 基于模型检验的ReLU神经网络验证[J]. 计算机科学技术学报, 2020, 35(6): 1365-1381.
[13] Bi-Ying Yan, Chao Yang, Pan Deng, Qiao Sun, Feng Chen, Yang Yu. 一种基于时空因果性的城市感知数据治理方法[J]. 计算机科学技术学报, 2020, 35(5): 1084-1098.
[14] Yue-Huan Wang, Ze-Nan Li, Jing-Wei Xu, Ping Yu, Taolue Chen, Xiao-Xing Ma. 基于鲁棒性预测的深度神经网络质量保障[J]. 计算机科学技术学报, 2020, 35(5): 999-1015.
[15] 梁盾, 郭元晨, 张少魁, 穆太江, 黄晓蕾. 车道检测-新结果和调查研究[J]. 计算机科学技术学报, 2020, 35(3): 493-505.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] 刘明业; 洪恩宇;. Some Covering Problems and Their Solutions in Automatic Logic Synthesis Systems[J]. , 1986, 1(2): 83 -92 .
[2] 陈世华;. On the Structure of (Weak) Inverses of an (Weakly) Invertible Finite Automaton[J]. , 1986, 1(3): 92 -100 .
[3] 高庆狮; 张祥; 杨树范; 陈树清;. Vector Computer 757[J]. , 1986, 1(3): 1 -14 .
[4] 陈肇雄; 高庆狮;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[5] 黄河燕;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[6] 闵应骅; 韩智德;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[7] 唐同诰; 招兆铿;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[8] 闵应骅;. Easy Test Generation PLAs[J]. , 1987, 2(1): 72 -80 .
[9] 朱鸿;. Some Mathematical Properties of the Functional Programming Language FP[J]. , 1987, 2(3): 202 -216 .
[10] 李明慧;. CAD System of Microprogrammed Digital Systems[J]. , 1987, 2(3): 226 -235 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持：support@magtech.com.cn
总访问量：