Journal of Computer Science and Technology ›› 2020, Vol. 35 ›› Issue (3): 506-521.doi: 10.1007/s11390-020-0264-1

Special Issue: Artificial Intelligence and Pattern Recognition; Computer Graphics and Multimedia

• Special Section of CVM 2020 • Previous Articles     Next Articles

Denoising Stochastic Progressive Photon Mapping Renderings Using a Multi-Residual Network

Zheng Zeng1, Lu Wang1,*, Member, CCF, ACM, Bei-Bei Wang2,*, Member, CCF Chun-Meng Kang3, Member, CCF, IEEE, Yan-Ning Xu1, Member, CCF        

  1. 1 School of Software, Shandong University, Jinan 250101, China;
    2 School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094, China;
    3 School of Information Science and Engineering, Shandong Normal University, Jinan 250358, China
  • Received:2020-01-02 Revised:2020-03-24 Online:2020-05-28 Published:2020-05-28
  • Contact: Lu Wang, Bei-Bei Wang;
  • About author:Zheng Zeng received his B.S. degree in digital media technology from Shandong University, Jinan, in 2018. He is currently a Master student in the School of Software at Shandong University, Jinan, supervised by Prof. Lu Wang. His research interests include photorealistic rendering and high-performance rendering.
  • Supported by:
    This work was partially supported by the National Key Research and Development Program of China under Grant No. 2017YFB0203000, the National Natural Science Foundation of China under Grant Nos. 61802187, 61872223, and 61702311, and the Natural Science Foundation of Jiangsu Province of China under Grant No. BK20170857.

Stochastic progressive photon mapping (SPPM) is one of the important global illumination methods in computer graphics. It can simulate caustics and specular-diffuse-specular lighting effects efficiently. However, as a biased method, it always suffers from both bias and variance with limited iterations, and the bias and the variance bring multi-scale noises into SPPM renderings. Recent learning-based methods have shown great advantages on denoising unbiased Monte Carlo (MC) methods, but have not been leveraged for biased ones. In this paper, we present the first learning-based method specially designed for denoising-biased SPPM renderings. Firstly, to avoid conflicting denoising constraints, the radiance of final images is decomposed into two components: caustic and global. These two components are then denoised separately via a two-network framework. In each network, we employ a novel multi-residual block with two sizes of filters, which significantly improves the model’s capabilities, and makes it more suitable for multi-scale noises on both low-frequency and high-frequency areas. We also present a series of photon-related auxiliary features, to better handle noises while preserving illumination details, especially caustics. Compared with other state-of-the-art learning-based denoising methods that we apply to this problem, our method shows a higher denoising quality, which could efficiently denoise multi-scale noises while keeping sharp illuminations.

Key words: denoising; stochastic progressive photon mapping (SPPM); deep learning; residual neural network;

[1] Hachisuka T, Jensen H W. Stochastic progressive photon mapping. ACM Transactions on Graphics, 2009, 28(5):Article No. 141.
[2] Jensen H W. Realistic Image Synthesis Using Photon Mapping (1st edition). Routledge, 2001.
[3] Hachisuka T, Ogaki S, Jensen H W. Progressive photon mapping. ACM Transactions on Graphics, 2008, 27(5):Article No. 130.
[4] Kang C, Wang L, Xu Y et al. A survey of photon mapping state-of-the-art research and future challenges. Frontiers of Information Technology&Electronic Engineering, 2016, 17(3):185-199.
[5] Ritschel T, Dachsbacher C, Grosch T et al. The state of the art in interactive global illumination. Computer Graphics Forum, 2012, 31(1):160-188.
[6] Bako S, Vogels T, McWilliams B et al. Kernel-predicting convolutional networks for denoising Monte Carlo renderings. ACM Transactions on Graphics, 2017, 36(4):Article No. 97.
[7] Vogels T, Rousselle F, McWilliams B et al. Denoising with kernel prediction and asymmetric loss functions. ACM Transactions on Graphics, 2018, 37(4):Article No. 124.
[8] Wong K M, Wong T T. Robust deep residual denoising for Monte Carlo rendering. In Proc. the 2018 SIGGRAPH Asia Technical Briefs, December 2018, Article No. 14.
[9] Wong K M, Wong T T. Deep residual learning for denoising Monte Carlo renderings. Computational Visual Media, 2019, 5(3):239-255.
[10] Kalantari N K, Bako S, Sen P. A machine learning approach for filtering Monte Carlo noise. ACM Transactions on Graphics, 2015, 34(4):Article No. 122.
[11] Xu B, Zhang J, Wang R et al. Adversarial Monte Carlo denoising with conditioned auxiliary feature modulation. ACM Transactions on Graphics, 2019, 38(6):Article No. 224.
[12] Yang X, Wang D, Hu W et al. DEMC:A deep dual-encoder network for denoising Monte Carlo rendering. Journal of Computer Science and Technology, 2019, 34(5):1123-1135.
[13] Gharbi M, Li T M, Aittala M et al. Sample-based Monte Carlo denoising using a kernel-splatting network. ACM Transactions on Graphics, 2019, 38(4):Article No. 125.
[14] Günther T, Grosch T. Distributed out-of-core stochastic progressive photon mapping. Computer Graphics Forum, 2014, 33(6):154-166.
[15] Havran V, Bittner J, Herzog R et al. Ray maps for global illumination. Rendering Techniques, 2005, 2005:43-54.
[16] Frisvad J R, Schjøth L, Erleben K et al. Photon differential splatting for rendering caustics. Computer Graphics Forum, 2014, 33(6):252-263.
[17] Spencer B, Jones M W. Into the blue:Better caustics through photon relaxation. Computer Graphics Forum, 2009, 28(2):319-328.
[18] Fu Z, Jensen H W. Noise reduction for progressive photon mapping. In Proc. the International Conference on Computer Graphics and Interactive Techniques, August 2012, Article No. 29.
[19] Kaplanyan A S, Dachsbacher C. Adaptive progressive photon mapping. ACM Transactions on Graphics, 2013, 32(2):Article No. 16.
[20] Bengio Y, Simard P, Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 1994, 5(2):157-166.
[21] He K, Zhang X, Ren S et al. Identity mappings in deep residual networks. In Proc. the 14th European Conference on Computer Vision, October 2016, pp.630-645.
[22] He K, Zhang X, Ren S et al. Deep residual learning for image recognition. In Proc. the 2016 IEEE Conference on Computer Vision and Pattern Recognition, June 2016, pp.770-778.
[23] Zagoruyko S, Komodakis N. Wide residual networks. arXiv:1605.07146, 2016., Feb. 2019.
[24] Abdi M, Nahavandi S. Multi-residual networks:Improving the speed and accuracy of residual networks. arXiv:1609.05672, 2016., Feb. 2019.
[25] Silverman B W. Density Estimation for Statistics and Data Analysis. Chapman and Hall, 1986.
[26] Pharr M, Jakob W, Humphreys G. Physically Based Rendering:From Theory to Implementation (3rd edition). Morgan Kaufmann, 2016.
[27] He K, Zhang X, Ren S et al. Delving deep into rectifiers:Surpassing human-level performance on imageNet classification. In Proc. the IEEE International Conference on Computer Vision, December 2015, pp.1026-1034.
[28] Schjøth L, Sporring J, Olsen F O. Diffusion based photon mapping. Computer Graphics Forum, 2008, 27(8):2114-2127.
[29] Abadi M, Barham P, Chen J et al. TensorFlow:A system for large-scale machine learning. In Proc. the 12th USENIX Symposium on Operating Systems Design and Implementation, November 2016, pp.265-283.
[30] Kingma D P, Ba J. Adam:A method for stochastic optimization. arXiv:1412.6980, 2014., Dec. 2019.
[31] Glorot X, Bengio Y. Understanding the difficulty of training deep feed forward neural networks. In Proc. the 13th International Conference on Artificial Intelligence and Statistics, May 2010, pp.249-256.
[1] Xin Zhang, Siyuan Lu, Shui-Hua Wang, Xiang Yu, Su-Jing Wang, Lun Yao, Yi Pan, and Yu-Dong Zhang. Diagnosis of COVID-19 Pneumonia via a Novel Deep Learning Architecture [J]. Journal of Computer Science and Technology, 2022, 37(2): 330-343.
[2] Songjie Niu, Shimin Chen. TransGPerf: Exploiting Transfer Learning for Modeling Distributed Graph Computation Performance [J]. Journal of Computer Science and Technology, 2021, 36(4): 778-791.
[3] Sheng-Luan Hou, Xi-Kun Huang, Chao-Qun Fei, Shu-Han Zhang, Yang-Yang Li, Qi-Lin Sun, Chuan-Qing Wang. A Survey of Text Summarization Approaches Based on Deep Learning [J]. Journal of Computer Science and Technology, 2021, 36(3): 633-663.
[4] Lan Chen, Juntao Ye, Xiaopeng Zhang. Multi-Feature Super-Resolution Network for Cloth Wrinkle Synthesis [J]. Journal of Computer Science and Technology, 2021, 36(3): 478-493.
[5] Yu-Jie Yuan, Yukun Lai, Tong Wu, Lin Gao, Li-Gang Liu. A Revisit of Shape Editing Techniques: From the Geometric to the Neural Viewpoint [J]. Journal of Computer Science and Technology, 2021, 36(3): 520-554.
[6] Wei Du, Yu Sun, Hui-Min Bao, Liang Chen, Ying Li, Yan-Chun Liang. DeepHBSP: A Deep Learning Framework for Predicting Human Blood-Secretory Proteins Using Transfer Learning [J]. Journal of Computer Science and Technology, 2021, 36(2): 234-247.
[7] Jun Gao, Paul Liu, Guang-Di Liu, Le Zhang. Robust Needle Localization and Enhancement Algorithm for Ultrasound by Deep Learning and Beam Steering Methods [J]. Journal of Computer Science and Technology, 2021, 36(2): 334-346.
[8] Hua Chen, Juan Liu, Qing-Man Wen, Zhi-Qun Zuo, Jia-Sheng Liu, Jing Feng, Bao-Chuan Pang, Di Xiao. CytoBrain: Cervical Cancer Screening System Based on Deep Learning Technology [J]. Journal of Computer Science and Technology, 2021, 36(2): 347-360.
[9] Nuo Qun, Hang Yan, Xi-Peng Qiu, Xuan-Jing Huang. Chinese Word Segmentation via BiLSTM+Semi-CRF with Relay Node [J]. Journal of Computer Science and Technology, 2020, 35(5): 1115-1126.
[10] Andrea Caroppo, Alessandro Leone, Pietro Siciliano. Comparison Between Deep Learning Models and Traditional Machine Learning Approaches for Facial Expression Recognition in Ageing Adults [J]. Journal of Computer Science and Technology, 2020, 35(5): 1127-1146.
[11] Dun Liang, Yuan-Chen Guo, Shao-Kui Zhang, Tai-Jiang Mu, Xiaolei Huang. Lane Detection: A Survey with New Results [J]. Journal of Computer Science and Technology, 2020, 35(3): 493-505.
[12] Xin Yang, Dawei Wang, Wenbo Hu, Li-Jing Zhao, Bao-Cai Yin, Qiang Zhang, Xiao-Peng Wei, Hongbo Fu. DEMC: A Deep Dual-Encoder Network for Denoising Monte Carlo Rendering [J]. Journal of Computer Science and Technology, 2019, 34(5): 1123-1135.
[13] Shuai Li, Zheng Fang, Wen-Feng Song, Ai-Min Hao, Hong Qin. Bidirectional Optimization Coupled Lightweight Networks for Efficient and Robust Multi-Person 2D Pose Estimation [J]. Journal of Computer Science and Technology, 2019, 34(3): 522-536.
[14] Lin Yue, Xiao-Xin Sun, Wen-Zhu Gao, Guo-Zhong Feng, Bang-Zuo Zhang. Multiple Auxiliary Information Based Deep Model for Collaborative Filtering [J]. , 2018, 33(4): 668-681.
[15] Bei-Ji Zou, Yun-Di Guo, Qi He, Ping-Bo Ouyang, Ke Liu, Zai-Liang Chen. 3D Filtering by Block Matching and Convolutional Neural Network for Image Denoising [J]. , 2018, 33(4): 838-848.
Full text



[1] Zhou Di;. A Recovery Technique for Distributed Communicating Process Systems[J]. , 1986, 1(2): 34 -43 .
[2] Chen Shihua;. On the Structure of Finite Automata of Which M Is an(Weak)Inverse with Delay τ[J]. , 1986, 1(2): 54 -59 .
[3] Li Wanxue;. Almost Optimal Dynamic 2-3 Trees[J]. , 1986, 1(2): 60 -71 .
[4] Wang Xuan; Lü Zhimin; Tang Yuhai; Xiang Yang;. A High Resolution Chinese Character Generator[J]. , 1986, 1(2): 1 -14 .
[5] C.Y.Chung; H.R.Hwa;. A Chinese Information Processing System[J]. , 1986, 1(2): 15 -24 .
[6] Zhang Cui; Zhao Qinping; Xu Jiafu;. Kernel Language KLND[J]. , 1986, 1(3): 65 -79 .
[7] Wang Jianchao; Wei Daozheng;. An Effective Test Generation Algorithm for Combinational Circuits[J]. , 1986, 1(4): 1 -16 .
[8] Chen Zhaoxiong; Gao Qingshi;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[9] Huang Heyan;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[10] Zheng Guoliang; Li Hui;. The Design and Implementation of the Syntax-Directed Editor Generator(SEG)[J]. , 1986, 1(4): 39 -48 .

ISSN 1000-9000(Print)

CN 11-2296/TP

Editorial Board
Author Guidelines
Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
  Copyright ©2015 JCST, All Rights Reserved