计算机科学技术学报 ›› 2022,Vol. 37 ›› Issue (3): 680-698.doi: 10.1007/s11390-020-0434-1

所属专题: Computer Networks and Distributed Computing

• • 上一篇    下一篇



  • 收稿日期:2020-03-10 修回日期:2020-08-19 接受日期:2020-10-15 出版日期:2022-05-30 发布日期:2022-05-30

NfvInsight: A Framework for Automatically Deploying and Benchmarking VNF Chains

Tian-Ni Xu1,2 (徐天妮), Hai-Feng Sun1,2 (孙海锋), Di Zhang1,2 (张笛), Xiao-Ming Zhou1,2 (周小明), Xiu-Feng Sui3 (隋秀峰), Sa Wang1,2,4 (王卅), Member, CCF, ACM, Qun Huang5 (黄群), Member, CCF, ACM, IEEE, and Yun-Gang Bao1,2,4,* (包云岗), Senior Member, CCF, Member, ACM, IEEE        

  1. 1State Key Laboratory of Computer Architecture, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China
    2University of Chinese Academy of Sciences, Beijing 100049, China
    3School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China
    4Peng Cheng Laboratory, Shenzhen 518055, China
    5Department of Computer Science and Technology, Peking University, Beijing 100871, China
  • Received:2020-03-10 Revised:2020-08-19 Accepted:2020-10-15 Online:2022-05-30 Published:2022-05-30
  • Contact: Yun-Gang Bao E-mail:baoyg@ict.ac.cn
  • About author:Yun-Gang Bao received his B.S. degree from Nanjing University, Nanjing, in 2003, and his Ph.D. degree in computer science from Institute of Computing Technology (ICT), Chinese Academy of Sciences (CAS), Beijing, in 2008. He is a professor at ICT, CAS, Beijing. From 2010 to 2012, he was a postdoctoral researcher at Department of Computer Science, Princeton University, New York city. His current research interests include computer architecture, operating system, and system performance modeling and evaluation. He is a senior member of CCF and a member of ACM and IEEE.
  • Supported by:
    This work was supported by the National Key Research and Development Program of China under Grant No. 2019YFB1802600, the National Natural Science Foundation of China under Grant Nos. 61420106013, 61702480, 61672499, and 61802365, the Youth Innovation Promotion Association of Chinese Academy of Sciences under Grant Nos. 2013073 and 2020105, and the Guangdong Province Key Laboratory of Popular High Performance Computers under Grant No. 2017B030314073.

网络功能虚拟化技术(Network Function Virtualization, NFV)将传统的网络功能(Network Function, NF)的硬件实现转化为软件实现。得益于虚拟化技术和软件定义网络的出现,虚拟网络功能(Virtual Network Function, VNF)得以运行在通用服务器上。软件实现相较于专用硬件设备存在着不可避免的性能差距,而准确定位出造成NFV系统性能低下的原因是具有挑战性的。其难点可以总结为三方面:NFV系统部署繁琐、链的转发路径具有管道效应,以及系统软件栈的复杂性。 为了准确定位性能问题,本文提出了自动化测试框架NfvInsight,它主要由三个模块组成:服务链生成、自动化部署,和细粒测试。框架每个模块的设计和实现都具有先进性:1)本文率先地尝试了利用规则库枚举合理的服务链拓扑;2)高度自动化的部署功能可以在一轮测试中减少手动执行多达391条命令;3)框架可收集细粒度且多维度的测试指标,尤其是针对网络栈延迟,仅引入至多2.2%的开销。本文通过测试五个典型的虚拟网络功能及其形成的链,展示了NfvInsight的实用性和易用性。经过对测试结果的分析,本文发现了分别来自于服务链和底层系统的性能瓶颈,并得到结论:在网络功能虚拟化场景下,网络栈的某些实现对数据包在同一台服务器内被多次转发的情况并不适用。本文的优化方法最多可将服务链的端到端带宽提高3倍。

关键词: 网络功能虚拟化, 服务链, 性能瓶颈, 网络栈延迟

Abstract: With the advent of virtualization techniques and software-defined networking (SDN), network function virtualization (NFV) shifts network functions (NFs) from hardware implementations to software appliances, between which exists a performance gap. How to narrow the gap is an essential issue of current NFV research. However, the cumbersomeness of deployment, the water pipe effect of virtual network function (VNF) chains, and the complexity of the system software stack together make it tough to figure out the cause of low performance in the NFV system. To pinpoint the NFV system performance, we propose NfvInsight, a framework for automatic deployment and benchmarking VNF chains. Our framework tackles the challenges in NFV performance analysis. The framework components include chain graph generation, automatic deployment, and fine granularity measurement. The design and implementation of each component have their advantages. To the best of our knowledge, we make the first attempt to collect rules forming a knowledge base for generating reasonable chain graphs. NfvInsight deploys the generated chain graphs automatically, which frees the network operators from executing at least 391 lines of bash commands for a single test. To diagnose the performance bottleneck, NfvInsight collects metrics from multiple layers of the software stack. Specifically, we collect the network stack latency distribution ingeniously, introducing only less than 2.2% overhead. We showcase the convenience and usability of NfvInsight in finding bottlenecks for both VNF chains and the underlying system. Leveraging our framework, we find several design flaws of the network stack, which are unsuitable for packet forwarding inside one single server under the NFV circumstance. Our optimization for these flaws gains at most 3x performance improvement.

Key words: network function virtualization (NFV), service chain, performance bottleneck, network stack latency

[1] Hwang J, Ramakrishnan K K, Wood T. NetVM: High performance and flexible networking using virtualization on commodity platforms. IEEE Transactions on Network and Service Management, 2015, 12(1): 34-47. DOI: 10.1109/TNSM.2015.2401568.

[2] Zhang W, Liu G, Zhang W, Shah N, Lopreiato P, Todeschi G, Ramakrishnan K, Wood T. OpenNetVM: A platform for high performance network service chains. In Proc. the 2016 ACM SIGCOMM Workshop on Hot Topics in Middleboxes and Network Function Virtualization, August 2016, pp.26-31. DOI: 10.1145/2940147.2940155.

[3] Gember-Jacobson A, Viswanathan R, Prakash C, Grandl R, Khalid J, Das S, Akella A. OpenNF: Enabling innovation in network function control. ACM SIGCOMM Computer Communication Review, 2014, 44(4): 163-174. DOI: 10.1145/2740070.2626313.

[4] Bremler-Barr A, Harchol Y, Hay D. Openbox: A software-defined framework for developing, deploying, and managing network functions. In Proc. the 2016 ACM SIGCOMM Conference, August 2016, pp.511-524. DOI: 10.1145/2934872.2934875.

[5] Katsikas G P, Barbette T, Kostic D, Steinert R, Maguire G Q. Metron: NFV service chains at the true speed of the underlying hardware. In Proc. the 15th USENIX Symposium on Networked Systems Design and Implementation, April 2018, pp.171-186.

[6] Sun C, Bi J, Zheng Z, Yu H, Hu H. NFP: Enabling network function parallelism in NFV. In Proc. the Conference of the ACM Special Interest Group on Data Communication, August 2017, pp.43-56. DOI: 10.1145/3098822.3098826.

[7] Yi X, Duan J, Wu C. GPUNFV: A GPU-accelerated NFV system. In Proc. the 1st Asia-Pacific Workshop on Networking, August 2017, pp.85-91. DOI: 10.1145/3106989.3106990.

[8] Bronstein Z, Roch E, Xia J, Molkho A. Uniform handling and abstraction of NFV hardware accelerators. IEEE Network, 2015, 29(3): 22-29. DOI: 10.1109/MNET.2015.7113221.

[9] Kachris C, Sirakoulis G, Soudris D. Network function virtualization based on FPGAs: A framework for all-programmable network devices. arXiv:1406.0309, 2014. https://arxiv.org/ftp/arxiv/papers/1406/1406. 0309.pdf, Dec. 2021.

[10] Ersue M. ETSI NFV management and orchestration—An overview. https://www.ietf.org/proceedings/88/ slides/slides-88-opsawg-6.pdf, Dec. 2021.

[11] Ma W, Sandoval O, Beltran J, Pan D, Pissinou N. Traffic aware placement of interdependent NFV middleboxes. In Proc. the 2017 IEEE Conference on Computer Communications, May 2017. DOI: 10.1109/INFOCOM.2017.8056993.

[12] Nam J, Seo J, Shin S. Probius: Automated approach for VNF and service chain analysis in software-defined NFV. In Proc. the Symposium on SDN Research, March 2018, Article No. 14. DOI: 10.1145/3185467.3185495.

[13] Sherry J, Hasan S, Scott C, Krishnamurthy A, Ratnasamy S, Sekar V. Making middleboxes someone else’s problem: Network processing as a cloud service. ACM SIGCOMM Computer Communication Review, 2012, 42(4): 13-24. DOI: 10.1145/2377677.2377680.

[14] Koshy T. Catalan Numbers with Applications. Oxford University Press, 2008.

[15] Prakash C, Lee J, Turner Y, Kang J M, Akella A, Banerjee S, Clark C, Ma Y, Sharma P, Zhang Y. PGA: Using graphs to express and automatically reconcile network policies. ACM SIGCOMM Computer Communication Review, 2015, 45(4): 29-42. DOI: 10.1145/2829988.2787506.

[16] Hubert B, Maxwell G, Van Mook R et al. Linux advanced routing & traffic control HOWTO. https://tldp. org/HOWTO/pdf/Adv-Routing-HOWTO.pdf, Dec. 2021.

[17] Blake G, Saidi A G. Where does the time go? Characterizing tail latency in memcached. In Proc. the 2015 IEEE International Symposium on Performance Analysis of Systems and Software, March 2015, pp.21-31. DOI: 10.1109/ISPASS.2015.7095781.

[18] Moradi F, Flinta C, Johnsson A, Meirosu C. ConMon: An automated container based network performance monitoring system. In Proc. the 2017 IFIP/IEEE Symposium on Integrated Network and Service Management, May 2017, pp.54-62. DOI: 10.23919/INM.2017.7987264.

[19] Naik P, Shaw D K, Vutukuru M. NFVPerf: Online performance monitoring and bottleneck detection for NFV. In Proc. the 2016 IEEE Conference on Network Function Virtualization and Software Defined Networks, November 2016, pp.154-160. DOI: 10.1109/NFV-SDN.2016.7919491.

[20] Peuster M, Karl H. Understand your chains: Towards performance profile-based network service management. In Proc. the 5th European Workshop on Software-Defined Networks, October 2016, pp.7-12. DOI: 10.1109/EWSDN.2016.9.

[21] Katsikas G P, Maguire G Q, Kostić D. Profiling and accelerating commodity NFV service chains with SCC. Journal of Systems and Software, 2017, 127: 12-27. DOI: 10.1016/j.jss.2017.01.005.

[22] Rajagopalan S, Williams D, Jamjoom H, Warfield A. Split/merge: System support for elastic ution in virtual middleboxes. In Proc. the 10th USENIX Symposium on Networked Systems Design and Implementation, April 2013, pp.227-240.

[23] Zhang Y, Anwer B, Gopalakrishnan V, Han B, Reich J, Shaikh A, Zhang Z L. ParaBox: Exploiting parallelism for virtual network functions in service chaining. In Proc. the Symposium on SDN Research, April 2017, pp.143-149. DOI: 10.1145/3050220.3050236.

[24] Duan Q. Cloud service performance evaluation: Status, challenges, and opportunities—A survey from the system modeling perspective. Digital Communications and Networks, 2017, 3(2): 101-111. DOI: 10.1016/j.dcan.2016.12.002.

[25] Feng H, Llorca J, Tulino A M, Raz D, Molisch A F. Approximation algorithms for the NFV service distribution problem. In Proc. the 2017 IEEE Conference on Computer Communications, May 2017. DOI: 10.1109/INFOCOM.2017.8057039.

[26] Agarwal S, Malandrino F, Chiasserini C F, De S. Joint VNF placement and CPU allocation in 5G. In Proc. the 2018 IEEE Conference on Computer Communications, April 2018, pp.1943-1951. DOI: 10.1109/INFOCOM.2018.8485943.

[27] Cziva R, Anagnostopoulos C, Pezaros D P. Dynamic, latency-optimal vNF placement at the network edge. In Proc. the 2018 IEEE Conference on Computer Communications, April 2018, pp.693-701. DOI: 10.1109/INFOCOM.2018.8486021.

[28] Luizelli M C, Raz D, Sa’ar Y. Optimizing NFV chain deployment through minimizing the cost of virtual switching. In Proc. the 2018 IEEE Conference on Computer Communications, April 2018, pp.2150-2158. DOI: 10.1109/INFOCOM.2018.8486315.

[29] Bari M F, Chowdhury S R, Ahmed R, Boutaba R. On orchestrating virtual network functions. In Proc. the 11th International Conference on Network and Service Management, November 2015, pp.50-56. DOI: 10.1109/CNSM.2015.7367338.

[1] Xue-Kai Du, Zhi-Hui Lu, Qiang Duan, Jie Wu, Cheng-Rong Wu. 在多租户数据中心中针对安全服务的负载自适应的流量转向和转发方案[J]. , 2017, 32(6): 1265-1278.
Full text



[1] 周笛;. A Recovery Technique for Distributed Communicating Process Systems[J]. , 1986, 1(2): 34 -43 .
[2] 陈世华;. On the Structure of Finite Automata of Which M Is an(Weak)Inverse with Delay τ[J]. , 1986, 1(2): 54 -59 .
[3] 王建潮; 魏道政;. An Effective Test Generation Algorithm for Combinational Circuits[J]. , 1986, 1(4): 1 -16 .
[4] 陈肇雄; 高庆狮;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[5] 黄河燕;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[6] 郑国梁; 李辉;. The Design and Implementation of the Syntax-Directed Editor Generator(SEG)[J]. , 1986, 1(4): 39 -48 .
[7] 黄学东; 蔡莲红; 方棣棠; 迟边进; 周立; 蒋力;. A Computer System for Chinese Character Speech Input[J]. , 1986, 1(4): 75 -83 .
[8] 许小曙;. Simplification of Multivalued Sequential SULM Network by Using Cascade Decomposition[J]. , 1986, 1(4): 84 -95 .
[9] 唐同诰; 招兆铿;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[10] 衷仁保; 邢林; 任朝阳;. An Interactive System SDI on Microcomputer[J]. , 1987, 2(1): 64 -71 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持:support@magtech.com.cn