›› 2016, Vol. 31 ›› Issue (1): 107-116.doi: 10.1007/s11390-016-1614-x

Special Issue: Computer Architecture and Systems; Computer Networks and Distributed Computing

• Computer Architectures and Systems • Previous Articles     Next Articles

Optimizations for High Performance Network Virtualization

Fan-Fu Zhou(周凡夫), Ru-Hui Ma(马汝辉), Jian Li(李健), Member, ACM, IEEE, Li-Xia Chen(陈丽霞), Wei-Dong Qiu(邱卫东), and Hai-Bing Guan*(管海兵), Member, CCF, ACM, IEEE   

  1. Shanghai Key Laboratory of Scalable Computing and Systems, Shanghai Jiao Tong University, Shanghai 200240, China
  • Received:2015-07-05 Revised:2015-09-25 Online:2016-01-05 Published:2016-01-05
  • Contact: Hai-Bing Guan E-mail:hbguan@sjtu.edu.cn
  • About author:Fan-Fu Zhou is currently a Ph.D. student in computer science and engineering at Shanghai Jiao Tong University, Shanghai. Prior to this, he received his M.S. and B.S. degrees in computer science in 2012 and 2008, respectively, both from Shanghai Jiao Tong University, Shanghai, and Huazhong University of Science and Technology, Wuhan, respectively. His main interest lies in computer network and security.
  • Supported by:

    This work was supported by the National High Technology Research and Development 863 Program of China under Grant No. 2012AA010905, the National Natural Science Foundation of China under Grant Nos. 61272100 and 61202374, the Ministry of Education Major Project of China under Grant No. 313035, and the National Research Foundation (NRF) Singapore under its CREATE Program.

The increasing requirements of intensive interoperaterbility among the distributed nodes desiderate the high performance network connections, owing to the substantial growth of cloud computing and datacenters. Network I/O virtualization aggregates the network resource and separates it into manageable parts for particular servers or devices, which provides effective consolidation and elastic management with high agility, flexibility and scalability as well as reduced cost and cabling. However, both network I/O virtualization aggregation and the increasing network speed incur higher traffic density, which generates a heavy system stress for I/O data moving and I/O event processing. Consequently, many researchers have dedicated to enhancing the system performance and alleviating the system overhead for high performance networking virtualization. This paper first elaborates the mainstreaming I/O virtualization methodologies, including device emulation, split-driver model and hardware assisted model. Then, the paper discusses and compares their specific advantages in addition to performance bottlenecks in practical utilities. This paper mainly focuses on the comprehensive survey of stateof-the-art approaches for performance optimizations and improvements as well as the portability management for network I/O virtualization. The approaches include various novel data delivery schemes, overhead mitigations for interrupt processing and adequate resource allocations for dynamic network states. Finally, we highlight the diversity of I/O virtualization besides the performance improvements in network virtualization infrastructure.

[1] Zhang H, Diao Y, Immerman N. On complexity and optimization of expensive queries in complex event processing. In Proc. the 2014 ACM SIGMOD International Conference on Management of Data, June 2014, pp.217-228.

[2] McWherter D T, Schroeder B, Ailamaki A, Harchol-Balter M. Priority mechanisms for OLTP and transactional web applications. In Proc. the 20th International Conference on Data Engineering, Mar.30-Apr.2, 2004, pp.535-546.

[3] Chen R, Shi J, Chen Y, Chen H. PowerLyra:Differentiated graph computation and partitioning on skewed graphs. In Proc. the 10th European Conference on Computer Systems, Apr. 2015.

[4] Cisco. Cisco global cloud index:Fore-cast and methodology, 2014-2019. http://www.cisco.com/c/en/us/solutions/collateral/service-provider/global-cloud-index-gci/Cloud Index White Paper.pdf, Sept. 2015.

[5] Amman M, Briceno C, Connell K, Rungta S. Upgrading data center network architecture to 10 gigabit Ethernet. 2011. http://intelethernet-ibm.com/pdf/Upgrading_Data_Center_Network_Architecture_10Gb_Ethernet2_Final.pdf, Sept. 2015.

[6] IDC. The new need for speed in the datacenter network. 2015. http://www.cisco.com/c/dam/en/us/products/collateral/switches/nexus-9000-series-switches/white-paper-c-11-734328.pdf, Sept. 2015.

[7] Porter G, Strong R, Farrington N, Forencich A, Sun P C, Rosing T, Fainman Y, Papen G, Vahdat A. Integrating microsecond circuit switching into the data center. In Proc. the 2013 ACM SIGCOMM, Aug. 2013, pp.447-458.

[8] Waldspurger C, Rosenblum M. I/O virtualization. Communications of the ACM, 2012, 55(1):66-73.

[9] Guan H, Dong Y, Tian K, Li J. SR-IOV based network interrupt-free virtualization with event based polling. IEEE Journal on Selected Areas in Communications, 2013, 31(12):2596-2609.

[10] Liu J, Abali B. Virtualization polling engine (VPE):Using dedicated CPU cores to accelerate I/O virtualization. In Proc. the 23rd International Conference on Supercomputing, June 2009, pp.225-234.

[11] Dey K, Mishra D, Kulkarni P. Vagabond:Dynamic network endpoint reconfiguration in virtualized environments. In Proc. the 5th ACM Symposium on Cloud Computing, Nov. 2014, pp.21:1-21:13.

[12] Barham P, Dragovic B, Fraser K, Hand S, Harris T, Ho A, Neugebauer R, Pratt I, Warfield A. Xen and the art of virtualization. In Proc. the 19th ACM SOSP, Oct. 2003, pp.164-177.

[13] Goldberg R P. Survey of virtual machine research. Computer, 1974, 7(9):34-45.

[14] Creasy R J. The origin of the VM/370 time-sharing system. IBM Journal of Research and Development, 1981, 25(5):483-490.

[15] Sugerman J, Venkitachalam G, Lim B H. Virtualizing I/O devices on VMware workstation's hosted virtual machine monitor. In Proc. the 2001 USENIX Annual Technical Conference, June 2001.

[16] Ahmad I, Anderson J M, Holler A M, Kambo R, Makhija V. An analysis of disk performance in VMware ESX server virtual machines. In Proc. the 6th Annual Workshop on Workload Characterization, Oct. 2003, pp.65-76.

[17] Bellard F. QEMU, a fast and portable dynamic translator. In Proc. the 2005 USENIX Annual Technical Conference, Apr. 2005, pp.41-46.

[18] Kivity A, Kamay Y, Laor D, Lublin U, Liguori A. kvm:The linux virtual machine monitor. In Proc. the Linux Symposium, June 2007, pp.225-230.

[19] Dong Y, Xu D, Zhang Y, Liao G. Optimizing network I/O virtualization with efficient interrupt coalescing and virtual receive side scaling. In Proc. the IEEE CLUSTER, Sept. 2011, pp.26-34.

[20] Ben-Yehuda M, Day M D, Dubitzky Z, Factor M, Har'El N, Gordon A, Liguori A, Wasserman O, Yassour B A. The turtles project:Design and implementation of nested virtualization. In Proc. the 9th USENIX Symposium on Operating Systems Design and Implementation, Oct. 2010, pp.423- 436.

[21] Fraser K, Hand S, Neugebauer R, Pratt I, Warfield A, Williamson M. Safe hardware access with the Xen virtual machine monitor. In Proc. the 1st Workshop on Operating System and Architectural Support for the on demand IT InfraStructure (OASIS), Oct. 2004.

[22] Russell R. Virtio:Towards a de-facto standard for virtual I/O devices. ACM SIGOPS Operating Systems Review, 2008, 42(5):95-103.

[23] Amsden Z, Arai D, Hecht D, Holler A, Subrahmanyam P. VMI:An interface for paravirtualization. In Proc. the 2006 Linux Symposium, July 2006, pp.363-378.

[24] Dong Y, Zhang X, Dai J, Guan H. HYVI:A hybrid virtualization solution balancing performance and manageability. IEEE Transactions on Parallel and Distributed Systems, 2014, 25(9):2332-2341.

[25] Menon A, Cox A L, Zwaenepoel W. Optimizing network virtualization in Xen. In Proc. the 2006 USENIX Annual Technical Conference, May 30-June 3, 2006, pp.15-28.

[26] Intel. PCI-SIG SR-IOV primer:An introduction to SR-IOV technology. 2011. http://www.intel.com/content/www/-us/en/pci-express/pci-sig-sr-iov-primer-sr-iov-technologypaper.html, Sept. 2015.

[27] Tian K, Dong Y, Mi X, Guan H. sEBP:Event based polling for efficient I/O virtualization. In Proc. the 2012 CLUSTER, Sept. 2012, pp.135-143.

[28] Huang Z, Ma R, Li J, Chang Z, Guan H. Adaptive and scalable optimizations for high performance SR-IOV. In Proc. the 2012 CLUSTER, Sept. 2012, pp.459-467.

[29] Wilson C, Ballani H, Karagiannis T, Rowtron A. Better never than late:Meeting deadlines in data-center networks. In Proc. the 2011 ACM SIGCOMM, Aug. 2011, pp.50-61.

[30] Vamanan B, Hasan J, Vijaykumar T. Deadline-aware datacenter TCP (D 2 TCP). In Proc. the 2012 ACM SIGCOMM, Aug. 2012, pp.115-126.

[31] Alizadeh M, Kabbani A, Edsall T, Prabhakar B, Vahdat A, Yasuda M. Less is more:Trading a little bandwidth for ultra-low latency in the data center. In Proc. the 9th USENIX Conference on Networked Systems Design and Implementation, Apr. 2012, pp.253-266.

[32] Meisner D, Sadler C M, Barroso L A, Weber W D, Wenisch T F. Power management of online data-intensive services. In Proc. the 38th Annual International Symposium on Computer Architecture (ISCA), June 2011, pp.319-330.

[33] Ahmad I, Gulati A, Mashtizadeh A. VIC:Interrupt coalescing for virtual machine storage device IO. In Proc. the 2011 USENIX Annual Technical Conference (ATC), June 2011, pp.45-58.

[34] Gordon A, Amit N, Har'El N, Ben-Yehuda M, Landau A, Schuster A, Tsafrir D. ELI:Bare-metal performance for I/O virtualization. ACM SIGARCH Computer Architecture News, 2012, 40(1):411-422.

[35] Dong Y, Yang X, Li X, Li J, Tian K, Guan H. High performance network virtualization with SR-IOV. In Proc. the 16th International Symposium on High Performance Computer Architecture (HPCA), Jan. 2010.

[36] Santos J R, Turner Y, Janakiraman G J, Pratt I. Bridging the gap between software and hardware techniques for I/O virtualization. In Proc. the 2008 USENIX Annual Technical Conference, June 2008, pp.29-42.

[37] Intel. Intel® 82576 gigabit Ethernet controller:Datasheet. 2014. http://www.intel.com/content/www/us/en/intelligent-systems/networking/82576eg-gbe-datasheet.html, Sept. 2015.

[38] Intel. Intel® 8259910 gigabit Ethernet controller:Datasheet. 2015. http://www.intel.com/content/www/us/en/embedded/products/networking/82599-10-gbe-controller-datasheet.html, Sept. 2015.

[39] Liao G, Guo D, Bhuyan L, King S R. Software techniques to improve virtualized I/O performance on multi-core systems. In Proc. the 4th ACM/IEEE Symposium on Architectures for Networking and Communications Systems, Nov. 2008, pp.161-170.

[40] Martins J, Ahmed M, Raiciu C, Olteanu V, Honda M, Bifulco R, Huici F. ClickOS and the art of network function virtualization. In Proc. the 2014 USENIX NSDI, Apr. 2014, pp.459-473.

[41] Ram K K, Santos J R, Turner Y, Cox A L, Rixner S. Achieving 10 Gb/s using safe and transparent network interface virtualization. In Proc. the 2009 ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, Mar. 2009, pp.61-70.

[42] Menon A, Santos J R, Turner Y, Janakiraman G J, Zwaenepoel W. Diagnosing performance overheads in the Xen virtual machine environment. In Proc. the 1st ACM/USENIX International Conference on Virtual Execution Environments, June 2005, pp.13-23.

[43] Zhang X, McIntosh S, Rohatgi P, Griffin J L. XenSocket:A high-throughput interdomain transport for virtual machines. In Proc. the 8th Middleware, Nov. 2007, pp.184-203.

[44] Huang W, Koop M J, Gao Q, Panda D K. Virtual machine aware communication libraries for high performance computing. In Proc. the 2007 ACM/IEEE Conference on Supercomputing, Nov. 2007, pp.9:1-9:12.

[45] Kim K, Kim C, Jung S I, Shin H S, Kim J S. Interdomain socket communications supporting high performance and full binary compatibility on Xen. In Proc. the 4th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, Mar. 2008, pp.11-20.

[46] Wang J, Wright K L, Gopalan K. XenLoop:A transparent high performance inter-VM network loopback. In Proc. the 17th International Symposium on High Performance Distributed Computing, June 2008, pp.109-118.

[47] Willmann P, Shafer J, Carr D, Menon A, Rixner S, Cox A L, Zwaenepoel W. Concurrent direct network access for virtual machine monitors. In Proc. the 13th HPCA, Feb. 2007, pp.306-317.

[48] Raj H, Schwan K. High performance and scalable I/O virtualization via self-virtualized devices. In Proc. the 16th International Symposium on High Performance Distributed Computing, June 2007, pp.179-188.

[49] Salim J H. When NAPI comes to town. In Proc. the Linux 2005 Conf, Apr. 2005.

[50] Har'El N, Gordon A, Landau A, Ben-Yehuda M, Traeger A, Ladelsky R. Efficient and scalable paravirtual I/O system. In Proc. the 2013 USENIX Annual Technical Conference, June 2013, pp.231-242.

[51] Tu C C, Ferdman M, Lee C, Chiueh T C. A comprehensive implementation and evaluation of direct interrupt delivery. In Proc. the 11th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, Mar. 2015, pp.1-15.

[52] Chang Z, Li J, Ma R, Huang Z, Guan H. Adjustable credit scheduling for high performance network virtualization. In Proc. the 2012 CLUSTER, Sept. 2012, pp.337-345.

[53] Guan H, Ma R, Li J. Workload-aware credit scheduler for improving network I/O performance in virtualization environment. IEEE Transactions on Cloud Computing, 2014, 2(2):130-142.

[54] Pfefferle J, Stuedi P, Trivedi A, Metzler B, Koltsidas I, Gross T R. A hybrid I/O virtualization framework for RDMA-capable network interfaces. In Proc. the 11th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, Mar. 2015, pp.17-30.

[55] Younge A J, Walters J P, Crago S P, Fox G C. Supporting high performance molecular dynamics in virtualized clusters using IOMMU, SR-IOV, GPUDirect. In Proc. the 11th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, Mar. 2015, pp.31-38.

[56] Walters J P, Younge A J, Kang D I, Yao K T, Kang M, Crago S P, Fox G C. GPU passthrough performance:A comparison of KVM, Xen, VMWare ESXi, and LXC for CUDA and OpenCL applications. In Proc. the 7th IEEE International Conference on Cloud Computing (CLOUD), June 27-July 2, 2014, pp.636-643.
No related articles found!
Full text



[1] Chen Yangjun;. Counting and Topological Order[J]. , 1997, 12(6): 497 -509 .
[2] Ju-Hum Kwon, Chee-Yang Song, Chang-Joo Moon, and Doo-Kwon Baik. Bridging Real World Semantics to Model World Semantics for Taxonomy Based Knowledge Representation System[J]. , 2005, 20(3): 296 -308 .
[3] Dun-Ren Che. Accomplishing Deterministic XML Query Optimization[J]. , 2005, 20(3): 357 -366 .
[4] Bo Yan, You-Xing Qu, Feng-Lou Mao, Victor N. Olman, and Ying Xu. PRIME: A Mass Spectrum Data Mining Tool for De Novo Sequencing and PTMs Identification[J]. , 2005, 20(4): 483 -490 .
[5] Zhou-Wang Yang, Chun-Lin Wu, Jian-Song Deng,and Fa-Lai Chen. Specification of Initial Shapes for Dynamic Implicit Curve/Surface Reconstruction[J]. , 2006, 21(2): 249 -254 .
[6] Zhi-Wei Xu, Hao-Jie Zhou, and Guo-Jie Li. Usability Issues of Grid System Software[J]. , 2006, 21(5): 641 -647 .
[7] Li-Guo Yu and Srini Ramaswamy. Component Dependency in Object-Oriented Software[J]. , 2007, 22(3): 379 -386 .
[8] Ji-Gang Wu, Thambipillai Srikanthan, and Guang-Wei Zou. New Model and Algorithm for Hardware/Software Partitioning[J]. , 2008, 23(4 ): 644 -651 .
[9] Qing-Shi Gao, Xiao-Yu Gao, and Yue Hu. A New Fuzzy Set Theory Satisfying All Classical Set Formulas[J]. , 2009, 24(4): 798 -804 .
[10] Osman Hasan and Sofiéne Tahar, Senior Member, IEEE, Member, ACM . Formally Analyzing Expected Time Complexity of Algorithms Using Theorem Proving[J]. , 2010, 25(6): 1305 -1320 .

ISSN 1000-9000(Print)

CN 11-2296/TP

Editorial Board
Author Guidelines
Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
E-mail: jcst@ict.ac.cn
  Copyright ©2015 JCST, All Rights Reserved