›› 2015, Vol. 30 ›› Issue (6): 1163-1174.doi: 10.1007/s11390-015-1591-5

Special Issue: Data Management and Data Mining

• Special Section on Networking and Distributed Computing for Big Data • Previous Articles     Next Articles

Towards Cost-Effective Cloud Downloading with Tencent Big Data

Zhen-Hua Li1,2(李振华), Member, CCF, ACM, IEEE, Gang Liu3(刘刚), Zhi-Yuan Ji4*(嵇智源), Roger Zimmermann5, Senior Member, IEEE, Member, ACM   

  1. 1 School of Software, Tsinghua University, Beijing 100084, China;
    2 Tsinghua National Laboratory for Information Science and Technology, Beijing 100084, China;
    3 QQXuanfeng System Group, Tencent Co., Ltd, Shanghai 200233, China;
    4 High Technology Research and Development Center, Ministry of Science and Technology, Beijing 100044, China;
    5 Department of Computer Science, National University of Singapore, Singapore 117417, Singapore
  • Received:2015-03-12 Revised:2015-08-04 Online:2015-11-05 Published:2015-11-05
  • About author:Zhen-Hua Li is an assistant professor at the School of Software, Tsinghua University, Beijing. He received his B.S. and M.S. degrees in computer science and technology from Nanjing University in 2005 and 2008 respectively, and his Ph.D. degree in computer science and technology from Peking University in 2013. His research areas mainly consist of Internet content distribution, mobile Internet, and cloud computing/storage.
  • Supported by:

    This work is sponsored by the National Natural Science Foundation of China under Grant Nos. 61471217 and 61472266, the China Postdoctoral Science Foundation under Grant No. 2014M550735, and the CCF-Tencent Open Fund under Grant No. AGR20150201.

The cloud downloading scheme, first proposed by us in 2011, has effectively optimized hundreds of millions of users' downloading experiences. Also, people start to build a variety of useful Internet services on top of cloud downloading. In brief, by using cloud facilities to download (and cache) the requested file from the “best-effort” Internet on behalf of the user, cloud downloading ensures the data availability and remarkably enhances the data delivery speed. Although this scheme seems simple and straightforward, designing a real-world cloud downloading system involves complicated and subtle trade-offs (between deployment cost and user experience) when serving a large number of users: 1) how to plan the cloud cache capacity to achieve a high and affordable cache hit ratio, 2) how to accelerate the data delivery from the cloud to numerous users, 3) how to handle the dense user requests for highly popular files, and 4) how to judge a potential downloading failure of the cloud. This paper addresses these design trade-offs from a practical perspective, based on big data from a nationwide commercial cloud downloading system, i.e., Tencent QQXuanfeng. Its running traces help us find reasonable design strategies and parameters, and its real performances confirm the efficacy of our design. Our study provides solid experiences and valuable heuristics for the developers of similar and relevant systems.

[1] Liu Y, Guo L, Li F, Chen S. An empirical evaluation of battery power consumption for streaming data transmission to mobile devices. In Proc. the 19th ACM-MM, Nov.28-Dec.1, 2011, pp.473-482.

[2] Chen G, Li Z. Peer-to-Peer Network: Structure, Application and Design. Tsinghua University Press, Sep. 2007. (in Chinese)

[3] Liu Y, Xiao L, Liu X, Ni L M, Zhang X. Location awareness in unstructured peer-to-peer systems. IEEE Transactions on Parallel and Distributed Systems, 2005, 16(2): 163-174.

[4] Wen Y, Zhu X, Rodrigues J, Chen C. Cloud mobile media: Reflections and outlook. IEEE Transactions on Multimedia, 2014, 16(4): 885-902.

[5] Huang Y, Li Z, Liu G, Dai Y. Cloud download: Using cloud utilities to achieve high-quality content distribution for unpopular videos. In Proc. the 19th ACM-MM, Nov.28-Dec.1, 2011, pp.213-222.

[6] Ao N, Xu Y, Chen C, Guo Y. Offline downloading: A nontraditional cloud-accelerated and peer-assisted file distribution service. In Proc. the 4th CyberC, Oct. 2012, pp.81-88.

[7] Zhou Y, Fu Z, Chiu D M, Huang Y. An adaptive cloud downloading service. IEEE Transactions on Multimedia, 2013, 15(4): 802-810.

[8] Li Z, Huang Y, Liu G, Wang F, Zhang Z L, Dai Y. Cloud Transcoder: Bridging the format and resolution gap between Internet videos and mobile devices. In Proc. the 22nd ACM NOSSDAV, Jun. 2012, pp.33-38.

[9] Hu H, Wen Y, Chua T, Li X. Towards scalable systems for big data analytics: A technology tutorial. IEEE Access, 2014, 2: 652-687.

[10] Podlipnig S, Böszörmenyi L. A survey of web cache replacement strategies. ACM Computing Surveys, 2003, 35(4): 374-398.

[11] Zhang J, Izmailov R, Reininger D, Ott M. Web caching framework: Analytical models and beyond. In Proc. IEEE Workshop on Internet Applications (WIA), Jul. 1999, pp.132-141.

[12] Arlitt M, Cherkasova L, Dilley J, Friedrich R, Jin T. Evaluating content management techniques for web proxy caches. ACM SIGMETRICS Performance Evaluation Review, 2000, 27(4): 3-11.

[13] Wu D, Hou Y T, Zhu W, Zhang Y Q, Peha J M. Streaming video over the Internet: Approaches and directions. IEEE Transactions on Circuits and Systems for Video Technology, 2001, 11(3): 282-300.

[14] Li Z, Cao J, Chen G. ContinuStreaming: Achieving high playback continuity of gossip-based peer-to-peer streaming. In Proc. the 22nd IEEE IPDPS, Apr. 2008.

[15] Liu F, Shen S, Li B, Li B, Yin H, Li S. Novasky: Cinematicquality VoD in a P2P storage cloud. In Proc. the 30th INFOCOM, Apr. 2011, pp.936-944.

[16] Yin H, Liu X, Zhan T, Sekar V, Qiu F, Lin C, Zhang H, Li B. Design and deployment of a hybrid CDN-P2P system for live video streaming: Experiences with LiveSky. In Proc. the 17th ACM-MM, Oct. 2009, pp.25-34.

[17] Wu C, Li B, Zhao S. On dynamic server provisioning in multichannel P2P live streaming. IEEE/ACM Transactions on Networking, 2011, 19(5): 1317-1330.

[18] Li Z, Huang Y, Liu G, Wang F, Liu Y, Zhang Z L, Dai Y. Challenges, designs and performances of large-scale open- P2SP content distribution. IEEE Transactions on Parallel and Distributed Systems, 2013, 24(11): 2181-2191.

[19] Li Z, Zhang T, Huang Y, Zhang Z L, Dai Y. Maximizing the bandwidth multiplier effect for hybrid Cloud-P2P content distribution. In Proc. the 20th ACM/IEEE IWQoS, Jun. 2012, pp.20:1-20:9.

[20] Li Z, Wilson C, Xu T, Liu Y, Lu Z, Wang Y. Offline downloading in China: A comparative study. In Proc. the 15th ACM IMC, Oct. 2015.
No related articles found!
Full text



[1] Liu Mingye; Hong Enyu;. Some Covering Problems and Their Solutions in Automatic Logic Synthesis Systems[J]. , 1986, 1(2): 83 -92 .
[2] Chen Shihua;. On the Structure of (Weak) Inverses of an (Weakly) Invertible Finite Automaton[J]. , 1986, 1(3): 92 -100 .
[3] Gao Qingshi; Zhang Xiang; Yang Shufan; Chen Shuqing;. Vector Computer 757[J]. , 1986, 1(3): 1 -14 .
[4] Chen Zhaoxiong; Gao Qingshi;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[5] Huang Heyan;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[6] Min Yinghua; Han Zhide;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[7] Tang Tonggao; Zhao Zhaokeng;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[8] Min Yinghua;. Easy Test Generation PLAs[J]. , 1987, 2(1): 72 -80 .
[9] Zhu Hong;. Some Mathematical Properties of the Functional Programming Language FP[J]. , 1987, 2(3): 202 -216 .
[10] Li Minghui;. CAD System of Microprogrammed Digital Systems[J]. , 1987, 2(3): 226 -235 .

ISSN 1000-9000(Print)

CN 11-2296/TP

Editorial Board
Author Guidelines
Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
E-mail: jcst@ict.ac.cn
  Copyright ©2015 JCST, All Rights Reserved