Journal of Computer Science and Technology ›› 2022, Vol. 37 ›› Issue (2): 468-486.doi: 10.1007/s11390-021-1083-8

Special Issue: Computer Architecture and Systems

• Regular Paper • Previous Articles     Next Articles

Unified Enclave Abstraction and Secure Enclave Migration on Heterogeneous Security Architectures

Jin-Yu Gu1,2 (古金宇), Hao Li1,2 (李浩), Yu-Bin Xia1,2,* (夏虞斌), Senior Member, CCF, Member, ACM, IEEE, Hai-Bo Chen1,2 (陈海波), Distinguished Member, CCF, ACM, Cheng-Gang Qin3 (秦承刚), and Zheng-Yu He3 (何征宇)        

  1. 1Engineering Research Center for Domain-Specific Operating Systems, Ministry of Education, Shanghai 200240, China
    2Institute of Parallel and Distributed Systems, Shanghai Jiao Tong University, Shanghai 200240, China
    3Ant Group, Hangzhou 310099, China
  • Received:2020-10-18 Revised:2021-02-12 Accepted:2021-02-21 Online:2022-03-31 Published:2022-03-31
  • Contact: Yu-Bin Xia E-mail:xiayubin@sjtu.edu.cn
  • About author:Yu-Bin Xia received his diploma degree from Software School, Fudan University, Shanghai, in 2004, and his Ph.D. degree in computer science and technology from Peking University, Beijing, in 2010. He is currently an associate professor in Shanghai Jiao Tong University, Shanghai. His research interests include computer architecture, operating system, virtualization, and security.
  • Supported by:
    This work is supported in part by the National Key Research and Development Program of China under Grant No. 2020AAA-0108502, the National Natural Science Foundation of China under Grant No. 61972244, U19A2060, and 61925206, and the HighTech Support Program from Shanghai Committee of Science and Technology under Grant No. 19511121100.

PDF (PC)

11

Supplement

1

Nowadays, application migration becomes more and more attractive. For example, it can make computation closer to data sources or make service closer to end-users, which may significantly decrease latency in edge computing. Yet, migrating applications among servers that are controlled by different platform owners raises security issues. We leverage hardware-secured trusted execution environment (TEE, aka., enclave) technologies, such as Intel SGX, AMD SEV, and ARM TrustZone, for protecting critical computations on untrusted servers. However, these hardware TEEs propose non-uniform programming abstractions and are based on heterogeneous architectures, which not only forces programmers to develop secure applications targeting some specific abstraction but also hinders the migration of protected applications. Therefore, we propose UniTEE which gives a unified enclave programming abstraction across the above three hardware TEEs by using a microkernel-based design and enables the secure enclave migration by integrating heterogeneous migration techniques. We have implemented the prototype on real machines. The evaluation results show the migration support incurs nearly-zero runtime overhead and the migration procedure is also efficient.


Key words: heterogeneous trusted execution environment (TEE); enclave abstraction; enclave migration ;

[1] Park H, Zhai S, Lu L, Lin F X. StreamBox-TZ: Secure stream analytics at the edge with TrustZone. In Proc. the 2019 USENIX Annual Technical Conference, July 2019, pp.537-554.
[2] Shi W, Cao J, Zhang Q, Li Y, Xu L. Edge computing: Vision and challenges. IEEE Internet of Things Journal, 2016, 3(5): 637-646. DOI: 10.1109/JIOT.2016.2579198.
[3] Hu Y C, Patel M, Sabella D, Sprecher N, Young V. Mobile edge computing---A key technology towards 5G. Technical Report, European Telecommunications Standards Institute, 2015. https://infotech.report/Resources/Whitepaper-s/f205849d-0109-4de3-8c47-be52f4e4fb27_etsi_wp11_mec_a_ key_technology_towards_5g.pdf, Dec. 2021.
[4] Satyanarayanan M. The emergence of edge computing. Computer, 2017, 50(1): 30-39. DOI: 10.1109/MC.2017.9.
[5] Shi W, Dustdar S. The promise of edge computing. Computer, 2016, 49(5): 78-81. DOI: 10.1109/MC.2016.145.
[6] Stojkoska B L R, Trivodaliev K V. A review of Internet of Things for smart home: Challenges and solutions. Journal of Cleaner Production, 2017, 140: 1454-1464. DOI: 10.1016/j.jclepro.2016.10.006.
[7] Nastic S, Rausch T, Scekic O, Dustdar S, Gusev M, Koteska B, Kostoska M, Jakimovski B, Ristov S, Prodan R. A serverless real-time data analytics platform for edge computing. IEEE Internet Computing, 2017, 21(4): 64-71. DOI: 10.1109/MIC.2017.2911430.
[8] Machen A, Wang S, Leung K K, Ko B J, Salonidis T. Live service migration in mobile edge clouds. IEEE Wireless Communications, 2017, 25(1): 140-147. DOI: 10.1109/MWC.2017.1700011.
[9] Wang S, Xu J, Zhang N, Liu Y. A survey on service migration in mobile edge computing. IEEE Access, 2018, 6: 23511-23528. DOI: 10.1109/ACCESS.2018.2828102.
[10] Islam M, Razzaque A, Islam J. A genetic algorithm for virtual machine migration in heterogeneous mobile cloud computing. In Proc. the 2016 International Conference on Networking Systems and Security, Jan. 2016. DOI: 10.1109/NSysS.2016.7400696.
[11] Barbalace A, Karaoui M L, Wang W, Xing T, Olivier P, Ravindran B. Edge computing: The case for heterogeneous-ISA container migration. In Proc. the 16th ACM SIGPLAN/SIGOPS International Conference on Virtual ution Environments, Mar. 2020, pp.73-87. DOI: 10.1145/3381052.3381321.
[12] Rodrigues T G, Suto K, Nishiyama H, Kato N, Temma K. Cloudlets activation scheme for scalable mobile edge computing with transmission power control and virtual machine migration. IEEE Transactions on Computers, 2018, 67(9): 1287-1300. DOI: 10.1109/TC.2018.2818144.
[13] Roman R, Lopez J, Mambo M. Mobile edge computing, fog et al.}: A survey and analysis of security threats and challenges. Future Generation Computer Systems, 2018, 78: 680-698. DOI: 10.1016/j.future.2016.11.009.
[14] Ning Z, Liao J, Zhang F, Shi W. Preliminary study of trusted ution environments on heterogeneous edge platforms. In Proc. the 2018 IEEE/ACM Symposium on Edge Computing, Dec. 2018, pp.421-426. DOI: 10.1109/SEC.2018.00057.
[15] Costan V, Devadas S. Intel SGX explained. IACR Cryptol. ePrint Arch., 2016, 2016: Article No. 86.
[16] Kaplan D, Powell J, Woller T. AMD memory encryption. https://developer.amd.com/wordpress/media/2013/12/A- MD\_Memory\_ Encryption\_ Whitepaper\_v7-Public.pdf, Dec. 2021.
[17] Ngabonziza B, Martin D, Bailey A, Cho H, Martin S. TrustZone explained: Architectural features and use cases. In Proc. the 2nd IEEE International Conference on Collaboration and Internet Computing, Nov. 2016, pp.445-451. DOI: 10.1109/CIC.2016.065.
[18] Kim T, Park J, Woo J, Jeon S, Huh J. ShieldStore: Shielded in-memory key-value storage with SGX. In Proc. the 14th EuroSys Conference 2019, Mar. 2019, Article No. 14. DOI: 10.1145/3302424.3303951.
[19] Arnautov S, Trach B, Gregor F et al. SCONE: Secure Linux containers with intel SGX. In Proc. the 12th USENIX Symposium on Operating Systems Design and Implementation, Nov. 2016, pp.689-703.
[20] Priebe C, Vaswani K, Costa M. EnclaveDB: A secure database using SGX. In Proc. the 2018 IEEE Symposium on Security and Privacy, May 2018, pp.264-278. DOI: 10.1109/SP.2018.00025.
[21] Tsai C C, Porter D E, Vij M. Graphene-SGX: A practical library OS for unmodified applications on SGX. In Proc. the 2017 USENIX Annual Technical Conference, July 2017, pp.645-658.
[22] Barbalace A, Lyerly R, Jelesnianski C, Carno A, Chuang H R, Legout V, Ravindran B. Breaking the boundaries in heterogeneous-ISA datacenters. ACM SIGARCH Computer Architecture News, 2017, 45(1): 645-659. DOI: 10.1145/3093337.3037738.
[23] Barbalace A, Sadini M, Ansary S, Jelesnianski C, Ravichandran A, Kendir C, Murray A, Ravindran B. Popcorn: Bridging the programmability gap in heterogeneous-ISA platforms. In Proc. the 10th European Conference on Computer Systems, Apr. 2015, Article No. 29. DOI: 10.1145/2741948.2741962.
[24] Gu J, Hua Z, Xia Y, Chen H, Zang B, Guan H, Li J. Secure live migration of SGX enclaves on untrusted cloud. In Proc. the 47th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, June 2017, pp.225-236. DOI: 10.1109/DSN.2017.37.
[25] Choy S, Wong B, Simon G, Rosenberg C. The brewing storm in cloud gaming: A measurement study on cloud to end-user latency. In Proc. the 11th Annual Workshop on Network and Systems Support for Games, Nov. 2012. DOI: 10.1109/NetGames.2012.6404024.
[26] Furlong M, Quinn A, Flinn J. The case for determinism on the edge. In Proc. the 2nd USENIX Workshop on Hot Topics in Edge Computing, July 2019.
[27] Ha K, Abe Y, Eiszler T, Chen Z, Hu W, Amos B, Upadhyaya R, Pillai P, Satyanarayanan M. You can teach elephants to dance: Agile VM handoff for edge computing. In Proc. the 2nd ACM/IEEE Symposium on Edge Computing, Oct. 2017, Article No. 12. DOI: 10.1145/3132211.3134453.
[28] Nadgowda S, Suneja S, Bila N, Isci C. Voyager: Complete container state migration. In Proc. the 37th IEEE International Conference on Distributed Computing Systems, June 2017, pp.2137-2142. DOI: 10.1109/ICDCS.2017.91.
[29] Jamshidi P, Ahmad A, Pahl C. Cloud migration research: A systematic review. IEEE Transactions on Cloud Computing, 2013, 1(2): 142-157. DOI: 10.1109/TCC.2013.10.
[30] Zhu J, Hou R, Wang X et al. Enabling rack-scale confidential computing using heterogeneous trusted ution environment. In Proc. the 2020 IEEE Symposium on Security and Privacy, May 2020, pp.1450-1465. DOI: 10.1109/SP40000.2020.00054.
[31] Hua Z, Gu J, Xia Y, Chen H, Zang B, Guan H. vTZ: Virtualizing ARM TrustZone. In Proc. the 26th USENIX Security Symposium, Aug. 2017, pp.541-556.
[32] Nightingale E B, Hodson O, McIlroy R, Hawblitzel C, Hunt G. Helios: Heterogeneous multiprocessing with satellite kernels. In Proc. the 22nd ACM SIGOPS Symposium on Operating Systems Principles, Oct. 2009, pp.221-234. DOI: 10.1145/1629575.1629597.
[33] Piraghaj S F, Dastjerdi A V, Calheiros R N, Buyya R. A framework and algorithm for energy efficient container consolidation in cloud data centers. In Proc. the 2015 IEEE International Conference on Data Science and Data Intensive Systems, Dec. 2015, pp.368-375. DOI: 10.1109/DSDIS.2015.67.
[34] Wang H, Shi P, Zhang Y. JointCloud: A cross-cloud cooperation architecture for integrated internet service customization. In Proc. the 37th IEEE International Conference on Distributed Computing Systems, June 2017, pp.1846-1855. DOI: 10.1109/ICDCS.2017.237.
[35] Baumann A, Peinado M, Hunt G. Shielding applications from an untrusted cloud with Haven. ACM Transactions on Computer Systems, 2015, 33(3): Article No. 8. DOI: 10.1145/2799647.
[36] Hunt T, Zhu Z, Xu Y, Peter S, Witchel E. Ryoan: A distributed sandbox for untrusted computation on secret data. In Proc. the 12th USENIX Symposium on Operating Systems Design and Implementation, Nov. 2016, pp.533-549.
[37] Ohrimenko O, Costa M, Fournet C, Nowozin S, Mehta A, Schuster F, Vaswani K. SGX-enabled oblivious machine learning. https://dl.acm.org/doi/10.5555/3241094.3241143 Oblivious multi-party machine learning on trusted processors, 2016.
[38] Shinde S, Le Tien D, Tople S, Saxena P. Panoply: Low-TCB Linux applications with SGX enclaves. In Proc. the 24th Annual Network and Distributed System Security Symp., Feb. 26-Mar. 1, 2017. DOI: 10.14722/ndss.2017.23500.
[39] Schuster F, Costa M, Fournet C, Gkantsidis C, Peinado M, Mainar-Ruiz G, Russinovich M. VC3: Trustworthy data analytics in the cloud using SGX. In Proc. the 2015 IEEE Symposium on Security and Privacy, May 2015, pp.38-54. DOI: 10.1109/SP.2015.10.
[40] Li M, Zhang Y, Lin Z, Solihin Y. Exploiting unprotected I/O operations in AMD's secure encrypted virtualization. In Proc. the 28th USENIX Security Symposium, Aug. 2019, pp.1257-1272.
[41] Morbitzer M, Huber M, Horsch J. Extracting secrets from encrypted virtual machines. In Proc. the 9th ACM Conference on Data and Application Security and Privacy, Mar. 2019, pp.221-230. DOI: 10.1145/3292006.3300022.
[42] Alves T, Felton D. TrustZone: Integrated hardware and software security. ARM White Paper, 2004, 3(4): 18-24.
[43] Sun H, Sun K, Wang Y, Jing J. TrustOTP: Transforming smartphones into secure one-time password tokens. In Proc. the 22nd ACM SIGSAC Conference on Computer and Communications Security, Oct. 2015, pp.976-988. DOI: 10.1145/2810103.2813692.
[44] Santos N, Raj H, Saroiu S, Wolman A. Using ARM TrustZone to build a trusted language runtime for mobile applications. In Proc. the 19th International Conference on Architectural Support for Programming Languages and Operating Systems, Feb. 2014, pp.67-80. DOI: 10.1145/2541940.2541949.
[45] Zhang N, Sun K, Lou W, Hou Y T. CaSE: Cache-assisted secure ution on ARM processors. In Proc. the 2016 IEEE Symposium on Security and Privacy, May 2016, pp.72-90. DOI: 10.1109/SP.2016.13.
[46] Guan L, Liu P, Xing X, Ge X, Zhang S, Yu M, Jaeger T. TrustShadow: Secure ution of unmodified applications with ARM TrustZone. In Proc. the 15th Annual International Conference on Mobile Systems, Applications, and Services, June 2017, pp.488-501. DOI: 10.1145/3081333.3081349.
[47] Zhao S, Zhang Q, Qin Y, Feng W, Feng D. SecTEE: A software-based approach to secure enclave architecture using TEE. In Proc. the 2019 ACM SIGSAC Conference on Computer and Communications Security, Nov. 2019, pp.1723-1740. DOI: 10.1145/3319535.3363205.
[48] Lind J, Priebe C, Muthukumaran D et al. Glamdring: Automatic application partitioning for Intel SGX. In Proc. the 2017 USENIX Annual Technical Conference, July 2017, pp.285-298.
[49] Soares L, Stumm M. FlexSC: Flexible system call scheduling with exception-less system calls. In Proc. the 9th USENIX Conference on Operating Systems Design and Implementation, Oct. 2010, pp.33-46.
[50] Rott J. Intel® advanced encryption standard instructions (AES-NI). https://www.intel.com/content/www/us/en/developer/articles/technical/advanced-encryption-standard-instructions-aes-ni.html, Dec. 2021.
[51] McCune J M, Li Y, Qu N, Zhou Z, Datta A, Gligor V, Perrig A. TrustVisor: Efficient TCB reduction and attestation. In Proc. the 2010 IEEE Symposium on Security and Privacy, May 2010, pp.143-158. DOI: 10.1109/SP.2010.17.
[52] Zhang F, Chen J, Chen H, Zang B. CloudVisor: Retrofitting protection of virtual machines in multi-tenant cloud with nested virtualization. In Proc. the 23rd ACM Symposium on Operating Systems Principles, Oct. 2011, pp.203-216. DOI: 10.1145/2043556.2043576.
[53] Dautenhahn N, Kasampalis T, Dietz W, Criswell J, Adve V. Nested kernel: An operating system architecture for intra-kernel privilege separation. ACM SIGPLAN Notices, 2015, 50(4): 191-206. DOI: 10.1145/2694344.2694386.
[54] Nelson L, Sigurbjarnarson H, Zhang K, Johnson D, Bornholt J, Torlak E, Wang X. Hyperkernel: Push-button verification of an OS kernel. In Proc. the 26th Symposium on Operating Systems Principles, Oct. 2017, pp.252-269. DOI: 10.1145/3132747.3132748.
[55] Klein G, Elphinstone K, Heiser G et al. sel4: Formal verification of an OS kernel. In Proc. the 22nd ACM SIGOPS Symposium on Operating Systems Principles, Oct. 2009, pp.207-220. DOI: 10.1145/1629575.1629596.
[56] Baumann A. Hardware is the new software. In Proc. the 16th Workshop on Hot Topics in Operating Systems, May 2017, pp.132-137. DOI: 10.1145/3102980.3103002.
[57] Ferraiuolo A, Baumann A, Hawblitzel C, Parno B. Komodo: Using verification to disentangle secure-enclave hardware from software. In Proc. the 26th Symposium on Operating Systems Principles, Oct. 2017, pp.287-305. DOI: 10.1145/3132747.3132782.
[58] Brasser F, Gens D, Jauernig P, Sadeghi A R, Stapf E. SANCTUARY: ARMing TrustZone with user-space enclaves. In Proc. the 26th Annual Network and Distributed System Security Symposium, Feb. 2019. DOI: 10.14722/ndss.2019.23448.
[59] Gu J, Wu X, Zhu B, Xia Y, Zang B, Guan H, Chen H. Enclavisor: A hardware-software co-design for enclaves on untrusted cloud. IEEE Transactions on Computers, 2021, 70(10): 1598-1611. DOI: 10.1109/TC.2020.3019704.
[60] Levin R, Cohen E, Corwin W, Pollack F, Wulf W. Policy/mechanism separation in hydra. In Proc. the 5th ACM Symposium on Operating Systems Principles, Nov. 1975, pp.132-140. DOI: 10.1145/800213.806531.
[61] Liedtke J. Improving IPC by kernel design. In Proc. the 14th ACM Symposium on Operating Systems Principles, Dec. 1993, pp.175-188. DOI: 10.1145/168619.168633.
[62] David F M, Chan E, Carlyle J C, Campbell R H. CuriOS: Improving reliability through operating system structure. In Proc. the 8th USENIX Conference on Operating Systems Design and Implementation, Dec. 2008, pp.59-72.
[63] Gu J, Wu X, Li W, Liu N, Mi Z, Xia Y, Chen H. Harmonizing performance and isolation in microkernels with efficient intra-kernel isolation and communication. In Proc. the 2020 USENIX Annual Technical Conference, July 2020, pp.401-417.
[64] Hildebrand D. An architectural overview of QNX. In Proc. the Workshop on Micro-Kernels and Other Kernel Architectures, Apr. 1992, pp.113-126.
[65] Ji D, Zhang Q, Zhao S, Shi Z, Guan Y. MicroTEE: Designing TEE OS based on the microkernel architecture. In Proc. the 18th IEEE International Conference on Trust, Security and Privacy in Computing and Communications/13th IEEE International Conference on Big Data Science and Engineering, Aug. 2019, pp.26-33. DOI: 10.1109/TrustCom/BigDataSE.2019.00014.
[66] DeVuyst M, Venkat A, Tullsen D M. ution migration in a heterogeneous-ISA chip multiprocessor. In Proc. the 17th International Conference on Architectural Support for Programming Languages and Operating Systems, Mar. 2012, pp.261-272. DOI: 10.1145/2150976.2151004.
[67] Gordon M S, Jamshidi D A, Mahlke S, Mao Z M, Chen X. COMET: Code offload by migrating ution transparently. In Proc. the 10th USENIX Symposium on Operating Systems Design and Implementation, Oct. 2012, pp.93-106.
No related articles found!
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
[1] Qing-Bin Liu, Shi-Zhu He, Kang Liu, Sheng-Ping Liu, Jun Zhao. A Unified Shared-Private Network with Denoising for Dialogue State Tracking[J]. Journal of Computer Science and Technology, 2021, 36(6): 1407 -1419 .
[2] . Online First Under Construction [J]. Journal of Computer Science and Technology, 0, (): 1 .
[3] Dan-Hao Zhu, Xin-Yu Dai, Jia-Jun Chen. Pre-Train and Learn: Preserving Global Information for Graph Neural Networks[J]. Journal of Computer Science and Technology, 2021, 36(6): 1420 -1430 .
[4] Bo Chen, Liang Liu, Hua-Dong Ma. CDM: Content Diffusion Model for Information-Centric Networks[J]. Journal of Computer Science and Technology, 2021, 36(6): 1431 -1451 .
[5] Li Zhang, Jia-Hao Tian, Jing Jiang, Yi-Jun Liu, Meng-Yuan Pu, Tao Yue. Empirical Research in Software Engineering-A Literature Survey[J]. Journal of Computer Science and Technology, 2018, 33(5): 876 -899 .
[6] Xue-Jun Yang (杨学军), Senior Member, CCF, Member, ACM, IEEE, Xiang-Ke Liao (廖湘科), Senior Member CCF, Member, ACM, Kai Lu . The TianHe-1A Supercomputer: Its Hardware and Software[J]. , 2011, 26(3): 344 -351 .
[7] Zhi-Neng Chen, Chong-Wah Ngo, Wei Zhang, Juan Cao, Yu-Gang Jiang. Name-Face Association in Web Videos: A Large-Scale Dataset, Baselines, and Open Issues[J]. , 2014, 29(5): 785 -798 .
[8] Fei Xia, De-Jun Jiang, Jin Xiong, Ning-Hui Sun. A Survey of Phase Change Memory Systems[J]. , 2015, 30(1): 121 -144 .
[9] Xue-Yan Wang, Qiang Zhou, Yi-Ci Cai, Gang Qu. Spear and Shield: Evolution of Integrated Circuit Camouflaging[J]. , 2018, 33(1): 42 -57 .
[10] Xianzhi Wang, Chaoran Huang, Lina Yao, Boualem Benatallah, Manqing Dong. A Survey on Expert Recommendation in Community Question Answering[J]. Journal of Computer Science and Technology, 2018, 33(4): 625 -653 .

ISSN 1000-9000(Print)

         1860-4749(Online)
CN 11-2296/TP

 Home
Editorial Board
Author Guidelines
Subscription		 Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
Tel.:86-10-62610746
E-mail: jcst@ict.ac.cn
 
  Copyright ©2015 JCST, All Rights Reserved