HmTest: Automated Testing of HarmonyOS Apps via Model-Driven Navigation and Reinforcement Learning

HarmonyOS is a new all-scenario operating system for smart devices. As its software ecosystem rapidly expands, how to conduct automated testing of HarmonyOS apps for quality assurance has become a crucial task. This paper presents HmTest, an automated testing framework for HarmonyOS apps, which consists of two complementary modules: Targeted Exploration and RL-based Exploration. Targeted Exploration performs white-box testing, leveraging static analysis to construct a page transition graph (PTG). By systematically traversing PTGs to guide testing, HmTest can quickly achieve high page coverages. On the other hand, RL-based Exploration performs black-box testing, utilizing reinforcement learning to achieve a comprehensive exploration of app states. Additionally, an automaton-based mechanism is employed to efficiently recover and restart the testing process when reinforcement learning encounters stagnation. We have evaluated HmTest on nine HarmonyOS NEXT apps and compared it with two official HarmonyOS app testing tools. The experimental results demonstrate that Targeted Exploration can generate highly-precise PTGs and help achieve high page coverages within a few minutes. RL-based Exploration can significantly outperform other methods in terms of finer-grained statement coverage on the majority of the tested apps and benefits from the recovery mechanism. To facilitate future research, we have made HmTest open-source at https://github.com/sqlab-sustech/hmtest and provided a video demo at https://youtu.be/HE6ku3Elc2U.