HUITWU:一个在事务数据库中有效挖掘高效用项集的算法
HUITWU: An Efficient Algorithm for High-Utility Itemset Mining in Transaction Databases
-
摘要: 从事务数据库中挖掘高效用项集是指寻找所有具有高效用属性的项集,例如具有高利润的项集。大部分存在的研究方法从事务数据库中以两个阶段寻找高效用项集。第一阶段,采用不同的高估技术,计算项集效用的上届。选择高估效用不小于用户指定阈值的项集做为候选高效用项集。第二阶段,对以第一阶段产生的候选高效用项集扫描数据库,计算候选高效用项集的精确效用,从而获得所有高效用项集。可是,候选高效用项集数量巨大,招致如下两个问题:1)大量的内存用于存储候选高效用项集;2)计算候选高效用项集的精确效用耗时巨大。最近垂直数据格式应用于高效用项集挖掘。可是这类方法不能有效的处理具有相同项的事务,从而导致数据库尺寸不能充分的减小,进而影响算法的性能。因此本文提出一个新算法HUITWU挖掘高效用项集。采用一种新的树结构HUITWU-Tree计算数据库中项集的效用。我们在稀松数据集和稠密数据集上进行了大量的实验,实验结果表明我们提出的算法在运行时间上比当前最好的算法快一个数量级,同时在内存使用方面少于当前最好的算法。Abstract: Mining high-utility itemsets (HUIs) from a transaction database refers to the discovery of itemsets with high utilities like profits. Most of existing studies discover HUIs from a transaction database in two phases. In phase 1, different overestimation methods are applied to calculate the upper bounds of the utilities of itemsets. Since the overestimated utilities of itemsets are adopted, the itemsets whose overestimated utilities are no less than a user-specified threshold are selected as candidate HUIs, and they are verified by scanning the database one more time in phase 2. However, a large number of candidate HUIs incur two problems:1) it requires excessive memory to store these candidates; 2) it needs a large amount of running time to calculate their exact utilities. Vertical data format has been applied to mine HUIs recently. However this kind of method cannot deal with transactions with the same items effectively so that the size of database cannot be reduced sufficiently. The overall performance of algorithms is degraded consequently. Thus an algorithm HUITWU is proposed in this paper for mining HUIs. A novel data structure HUITWU-Tree is adopted to efficiently calculate the utilities of itemsets in a database. Extensive studies with both sparse and dense datasets have demonstrated that our proposed algorithm is more than an order of magnitude faster and consumes less memory than the state-of-the-art algorithms.