基于自监督任务增强的小样本意图识别
Self-Supervised Task Augmentation for Few-Shot Intent Detection
-
摘要: 1、研究背景(context):近年来,元学习已经成为解决小样本学习问题的主流范式。但这些方法的卓越性能依赖于足够的元训练任务。在元训练任务不足的情况下,元学习器可能会过拟合这些元训练任务,这限制了它们的泛化能力。不幸的是,上述问题在小样本意图识别应用中更为明显。这是因为现实场景中有很多意图,为每个意图收集足够的数据并构造大量元训练任务是不切实际的。
2、目的(Objective):本章提出了一种新颖的基于自监督任务增强模型——STAM,该模型通过在元训练阶段生成大量的元训练任务,并添加两个额外的学习目标,将自监督学习与元学习相结合,以提高元学习在元任务不足情况下的泛化能力。
3、方法(Method):STAM 模型先是通过静态增强和动态增强生成语义相似但具有不同特征的元训练任务。除了任务增强策略外,我们还在STAM模型中添加两个辅助损失将自监督学习集成到元学习中。目标是利用数据本身的结构信息作为监督信号,学习到可迁移的特征,以提高元学习在元任务不足场景下的泛化能力。
4、结果(Result & Findings):本文在4个数据集上进行了实验,实验结果发现本文提出的STAM与其他先进的方法相比,STAM具有更好的性能。具体来说,相对于每个数据集的最佳基线,STAM在1-shot和5-shot下的平均准确率,分别提高了1.3%和2.2%。
5、结论(Conclusions):本文提出了一种新的STAM模型来克服元训练任务不足导致的潜在过拟合问题。通过广泛的实验,表明本文所提出的STAM模型在四个公开数据集上实现了最先进的性能。在未来的工作中,我们将探索 STAM模型在其他领域的有效性。Abstract: Few-shot intent detection is a practical challenge task, because new intents are frequently emerging and collecting large-scale data for them could be costly. Meta-learning, a promising technique for leveraging data from previous tasks to enable efficient learning of new tasks, has been a popular way to tackle this problem. However, the existing meta-learning models have been evidenced to be overfitting when the meta-training tasks are insufficient. To overcome this challenge, we present a novel self-supervised task augmentation with meta-learning framework, namely STAM. Firstly, we introduce the task augmentation, which explores two different strategies and combines them to extend meta-training tasks. Secondly, we devise two auxiliary losses for integrating self-supervised learning into meta-learning to learn more generalizable and transferable features. Experimental results show that STAM can achieve consistent and considerable performance improvement to existing state-of-the-art methods on four datasets.