基于新类增强自蒸馏的联邦增量学习
Federated Class-Incremental Learning with New-Class Augmented Self-Distillation
-
摘要:研究背景 随着数据隐私保护意识的增强,联邦学习成为在保障数据隐私的前提下进行协作式模型训练的重要技术。然而,现有主流联邦学习方法普遍假设参与方的数据集是静态的,忽略了现实中数据量和类别不断扩展的动态特性。这导致联邦学习在新类别数据引入时出现灾难性遗忘问题,即模型在学习新知识时丧失对旧知识的识别能力,严重影响整体性能。目的 本研究旨在解决联邦学习中因数据类别动态扩展而引发的灾难性遗忘问题。作者提出了一种新颖的联邦类增量学习方法FedCLASS,通过增强历史模型缺失的新类别信息,实现更精准的知识迁移,提升模型在多任务、动态数据环境下的整体性能和稳定性。方法 3、FedCLASS提出了新类别增强自蒸馏机制,即在自蒸馏过程中将当前模型预测的新类别得分与历史模型的旧类别得分结合,构建完整的类别概率分布。具体方法包括:基于条件概率建模,分别处理旧类别和新类别分数;设计新类别增强的损失函数,兼顾交叉熵和Kullback-Leibler散度;理论上证明了FedCLASS的合理性和可靠性。结果 实验证明FedCLASS在四个数据集和不同增量任务设置下,全局准确率与平均遗忘率均优于现有基线方法。结论 FedCLASS有效缓解了联邦类增量学习中的灾难性遗忘问题,从理论上建立了增量新类别与历史知识结合的概率建模框架。通过新类别增强自蒸馏,FedCLASS实现了更完整的知识迁移,显著提升了模型的全局性能。Abstract: Federated learning (FL) enables collaborative model training among participants while guaranteeing the privacy of raw data. Mainstream FL methodologies overlook the dynamic nature of real-world data, particularly its tendency to grow in volume and diversify in classes over time. This oversight results in FL methods suffering from catastrophic forgetting, where the trained models inadvertently discard previously learned information upon assimilating new data. In response to this challenge, we propose a novel federated class-incremental learning (FCIL) method, named Federated Class-incremental Learning with New-Class Augmented Self-Distillation (FedCLASS). The core of FedCLASS is to enrich the class scores of historical models with new class scores predicted by current models and utilize the combined knowledge for self-distillation, enabling a more sufficient and precise knowledge transfer from historical models to current models. Theoretical analyses demonstrate that FedCLASS stands on reliable foundations, considering the scores of old classes predicted by historical models as conditional probabilities in the absence of new classes, and the scores of new classes predicted by current models as the conditional probabilities of class scores derived from historical models. Empirical experiments demonstrate the superiority of FedCLASS over four baseline algorithms in reducing average forgetting rate and boosting global accuracy.
下载: