Journal of Computer Science and Technology

   

Multimodal Interactive Network for Sequential Recommendation

Teng-Yue Han1, Peng-Fei Wang2,*, and Shao-Zhang Niu1   

  1. 1Beijing Key Laboratory of Intelligent Telecommunication Software and Multimedia, Beijing University of Posts and Telecommunications, Beijing 100876, China
    2School of Computer Science, Beijing University of Posts and Telecommunications, Beijing 100876, China

Building an effective sequential recommendation system is still a challenging task due to limited interactions among users and items. Recent work has shown the effectiveness of incorporating textual or visual information into sequential recommendation to alleviate the data sparse problem. The data sparse problem now is attracting a lot of attention in both industry and academic community. % However, to the best of our knowledge, seldom work considers interactions among modalities on sequential scenario, which is an interesting yet challenging task. However, considering interactions among modalities on sequential scenario is an interesting yet challenging task because of the multimodal heterogeneity. In this paper, we introduce a novel recommendation approach of considering both textual and visual information, namely Multimodal Interactive Network ~(MIN). The advantage of MIN lies in the fact of designing a learning framework to leverage the interactions among modalities from both the item level and the sequence level for building an efficient system. Firstly, an item-wise interactive layer based on encoder-decoder mechanism is utilized to model the item-level interactions among modalities to select the informative information. Secondly, a sequence interactive layer based on the attention strategy is designed to capture the sequence-level preference of each modality. MIN seamlessly incorporates interactions among modalities from both the item level and the sequence level for sequential recommendation. It is the first time that interactions in each modality have been explicitly discussed and utilized in sequential recommenders. Experimental results on four real-world datasets show that our approach can significantly outperform all the baselines in sequential recommendation task.


中文摘要

1、研究背景
随着移动互联网技术的飞速发展,网络数据呈现爆炸式的增长,越来越多的信息服务充斥着网络。随着网络技术的迅速普及,人们可以更加方便快捷的接触到网络中各种各样的信息,但同时这也增加了人们寻求目标信息的难度。在面临海量的互联网信息时,人们往往无法快速准确找到自己需要的信息,所以如何从海量的数据中提取出对用户有价值的信息成为研究的焦点。尤其是在互联网技术发达的当今社会,各种网络购物平台不断涌现,电子商务的规模逐步扩大,建立有效的个性化推荐系统,为用户提供更加智能化个性化的服务,具有重要的商业价值和使用意义。
2、目的
在推荐系统中,用户与商品的交互具有天然的时序性,用户的喜好和品位会随着时间的发展而变化,而这种动态特征只能通过序列推荐系统捕获。但用户与商品的交互行为是有限的,根据这些有限的交互行为建立高效的推荐系统仍然是一个具有挑战性的问题。近年来,人们发现将商品的多模态信息融入到推荐系统中对于提高推荐准确性大有助益,但是在序列推荐场景中,如何建模多模态信息之间的交互仍然是一个需要解决的问题。需要思考如何建模多模态的序列交互特征,包括单一模态内和多模态之间的序列交互特征,这些特征如何用于提升序列推荐的准确性,这些都是我们需要解决的问题。
3、方法
针对以上问题,我们提出了多模态交互网络模型。在这个模型中,我们分别建模了两种层次的模态交互特征,一是商品层次,即每个商品的文本模态和视觉模态的特征交互;二是序列层次,即序列上文本模态、视觉模态和跨模态之间的特征交互。在商品层次,我们用编解码器对齐模态间的信息;在序列层次,我们用自注意力机制建模模态内部的序列交互特征,用交互自注意力机制建模序列上跨模态的特征交互。此外,由于商品的表达与商品多模态表达之间是独立的,为了建立商品表达之间的联系,利用商品多模态数据增强商品表达的能力,我们设计了自监督学习信号提升模型的泛化能力,并将自监督学习任务作为辅助任务训练模型。最后根据得到的用户偏好的表示,对用户的下一次购物行为进行预测。
4、结果
我们在四个不同领域的数据集上进行了实验,实验结果证明在序列推荐场景下,建模商品的多模态信息之间的交互特征对于提升推荐结果的准确性有很大帮助。我们分别在亚马逊三个子领域,分别是手机、运动和户外、衣服,以及电影数据集上进行实验,评价指标采用Hit-Ratio@20和NDCG@20,实验结果表明,在手机数据集,Hit-Ratio@20提升约7.81%,NDCG@20提升约8.02%;在运动和户外数据集,Hit-Ratio@20提升约5.08%,NDCG@20提升约6.68%;在电影数据集,Hit-Ratio@20提升约4.68%,NDCG@20提升约5.24%; 在衣服数据集,Hit-Ratio@20提升约12.5%,NDCG@20提升约13.2%。
5、结论
在这篇论文中,我们提出了多模态交互网络用于序列推荐。多模态交互网络建模了序列上模态内部以及跨模态之间的特征交互,实验证明建模多模态间的特征交互有助于提升推荐系统的准确性。这个工作主要解决了多模态交互的问题,在学习多模态的表达方面,引入了一些噪声,未来我们将考虑如何对多模态的信息进行细粒度的区分,以及如何自适应的选取所需不同模态的信息。


Key words: multi-modality, interactive network, sequential recommendation

;

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] Zhou Di;. A Recovery Technique for Distributed Communicating Process Systems[J]. , 1986, 1(2): 34 -43 .
[2] Chen Shihua;. On the Structure of Finite Automata of Which M Is an(Weak)Inverse with Delay τ[J]. , 1986, 1(2): 54 -59 .
[3] Feng Yulin;. Recursive Implementation of VLSI Circuits[J]. , 1986, 1(2): 72 -82 .
[4] Jin Lan; Yang Yuanyuan;. A Modified Version of Chordal Ring[J]. , 1986, 1(3): 15 -32 .
[5] Wang Jianchao; Wei Daozheng;. An Effective Test Generation Algorithm for Combinational Circuits[J]. , 1986, 1(4): 1 -16 .
[6] Min Yinghua; Han Zhide;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[7] Huang Xuedong; Cai Lianhong; Fang Ditang; Chi Bianjin; Zhou Li; Jiang Li;. A Computer System for Chinese Character Speech Input[J]. , 1986, 1(4): 75 -83 .
[8] Lu Xuemiao;. On the Complexity of Induction of Structural Descriptions[J]. , 1987, 2(1): 12 -21 .
[9] Wu Yunzeng;. On the Development of Applications of Logic in Programming[J]. , 1987, 2(1): 30 -34 .
[10] Tang Tonggao; Zhao Zhaokeng;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .

ISSN 1000-9000(Print)

         1860-4749(Online)
CN 11-2296/TP

Home
Editorial Board
Author Guidelines
Subscription
Journal of Computer Science and Technology
Institute of Computing Technology, Chinese Academy of Sciences
P.O. Box 2704, Beijing 100190 P.R. China
Tel.:86-10-62610746
E-mail: jcst@ict.ac.cn
 
  Copyright ©2015 JCST, All Rights Reserved