计算机科学技术学报 ›› 2018,Vol. 33 ›› Issue (6): 1307-1319.doi: 10.1007/s11390-018-1889-1

所属专题: Artificial Intelligence and Pattern Recognition Data Management and Data Mining

• • 上一篇    下一篇

一种使用上下文自动编码器进行情感分析的新方法

Hanen Ameur, Salma Jamoussi, Abdelmajid Ben Hamadou   

  1. Multimedia Information Systems and Advanced Computing Laboratory, Sfax University, Sfax Technopole Sfax 3021, Tunisia
  • 收稿日期:2017-07-05 修回日期:2018-07-11 出版日期:2018-11-15 发布日期:2018-11-15
  • 作者简介:Hanen Ameur is a Ph.D. student in the Sfax University, Sfax. She obtained her Master's degree from the Higher Institute of Computer Sciences and Multimedia, Sfax, in 2013. She is currently a research member in the Multimedia Information Systems and Advanced Computing Laboratory (MIRACL). Her research is centered on sentiment analysis, opinion mining, data mining, and social media science.

A New Method for Sentiment Analysis Using Contextual Auto-Encoders

Hanen Ameur, Salma Jamoussi, Abdelmajid Ben Hamadou   

  1. Multimedia Information Systems and Advanced Computing Laboratory, Sfax University, Sfax Technopole Sfax 3021, Tunisia
  • Received:2017-07-05 Revised:2018-07-11 Online:2018-11-15 Published:2018-11-15
  • About author:Hanen Ameur is a Ph.D. student in the Sfax University, Sfax. She obtained her Master's degree from the Higher Institute of Computer Sciences and Multimedia, Sfax, in 2013. She is currently a research member in the Multimedia Information Systems and Advanced Computing Laboratory (MIRACL). Her research is centered on sentiment analysis, opinion mining, data mining, and social media science.

情感分析,对理解用户在线表达的观点和看法提出了新的挑战。它们旨在通过分配给用户一个极性标签对主观文本进行分类。在此,我们介绍了一个通过使用自动编码器网络推测词语和句子层次的情感极性标签的新机器学习框架。受自动编码器的降维和特征提取能力的启发,我们通过将pointwise-mutual-information "PMI"词向量作为输入,提出了一个分布式词向量表征"PMI-SA"新模型。所得到的连续词向量结合表征一个句子。我们也发了一个用于学习句子表达的非监督句子嵌入方法:为上下文递归自动编码器"CoRAE"。CoRAE遵循递归自动编码器的基本观点,组合成构成句子的词的向量,但不依赖任何词法解析树。CoRAE模型通过考虑词的顺序,递归地将每个词和它的上下文词(邻词:其前的和其后的)组合起来。我们使用了一个具有微调技术的支持向量机分类器,以显示CoRAE提高了情感分析任务的准确度。实验结果表明CoRAE在Sanders推特数据库和Facebook评论数据库上显著优于其它几个竞争力强的基准方法。CoRAE模型在Facebook数据集到达了83.28%的效率,并且在Sanders数据集到达了97.57%的效率。

关键词: 情感分析, 递归自动编码器, 堆栈自动编码器, 点状互信息(pointwise-mutual-information), 深度嵌入表示

Abstract: Sentiment analysis, a hot research topic, presents new challenges for understanding users' opinions and judgments expressed online. They aim to classify the subjective texts by assigning them a polarity label. In this paper, we introduce a novel machine learning framework using auto-encoders network to predict the sentiment polarity label at the word level and the sentence level. Inspired by the dimensionality reduction and the feature extraction capabilities of the auto-encoders, we propose a new model for distributed word vector representation "PMI-SA" using as input pointwisemutual-information "PMI" word vectors. The resulted continuous word vectors are combined to represent a sentence. An unsupervised sentence embedding method, called Contextual Recursive Auto-Encoders "CoRAE", is also developed for learning sentence representation. Indeed, CoRAE follows the basic idea of the recursive auto-encoders to deeply compose the vectors of words constituting the sentence, but without relying on any syntactic parse tree. The CoRAE model consists in combining recursively each word with its context words (neighbors' words:previous and next) by considering the word order. A support vector machine classifier with fine-tuning technique is also used to show that our deep compositional representation model CoRAE improves significantly the accuracy of sentiment analysis task. Experimental results demonstrate that CoRAE remarkably outperforms several competitive baseline methods on two databases, namely, Sanders twitter corpus and Facebook comments corpus. The CoRAE model achieves an efficiency of 83.28% with the Facebook dataset and 97.57% with the Sanders dataset.

Key words: sentiment analysis, recursive auto-encoder, stacked auto-encoder, pointwise mutual information, deep embedding representation

[1] Fu X, Xu Y. Recursive autoencoder with HowNet lexicon for sentence-level sentiment analysis. In Proc. ASE BigData and Social Informatics, Oct. 2015, Article No. 20.
[2] Ameur H, Jamoussi S. Dynamic construction of dictionaries for sentiment classification. In Proc. the 13th IEEE International Conference on Data Mining Workshops, Dec. 2013, pp.896-903.
[3] Cambria E, Schuller B, Xia Y, Havasi C. New avenues in opinion mining and sentiment analysis. IEEE Intelligent Systems, 2013, 28(2):15-21.
[4] Socher R, Perelygin A, Wu J, Chuang J, Manning C D, Andrew Y N, Christopher P. Recursive deep models for semantic compositionality over a sentiment treebank. In Proc. Conference on Empirical Methods in Natural Language Processing, Oct. 2013, pp.1631-1642.
[5] Yin H, Zhang C, Zhu Y, Ji Y. Representing sentence with unfolding recursive autoencoders and dynamic average pooling. In Proc. IEEE International Conference on Data Science and Advanced Analytics, Oct. 2014, pp.413-419.
[6] Ameur H, Jamoussi S, Hamadou A B. Sentiment lexicon enrichment using emotional vector representation. In Proc. the 14th IEEE/ACS International Conference on Computer Systems and Applications, Oct. 2017, pp.951-958.
[7] Rong W, Nie Y, Ouyang Y, Peng B, Xiong Z. Auto-encoder based bagging architecture for sentiment analysis. Journal of Visual Languages and Computing, 2014, 25(6):840-849.
[8] Pang B, Lee L, Vaithyanathan S. Thumbs up? Sentiment classification using machine learning techniques. In Proc. ACL-02 Conference on Empirical Methods in Natural Language Processing, Volume 10, Jul. 2002, pp.79-86.
[9] Bengio Y. Learning deep architectures for AI. Foundations and Trendsr in Machine Learning, 2009, 2(1):1-127.
[10] Blacoe W, Lapata M. A comparison of vector-based representations for semantic composition. In Proc. Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Jul. 2012, pp.546-556.
[11] Socher R, Pennington J, Huang E H, Ng A Y, Manning C D. Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proc. the 11th Conference on Empirical Methods in Natural Language Processing, Jul. 2011, pp.151-161.
[12] Poirier D. Des textes communautaires à la recommandation[Ph.D. Thesis], Orleans University, 2011. (in French)
[13] Martineau J, Finin T. Delta TFIDF:An improved feature space for sentiment analysis. In Proc. the 3rd AAAI International Conference on Weblogs and Social Media, May 2009, pp.258-261.
[14] Turney P D., Pantel P. From frequency to meaning:Vector space models of semantics. Journal of Artificial Intelligence Research, 2010, 37(1):141-188.
[15] Chen L. Curse of dimensionality. In Encyclopedia of Database Systems, Liu L, Özsu M T (eds.), Springer, 2009, pp.545-546.
[16] Mikolov T, Yih S W, Zweig G. Linguistic regularities in continuous space word representations. In Proc. the 2013 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, May 2013, pp.746-751.
[17] Zhang P, Komachi M. Japanese sentiment classification with stacked denoising autoencoder using distributed word representation. In Proc. the 29th Pacific Asia Conference on Language, Information and Computation, Oct. 2015, pp.150-159.
[18] Cho K, van Merrienboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y. Learning phrase representations using RNN encoder decoder for statistical machine translation. In Proc. the 2014 Conference on Empirical Methods in Natural Language Processing, Oct. 2014, pp.1724-1734.
[19] Zhang Y, Er M J, Venkatesan R, Wang N, Pratama M. Sentiment classification using comprehensive attention recurrent models. In Proc. International Joint Conference on Neural Networks, July 2016, pp.1562-1569.
[20] Kim Y. Convolutional neural networks for sentence classification. In Proc. the 2014 Conference on Empirical Methods in Natural Language Processing, Oct. 2014, pp.1746-1751.
[21] Severyn A, Moschitti A. Twitter sentiment analysis with deep convolutional neural networks. In Proc. the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, Aug. 2015, pp.959-962.
[22] Sun X, Li C, Ren F. Sentiment analysis for Chinese microblog based on deep neural networks with convolutional extension features. Neurocomputing, 2016, 210:227-236.
[23] Zhao R, Mao K. Topic-aware deep compositional models for sentence classification. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2017, 25(2):248-260.
[24] Bengio Y. Deep learning of representations:Looking forward. arXiV:1305.0445, 2013. https://arXiv.org/abs/1305.0445/, May 2018.
[25] Kumar V. Sentiment analysis using semi-supervised recursive autoencoder. In Proc. the International Joint Conferences on Web Intelligence and Intelligent Agent Technologies, December 2015.
[26] Tang, Y. Deep learning using support vector machines. arXiV:1306.0239, 2013. https://arXiv.org/abs/1306.0239V/, May 2018.
[27] Ebert S, Vu T N, Schütze H. CIS-positive:A combination of convolutional neural networks and support vector machines for sentiment analysis in Twitter. In Proc. the 9th International Workshop on Semantic Evaluation, Jun. 2015, pp.527-532.
[28] Turian J, Ratinov L, Bengio Y. Word representations:A simple and general method for semi-supervised learning. In Proc. the 48th Annual Meeting of the Association for Computational Linguistics, Jul. 2010, pp.384-394.
[1] Fei Jiang, Yi-Qun Liu, Huan-Bo LuanJia-Shen Sun, Xuan Zhu, Min Zhang, Shao-Ping Ma. 利用表情符空间进行微博情感分析[J]. , 2015, 30(5): 1120-1129.
[2] Lei Fang, Biao Liu, Min-Lie Huang. 基于弱监督信息和大量数据抽取评论的特征词和情感词[J]. , 2015, 30(4): 903-916.
[3] Luke Kien-Weng Tan (陈坚永), Jin-Cheon Na (罗镇川), Member, ACM, Yin-Leng Theng (邓燕玲),. 一种运用规则类型依赖与复杂短语分析的词语情感极性分类方法[J]. , 2012, 27(3): 650-666.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 刘明业; 洪恩宇;. Some Covering Problems and Their Solutions in Automatic Logic Synthesis Systems[J]. , 1986, 1(2): 83 -92 .
[2] 陈世华;. On the Structure of (Weak) Inverses of an (Weakly) Invertible Finite Automaton[J]. , 1986, 1(3): 92 -100 .
[3] 高庆狮; 张祥; 杨树范; 陈树清;. Vector Computer 757[J]. , 1986, 1(3): 1 -14 .
[4] 陈肇雄; 高庆狮;. A Substitution Based Model for the Implementation of PROLOG——The Design and Implementation of LPROLOG[J]. , 1986, 1(4): 17 -26 .
[5] 黄河燕;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[6] 闵应骅; 韩智德;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[7] 唐同诰; 招兆铿;. Stack Method in Program Semantics[J]. , 1987, 2(1): 51 -63 .
[8] 闵应骅;. Easy Test Generation PLAs[J]. , 1987, 2(1): 72 -80 .
[9] 朱鸿;. Some Mathematical Properties of the Functional Programming Language FP[J]. , 1987, 2(3): 202 -216 .
[10] 李明慧;. CAD System of Microprogrammed Digital Systems[J]. , 1987, 2(3): 226 -235 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持:support@magtech.com.cn
总访问量: