Image sentiment classification, which aims to predict the polarities of sentiments conveyed by the images, has gained a lot of attention. Most existing methods address this problem by training a general classifier with certain visual features, ignoring the discrepancies across domains. In this paper, we propose a novel weighted co-training method for cross-domain image sentiment classification, which iteratively enlarges the labeled set by introducing new high-confidence classified samples to reduce the gap between the two domains. We train two sentiment classifiers with both the images and the corresponding textual comments separately, and set the similarity between the source domain and the target domain as the weight of a classifier. We perform extensive experiments on a real Flickr dataset to evaluate the proposed method, and the empirical study reveals that the weighted co-training method significantly outperforms some baseline solutions.
This work was supported in part by the National Basic Research 973 Program of China under Grant No. 2015CB352502, the National Natural Science Foundation of China under Grant Nos. 61272092 and 61572289, the Natural Science Foundation of Shandong Province of China under Grant No. ZR2015FM002, the Science and Technology Development Program of Shandong Province of China under Grant No. 2014GGE27178, and the NSERC (Natural Sciences and Engineering Research Council of Canada) Discovery Grants.
通讯作者: Yang Liu
About author: Meng Chen received his Ph.D. degree in computer science and technology in 2016 from Shandong University, Jinan. He is currently a postdoctoral fellow in the School of Information Technology, York University, Toronto. His research interest is in the area of data mining.
Meng Chen, Lin-Lin Zhang, Xiaohui Yu, Yang Liu.基于加权协同训练的跨领域图像情感分类[J] Journal of Computer Science and Technology , 2017,V32(4): 714-725
Meng Chen, Lin-Lin Zhang, Xiaohui Yu, Yang Liu.Weighted Co-Training for Cross-Domain Image Sentiment Classification[J] Journal of Computer Science and Technology, 2017,V32(4): 714-725
 Siersdorfer S, Minack E, Deng F, Hare J. Analyzing and predicting sentiment of images on the social web. In Proc. International Conference on Multimedia, October 2010, pp.715-718. Machajdik J, Hanbury A. Affective image classification using features inspired by psychology and art theory. In Proc. International Conference on Multimedia, October 2010, pp.83-92. Borth D, Chen T, Ji R, Chang S F. SentiBank:Large-scale ontology and classifiers for detecting sentiment and emotions in visual content. In Proc. International Conference on Multimedia, October 2013, pp.459-460. Borth D, Ji R, Chen T, Breuel T, Chang S F. Large-scale visual sentiment ontology and detectors using adjective noun pairs. In Proc. International Conference on Multimedia, October 2013, pp.223-232. You Q, Luo J, Jin H, Yang J. Robust image sentiment analysis using progressively trained and domain transferred deep networks. arXiv:1509.06041, 2015. https://arxiv.org/pdf/1509.06041.pdf, May 2017. Pang B, Lee L, Vaithyanathan S. Thumbs up? Sentiment classification using machine learning techniques. In Proc. Conference on Empirical Methods in Natural Language Processing, July 2002, pp.79-86. Baccianella S, Esuli A, Sebastiani F. SentiWordNet 3.0:An enhanced lexical resource for sentiment analysis and opinion mining. In Proc. International Conference on Language Resources and Evaluation, Volume 10, May 2010, pp.2200-2204. Liao C, Feng C, Yang S, Huang H Y. A hybrid method of domain lexicon construction for opinion targets extraction using syntax and semantics. Journal of Computer Science and Technology, 2016, 31(3):595-603. Kim Y. Convolutional neural networks for sentence classification. arXiv:1408.5882, 2014. https://arxiv.org/abs/14-08.5882, May 2017. Li X, Pang J, Mo B, Rao Y, Wang F L. Deep neural network for short-text sentiment classification. In Proc. International Conference on Database Systems for Advanced Applications, April 2016, pp.168-175. Ding X, Liu B, Yu P S. A holistic lexicon-based approach to opinion mining. In Proc. International Conference on Web Search and Data Mining, February 2008, pp.231-240. Jiang F, Liu Y Q, Luan H B, Sun J S, Zhu X, Zhang M, Ma S P. Microblog sentiment analysis with emoticon space model. Journal of Computer Science and Technology, 2015, 30(5):1120-1129. Liu B, Zhang L. A survey of opinion mining and sentiment analysis. In Mining Text Data, Aggarwal C C, Zhai C X (eds.), Spinger, 2012, pp.415-463. Tang D, Qin B, Liu T. Document modeling with gated recurrent neural network for sentiment classification. In Proc. the Conference on Empirical Methods in Natural Language Processing, September 2015, pp.1422-1432. Campos V, Salvador A, Nieto X, Jou B. Diving deep into sentiment:Understanding fine-tuned CNNs for visual sentiment prediction. In Proc. International Workshop on Affect and Sentiment in Multimedia, October 2015, pp.57-62. Zhang Y, Shang L, Jia X. Sentiment analysis on microblogging by integrating text and image features. In Proc. Pacific-Asia Conference on Knowledge Discovery and Data Mining, May 2015, pp.52-63. Tan J, Xu M, Shang L, Jia X. Sentiment analysis for images on microblogging by integrating textual information with multiple kernel learning. In Proc. Pacific Rim International Conference on Artificial Intelligence, August 2016, pp.496-506. Xu X, Shen F, Yang Y, Shen H T, Li X. Learning discriminative binary codes for large-scale cross-modal retrieval. IEEE Transactions on Image Processing, 2017, 26(5):2494-2507. Zhu X, Huang Z, Shen H T, Zhao X. Linear cross-modal hashing for efficient multimedia search. In Proc. International Conference on Multimedia, October 2013, pp.143-152. Song J, Yang Y, Yang Y, Huang Z, Shen H T. Inter-media hashing for large-scale retrieval from heterogeneous data sources. In Proc. ACM SIGMOD International Conference on Management of Data, June 2013, pp.785-796. Zhou G, Zhou Y, Guo X, Tu X, He T. Cross-domain sentiment classification via topical correspondence transfer. Neurocomputing, 2015, 159(2):298-305. Bollegala D, Mu T, Goulermas J Y. Cross-domain sentiment classification using sentiment sensitive embeddings. Transactions on Knowledge and Data Engineering, 2016, 28(2):398-410. Zhang Y, Hu X, Li P, Li L, Wu X. Cross-domain sentiment classification-feature divergence, polarity divergence or both? Pattern Recognition Letters, 2015, 65:44-50. Kisilevich S, Rohrdant C, Keim D. "Beautiful picture of an ugly place". Exploring photo collections using opinion and sentiment analysis of user comments. In Proc. International Multiconference on Computer Science and Information Technology, October 2010, pp.419-428. Chang C C, Lin C J. LIBSVM:A library for support vector machines. IEEE Transactions on Intelligent Systems and Technology, 2011, 2(3):27. Mikolov T, Sutskever I, Chen K, Corrado G S, Dean J. Distributed representations of words and phrases and their compositionality. In Proc. Advances in Neural Information Processing Systems, December 2013, pp.3111-3119. Cao Y, Xu R, Chen T. Combining convolutional neural network and support vector machine for sentiment classification. In Proc. Social Media Processing, November 2015, pp.144-155. Chen T, Borth D, Darrell T, Chang S F. DeepSentiBank:Visual sentiment concept classification with deep convolutional neural networks. arXiv:1410.8586, 2014. https://arxiv.org/abs/1410.8586, May 2017. Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. In Proc. Advances in Neural Information Processing Systems, December 2012, pp.1097-1105. Shen H T, Jiang S, Tan K L, Huang Z, Zhou X. Speed up interactive image retrieval. The VLDB Journal, 2009, 18(1):329-343. Xu X, Liang H, Baldwin T. UNIMELB at SemEval-2016 tasks 4A and 4B:An ensemble of neural networks and a Word2Vec based model for sentiment classification. In Proc. International Workshop on Semantic Evaluation, June 2016, pp.183-189. Yu Y, Lin H, Meng J, Zhao Z. Visual and textual sentiment analysis of a microblog using deep convolutional neural networks. Algorithms, 2016, 9(2):41. Tieleman T, Hinton G. Lecture 6.5-RmsProp:Divide the gradient by a running average of its recent magnitude. COURSERA:Neural Networks for Machine Learning, 2012, 4(2).