›› 2016,Vol. 31 ›› Issue (6): 1151-1160.doi: 10.1007/s11390-016-1689-4

所属专题: Artificial Intelligence and Pattern Recognition

• Special Section on Selected Paper from NPC 2011 • 上一篇    下一篇

面向问题检索的层级自训练张量神经网络模型

包新启, 吴云芳   

  1. Key Laboratory of Computational Linguistics, Peking University, Beijing 100871, China
  • 收稿日期:2015-07-29 修回日期:2016-05-27 出版日期:2016-11-05 发布日期:2016-11-05
  • 通讯作者: Xin-Qi Bao, Yun-Fang Wu E-mail:yikusitian1990@163.com;wuyf@pku.edu.cn
  • 作者简介:Xin-Qi Bao is currently a Master student at Peking University, Beijing. He received his B.S.degree in electronic engineering and computer science from Peking University, Beijing, in 2013.

A Tensor Neural Network with Layerwise Pretraining: Towards Effective Answer Retrieval

Xin-Qi Bao, Yun-Fang Wu*   

  1. Key Laboratory of Computational Linguistics, Peking University, Beijing 100871, China
  • Received:2015-07-29 Revised:2016-05-27 Online:2016-11-05 Published:2016-11-05
  • Supported by:

    This work is supported by the National High Technology Research and Development 863 Program of China under Grant No. 2015AA015403, the National Natural Science Foundation of China under Grant Nos. 61371129 and 61572245, and the Key Program of Social Science Foundation of China under Grant No. 12&ZD227.

这篇文章研究了社区问答中的问题检索问题。提出了张量模型去更好地捕捉问题和答案之前的关联。问题和候选答案分别映射到不同的隐层语义空间,继而用一个三维张量去构建隐层语义空间中的问题答案关联。为了更好地初始化张量网络的各个层级,创新性地提出了张量降噪自动编码算法(denoising tensor autoencoder ,DTAE)。在词向量层使用降噪自动编码算法,而在张量层使用张量降噪自动编码算法,由此实现了对张量网络的层级敏感的自训练策略。实现结果表明,本文提出的张量网络模型超越了其他神经网络方法,本文提出的张量降噪自动编码算法显著提升了系统的性能和健壮性。

Abstract: In this paper we address the answer retrieval problem in community-based question answering.To fully capture the interactions between question-answer pairs,we propose an original tensor neural network to model the relevance between them.The question and candidate answers are separately embedded into different latent semantic spaces,and a 3-way tensor is then utilized to model the interactions between latent semantics.To initialize the network layers properly,we propose a novel algorithm called denoising tensor autoencoder (DTAE),and then implement a layerwise pretraining strategy using denoising autoencoders (DAE) on word embedding layers and DTAE on the tensor layer.The experimental results show that our tensor neural network outperforms various baselines with other competitive neural network methods,and our pretraining DTAE strategy improves the system's performance and robustness.

[1] Wang B, Liu B, Wang X, Sun C, Zhang D. Deep learning approaches to semantic relevance modeling for Chinese question-answer pairs. ACM Transactions on Asian Language Information Processing (TALIP), 2011, 10(4):Article No. 21.

[2] Hu H, Liu B, Wang B, Liu M, Wang X. Multimodal DBN for predicting high-quality answers in cQA portals. In Proc. the 51st Annual Meeting of the Association for Computational Linguistics, August 2013, pp.843-847.

[3] Lu Z, Li H. A deep architecture for matching short texts. In Proc. the 27th Advances in Neural Information Processing Systems, December 2013, pp.1367-1375.

[4] Hinton G, Osindero S, Teh Y W. A fast learning algorithm for deep belief nets. Neural Computation, 2006, 18(7):1527-1554.

[5] Salakhutdinov R, Hinton G. Deep Boltzmann machines. In Proc.the 12th International Conference on Artificial Intelligence and Statistics, April 2009, pp.448-455.

[6] Iyyer M, Boyd-Graber J L, Boyd J, Claudino L, Socher R, Daumé Ⅲ H. A neural network for factoid question answering over paragraphs. In Proc. the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), October 2014, pp.633-644.

[7] Bordes A, Chopra S, Weston J. Question answering with subgraph embeddings. In Proc. the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), October 2014, pp.615-620

[8] Blei D M, Ng A Y, Jordan M I. Latent Dirichlet allocation. The Journal of Machine Learning Research, 2003, 3:993-1022.

[9] Yih W T, Chang M W, Meek C, Pastusiak A. Question answering using enhanced lexical semantic models. In Proc. the 51st Annual Meeting of the Association for Computational Linguistics, August 2013, pp.1744-1753.

[10] Zhang J, Salwen J, Glass M, Gliozzo A. Word semantic representations using Bayesian probabilistic tensor factorization. In Proc. the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), October 2014, pp.1522-1531.

[11] Lei T, Xin Y, Zhang Y, Barzilay R, Jaakkola T. Low-rank tensors for scoring dependency structures. In Proc. the 52nd Annual Meeting of the Association for Computational Linguistics, June 2014, pp.1381-1391.

[12] Pei W, Ge T, Chang B. Max margin tensor neural network for Chinese word segmentation. In Proc. the 52nd Annual Meeting of the Association for Computational Linguistics, June 2014, pp.293-303.

[13] Chang K W, Yih W T, Yang B, Meek C. Typed tensor decomposition of knowledge bases for relation extraction. In Proc. the 2014 Conference on Empirical Methods in Natural Language Processing, October 2014, pp.1568-1579.

[14] Yan Z, Zhou J. A new approach to answerer recommendation in community question answering services. In Proc. the 34th Advances in Information Retrieval, April 2012, pp.121-132.

[15] Qiu X, Tian L, Huang X. Latent semantic tensor indexing for community-based question answering. In Proc. the 51st Annual Meeting of the Association for Computational Linguistics, August 2013, pp.434-439.

[16] Zhou X, Hu B, Chen Q et al. Answer sequence learning with neural networks for answer selection in community question answering. arXiv:1506.06490, 2015. http://arxiv.org/abs/1506.06490,June 2016.

[17] Qiu X, Huang X. Convolutional neural tensor network architecture for community-based question answering. In Proc. the 24th International Joint Conference on Artificial Intelligence, July 2015, pp.1305-1311.

[18] Mansur M, Pei W, Chang B. Feature-based neural language model and Chinese word segmentation. In Proc. the 6th International Joint Conference on Natural Language Processing, October 2013, pp.1271-1277.

[19] Mikolov T, Chen K, Corrado G, Dean J. Efficient estimation of word representations in vector space. arXiv:1301.3781, 2013. http://arxiv.org/abs/1301.3781,June 2016.

[20] Vincent P, Larochelle H, Bengio Y, Manzagol P A. Extracting and composing robust features with denoising autoencoders. In Proc. the 25th International Conference on Machine Learning, June 2008, pp.1096-1103.

[21] Socher R, Pennington J, Huang E H, Ng A Y, Manning C D. Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proc. the Conference on Empirical Methods in Natural Language Processing, July 2011, pp.151-161.

[22] Silberer C, Lapata M. Learning grounded meaning representations with autoencoders. InProc. the 52nd Annual Meeting of the Association for Computational Linguistics, June 2014, pp.721-732.

[23] Zhang H, Yu H, Xiong D, Liu Q. HHmm-based Chinese lexical analyzer ICTCLAS.In Proc. the 2nd SIGHAN Workshop on Chinese Language Processing, July 2003, pp.184-187.

[24] Hu B, Lu Z, Li H, Chen Q. Convolutional neural network architectures for matching natural language sentences. In Proc. Advances in Neural Information Processing Systems, December 2015, pp.2042-2050.
No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 李明慧;. CAD System of Microprogrammed Digital Systems[J]. , 1987, 2(3): 226 -235 .
[2] 周启海;. An Improved Graphic Representation for Structured Program Design[J]. , 1991, 6(2): 205 -208 .
[3] 王晖; 刘大有; 王亚飞;. Sequential Back-Propagation[J]. , 1994, 9(3): 252 -260 .
[4] 曾建超; HidehikoSanada; YoshikazuTezuka;. A Form Evaluation System and Its Data Structure for Brush-Written Chinese Characters[J]. , 1995, 10(1): 35 -41 .
[5] 王仕军; 王树林;. Research and Design of a Fuzzy Neural Expert System[J]. , 1995, 10(2): 112 -123 .
[6] 赵著行; 闵应骅; 李忠诚;. Path Sensitization[J]. , 1997, 12(3): 271 -282 .
[7] 郝瑞兵; 吴建平;. A Formal Approach to Protocol Interoperability Testing[J]. , 1998, 13(1): 79 -90 .
[8] 肖利民; 祝明发;. Exploiting the Capabilities of the Interconnection Network on Dawning-1000[J]. , 1999, 14(1): 49 -55 .
[9] 魏华; 罗予频; 杨士元;. Fault Tolerance of Reconfigurable Bi-Directional Double-Loop LANs[J]. , 1999, 14(4): 379 -385 .
[10] 张树功; 刘颖; 冯果忱;. The Multiplicity of Zeros of Algebraic System in Eigenvalue Method[J]. , 1999, 14(5): 510 -517 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持:support@magtech.com.cn
总访问量: