计算机科学技术学报 ›› 2022,Vol. 37 ›› Issue (4): 814-838.doi: 10.1007/s11390-022-2031-y

所属专题: Artificial Intelligence and Pattern Recognition

• • 上一篇    下一篇

基于文档级与查询级段落累积收益的文档排序方法

  

  • 收稿日期:2021-11-19 修回日期:2022-06-18 接受日期:2022-06-29 出版日期:2022-07-25 发布日期:2022-07-25

Leveraging Document-Level and Query-Level Passage Cumulative Gain for Document Ranking

Zhi-Jing Wu1,2 (吴之璟), Yi-Qun Liu11,2,* (刘奕群), Distinguished Member, ACM, CCF, Senior Member, IEEE, Jia-Xin Mao3 (毛佳昕), Member, ACM, CCF, Min Zhang1,2 (张敏), Senior Member, ACM, CCF, and Shao-Ping Ma1,2 (马少平)        

  1. 1Department of Computer Science and Technology, Tsinghua University, Beijing 100084, China
    2Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing 100084, China
    3Gaoling School of Artificial Intelligence, Renmin University of China, Beijing 100084, China
  • Received:2021-11-19 Revised:2022-06-18 Accepted:2022-06-29 Online:2022-07-25 Published:2022-07-25
  • Contact: Yi-Qun Liu E-mail:yiqunliu@tsinghua.edu.cn
  • About author:Yi-Qun Liu received his B.S. and Ph.D. degrees in computer science and technology from Tsinghua University, Beijing, in 2003 and 2007, respectively. He is now a professor in the Department of Computer Science and Technology at Tsinghua University, Beijing. His research interests include Web search, user behavior analysis, and natural language processing.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China under Grant No. 61732008 and Tsinghua University Guoqiang Research Institute.

      文档排序是信息检索领域相关研究中最重要也最具挑战性的问题之一。给定一个搜索查询和候选文档集合,文档排序的目标为根据文档与查询的相关性分数生成文档排序列表。近年来,随着互联网文档长度的增长和人类获取直接答案片段的需求的增加,一些学者开始尝试建模细粒度(例如句子级,段落级等)的文档相关性,并基于此提升文档排序任务的表现。然而,大部分相关工作通常仅建模独立的细粒度相关性信号,造成上下文信息缺失,限制了排序模型的表现。
      在本文中,我们尝试通过建模上下文感知的细粒度相关性,将其应用在文档排序相关任务上,提升排序任务的表现。我们首先研究了用户在从上至下阅读一篇文档的过程中,其信息收益如何逐段累积,并基于此提出了上下文感知的段落累积收益(PCG,Passage Cumulative Gain)。一篇文档上的PCG序列模拟了用户在每读完一段内容后,其感知到的收益水平逐段变化的过程。因此,PCG避免了独立的段落相关性标注引起的上下文信息缺失。
      我们分别在文档层面(DPCG,Document-level PCG)和查询层面(QPCG,Query-level PCG)研究了PCG序列的规律。我们发现PCG序列为非减序列,即收益水平逐段累积不会减少。一个查询会话中靠前位置的文档收益水平会影响后续文档上收益序列的变化规律。基于PCG的序列规律,我们提出了一个基于BERT的序列模型,段落累积收益模型(PCGM,Passage Cumulative Gain Model)。该模型能够有效地预测PCG序列以及应用到多个文档排序任务上,即单文档收益预测与边际相关性预测。在多个数据集上的实验结果表明,PCGM能够在文档排序指标上取得优于现有文档排序模型的效果,并能够在边际相关性估计任务上取得与用户偏好更一致的结果。

关键词: 文档排序, 神经网络, 段落累积收益

Abstract: Document ranking is one of the most studied but challenging problems in information retrieval (IR). More and more studies have begun to address this problem from fine-grained document modeling. However, most of them focus on context-independent passage-level relevance signals and ignore the context information. In this paper, we investigate how information gain accumulates with passages and propose the context-aware Passage Cumulative Gain (PCG). The fine-grained PCG avoids the need to split documents into independent passages. We investigate PCG patterns at the document level (DPCG) and the query level (QPCG). Based on the patterns, we propose a BERT-based sequential model called Passage-level Cumulative Gain Model (PCGM) and show that PCGM can effectively predict PCG sequences. Finally, we apply PCGM to the document ranking task using two approaches. The first one is leveraging DPCG sequences to estimate the gain of an individual document. Experimental results on two public ad hoc retrieval datasets show that PCGM outperforms most existing ranking models. The second one considers the cross-document effects and leverages QPCG sequences to estimate the marginal relevance. Experimental results show that predicted results are highly consistent with users' preferences. We believe that this work contributes to improving ranking performance and providing more explainability for document ranking.

Key words: document ranking, neural network, passage cumulative gain

<p> <span>[1]</span> Robertson S E, Walker S. Some simple effective approximations to the 2-poisson model for probabilistic weighted retrieval. In <span><em>Proc. the 17th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval</em></span>, Jul. 1994, pp.232-241. DOI: <span>10.1007/978-1-4471-2099-5</span>. </p> <p> <span>[2]</span> Ponte J M. A language modeling approach to information retrieval [Ph.D. Thesis]. University of Massachusetts, 1998. </p> <p> <span>[3]</span> Zhai C, Lafferty J. A study of smoothing methods for language models applied to ad hoc information retrieval. <span><em>ACM SIGIR Forum</em></span>, 2017, 51(2): 268-276. DOI: <span>10.1145/3130348.3130377</span>. </p> <p> <span>[4]</span> Burges C J. From RankNet to LambdaRank to LambdaMART: An overview. Technical Report, MSR-TR-2010-82, Microsoft, 2010. https://www.microsoft.com/en-us/research/wp-content/uploads/2/02/MSR-TR-2010-82.pdf, Apr. 2022. </p> <p> <span>[5]</span> Liu T. Learning to Rank for Information Retrieval. Springer, 2011. DOI: <span>10.1007/978-3-642-14267-3</span>. </p> <p> <span>[6]</span> Pang L, Lan Y, Guo J, Xu J, Cheng X. A deep investigation of deep IR models. arXiv:1707.07700, 2017. https://arxiv.org/abs/1707.07700, May 2022. </p> <p> <span>[7]</span> Clarke C L, Scholer F, Soboroff I. The TREC 2005 terabyte track. In <span><em>Proc. the 14th Text Retrieval Conference</em></span>, Nov. 2005. </p> <p> <span>[8]</span> Callan J P. Passage-level evidence in document retrieval. In <span><em>Proc. the 17th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval</em></span>, July 1994, pp.302-310. DOI: <span>10.1007/978-1-4471-2099-5</span>. </p> <p> <span>[9]</span> Kaszkiel M, Zobel J. Effective ranking with arbitrary passages. <span><em>Journal of the American Society for Information Science and Technology</em></span>, 2001, 52(4): 344-364. DOI: <span>10.1002/1532-2890(2000)9999:9999<::AID-ASI1075>3.0.CO;2-23</span>. </p> <p> <span>[10]</span> Xi W, Xu R R, Khoo C S, Lim E P. Incorporating window-based passage-level evidence in document retrieval. <span><em>Journal of Information Science</em></span>, 2001, 27(2): 73-80. DOI: <span>10.1177/016555150102700202</span>. </p> <p> <span>[11]</span> Dai Z, Callan J. Deeper text understanding for IR with contextual neural language modeling. In <span><em>Proc. the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval</em></span>, Jul. 2019, pp.985-988. DOI: <span>10.1145/3331184.3331303</span>. </p> <p> <span>[12]</span> Wu Z, Mao J, Liu Y, Zhang M, Ma S. Investigating passage-level relevance and its role in document-level relevance judgment. In <span><em>Proc. the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval</em></span>, Jul. 2019, pp.605-614. DOI: <span>10.1145/3331184.3331233</span>. </p> <p> <span>[13]</span> Fan Y, Guo J, Lan Y, Xu J, Zhai C, Cheng X. Modeling diverse relevance patterns in ad-hoc retrieval. In <span><em>Proc. the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval</em></span>, Jul. 2018, pp.375-384. DOI: 10.1145/3209978.3209980. </p> <p> <span>[14]</span> Pang L, Lan Y, Guo J, Xu J, Xu J, Cheng X. DeepRank: A new deep architecture for relevance ranking in information retrieval. In <span><em>Proc. the 2017 ACM Conference on Information and Knowledge Management</em></span>, Nov. 2017, pp.257-266. DOI: <span>10.1145/3132847.3132914</span>. </p> <p> <span>[15]</span> Li X, Liu Y, Mao J, He Z, Zhang M, Ma S. Understanding reading attention distribution during relevance judgement. In <span><em>Proc. the 27th ACM International Conference on Information and Knowledge Management</em></span>, Oct. 2018, pp.733-742. DOI: <span>10.1145/3269206.3271764</span>. </p> <p> <span>[16]</span> Järvelin K, Kekäläinen J. IR evaluation methods for retrieving highly relevant documents. In <span><em>Proc. the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval</em></span>, Jul. 2000, pp.41-48. DOI: <span>10.1145/345508.345545</span>. </p> <p> <span>[17]</span> Järvelin K, Kekäläinen J. Cumulated gain-based evaluation of IR techniques. <span><em>ACM Transactions on Information Systems</em></span>, 2002, 20(4): 422-446. DOI: <span>10.1145/582415.582418</span>. </p> <p> <span>[18]</span> Järvelin K, Price S L, Delcambre L M, Nielsen M L. Discounted cumulated gain based evaluation of multiple-query IR sessions. In <span><em>Proc. the 30th European Conference on Information Retrieval Research</em></span>, March 30-April 3, 2008, pp.4-15. DOI: <span>10.1007/978-3-540-78646-7</span>. </p> <p> <span>[19]</span> Carbonell J, Goldstein J. The use of MMR, diversity-based reranking for reordering documents and producing summaries. In <span><em>Proc. the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval</em></span>, Aug. 1998, pp.335-336. DOI: <span>10.1145/290941.291025</span>. </p> <p> <span>[20]</span> Liu M, Liu Y, Mao J, Luo C, Zhang M, Ma S. “Satisfaction with failure” or “unsatisfied success”: Investigating the relationship between search success and user satisfaction. In <span><em>Proc. the 2018 World Wide Web Conference</em></span>, Apr. 2018, pp.1533-1542. DOI: <span>10.1145/3178876.3186065</span>. </p> <p> <span>[21]</span> Devlin J, Chang M, Lee K, Toutanova K. BERT: Pre-training of deep bidirectional transformers for language understanding. In <span><em>Proc. the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies</em></span>, Jun. 2019, pp.4171-4186. DOI: <span>10.18653/v1/N19-1423</span>. </p> <p> <span>[22]</span> Hochreiter S, Schmidhuber J. Long short-term memory. <span><em>Neural Computation</em></span>, 1997, 9(8): 1735-1780. DOI: <span>10.1162/neco.1997.9.8.1735</span>. </p> <p> <span>[23]</span> Wu Z, Mao J, Liu Y, Zhan J, Zheng Y, Zhang M, Ma S. Leveraging passage-level cumulative gain for document ranking. In <span><em>Proc. the Web Conference 2020</em></span>, Apr. 2020, pp.2421-2431. DOI: <span>10.1145/3366423.3380305</span>. </p> <p> <span>[24]</span> Liu X, Croft W B. Passage retrieval based on language models. In <span><em>Proc. the 2002 ACM CIKM International Conference on Information and Knowledge Management</em></span>, Nov. 2002, pp.375-382. DOI: <span>10.1145/584792.584854</span>. </p> <p> <span>[25]</span> Wu Z, Mao J, Liu Y, Zhang M, Ma S. Investigating reading behavior in fine-grained relevance judgment. In <span><em>Proc. the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval</em></span>, Jul. 2020, pp.1889-1892. DOI: <span>10.1145/3397271.3401305</span>. </p> <p> <span>[26]</span> Hearst M A, Plaunt C. Subtopic structuring for full-length document access. In <span><em>Proc. the 16th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval</em></span>, June 27-July 1, 1993, pp.59-68. DOI: <span>10.1145/160688.160695</span>. </p> <p> <span>[27]</span> Salton G, Allan J, Buckley C. Approaches to passage retrieval in full text information systems. In <span><em>Proc. the 16th Annual International ACM-SIGIR Conference on Research and Development in Information Retrieval</em></span>, June 27-July 1, 1993, pp.49-58. DOI: <span>10.1145/160688.160693</span>. </p> <p> <span>[28]</span> Hui K, Yates A, Berberich K, De Melo G. PACCR: A position-aware neural IR model for relevance matching. In <span><em>Proc. the 2017 Conference on Empirical Methods in Natural Language Processing</em></span>, Sept. 2017, pp.1049-1058. DOI: <span>10.18653/v1/D17-1110</span>. </p> <p> <span>[29]</span> Hu B, Lu Z, Li H, Chen Q. Convolutional neural network architectures for matching natural language sentences. In <span><em>Proc. the 27th International Conference on Neural Information Processing Systems</em></span>, Dec. 2014, pp.2042-2050. </p> <p> <span>[30]</span> Guo J, Fan Y, Ai Q, Croft W B. A deep relevance matching model for ad-hoc retrieval. In <span><em>Proc. the 25th ACM International Conference on Information and Knowledge Management</em></span>, Oct. 2016, pp.55-64. DOI: <span>10.1145/2983323.2983769</span>. </p> <p> <span>[31]</span> Pang L, Lan Y, Guo J, Xu J, Wan S, Cheng X. Text matching as image recognition. In <span><em>Proc. the 30th AAAI Conference on Artificial Intelligence</em></span>, Feb. 2016, pp.2793-2799. </p> <p> <span>[32]</span> Xiong C, Dai Z, Callan J, Liu Z, Power R. End-to-end neural ad-hoc ranking with kernel pooling. In <span><em>Proc. the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval</em></span>, Aug. 2017, pp.55-64. DOI: <span>10.1145/3077136.3080809</span>. </p> <p> <span>[33]</span> Li X, Mao J, Wang C, Liu Y, Zhang M, Ma S. Teach machine how to read: Reading behavior inspired relevance estimation. In <span><em>Proc. the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval</em></span>, Jul. 2019, pp.795-804. DOI: <span>10.1145/3331184.3331205</span>. </p> <p> <span>[34]</span> Robertson S E. The probability ranking principle in IR. In <span><em>Readings in Information Retrieval</em></span>, Jones K S, Willett P (eds.), Morgan Kaufmann Publishers Inc., 1997, pp.281-286. </p> <p> <span>[35]</span> Goffman W. A searching procedure for information retrieval. <span><em>Information Storage and Retrieval</em></span>, 1964, 2: 73-78. DOI: <span>10.1016/0020-0271(64)90006-3</span>. </p> <p> <span>[36]</span> Fuhr N. A probability ranking principle for interactive information retrieval. <span><em>Information Retrieval</em></span>, 2008, 11(3): 251-265. DOI: <span>10.1007/s10791-008-9045-0</span>. </p> <p> <span>[37]</span> Zuccon G, Azzopardi L A, Van Rijsbergen K. The quantum probability ranking principle for information retrieval. In <span><em>Proc. the 2nd Conference on the Theory of Information Retrieval</em></span>, Sept. 2009, pp.232-240. DOI: <span>10.1007/978-3-642-04417-5</span>. </p> <p> <span>[38]</span> Chen H, Karger D R. Less is more: Probabilistic models for retrieving fewer relevant documents. In <span><em>Proc. the 29th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval</em></span>, Aug. 2006, pp.429-436. DOI: <span>10.1145/1148170.1148245</span>. </p> <p> <span>[39]</span> Hayes A F, Krippendorff K. Answering the call for a standard reliability measure for coding data. <span><em>Communication Methods and Measures</em></span>, 2007, 1(1): 77-89. DOI: <span>10.1080/19312450709336664</span>. </p> <p> <span>[40]</span> Roitero K, Maddalena E, Demartini G, Mizzaro S. On fine-grained relevance scales. In <span><em>Proc. the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval</em></span>, Jul. 2018, pp.675-684. DOI: <span>10.1145/3209978.3210052</span>. </p> <p> <span>[41]</span> Sarkar P, Pillai J S. User expectations of augmented reality experience in Indian school education. In <span><em>Proc. the 7th International Conference on Research into Design</em></span>, Jan. 2019, pp.745-755. DOI: <span>10.1007/978-981-13-5977-4</span>. </p> <p> <span>[42]</span> Kingma D P, Ba J. Adam: A method for stochastic optimization. arXiv:1412.6980, 2014. https://arxiv.org/abs/1, May 2022. </p> <p> <span>[43]</span> Sakai T, Song R. Evaluating diversified search results using per-intent graded relevance. In <span><em>Proc. the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval</em></span>, Jul. 2011, pp.1043-1052. DOI: <span>10.1145/2009916.2010055</span>. </p> <p> <span>[44]</span> Luo C, Zheng Y, Liu Y, Wang X, Xu J, Zhang M, Ma S. SogouT-16: A new web corpus to embrace IR research. In <span><em>Proc. the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval</em></span>, Aug. 2017, pp.1233-1236. DOI: <span>10.1145/3077136.3080694</span>. </p> <p> <span>[45]</span> Guo J, Fan Y, Ji X, Cheng X. MatchZoo: A learning, practicing, and developing system for neural text matching. In <span><em>Proc. the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval</em></span>, Jul. 2019, pp.1297-1300. DOI: <span>10.1145/3331184.3331403</span>. </p> <p> <span>[46]</span> Zheng Y, Fan Z, Liu Y, Luo C, Zhang M, Ma S. Sogou-QCL: A new dataset with click relevance label. In <span><em>Proc. the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval</em></span>, Jul. 2018, pp.1117-1120. DOI: <span>10.1145/3209978.3210092</span>. </p> <p> <span>[47]</span> Wang C, Liu Y, Wang M, Zhou K, Nie J, Ma S. Incorporating non-sequential behavior into click models. In <span><em>Proc. the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval</em></span>, Aug. 2015, pp.283-292. DOI: <span>10.1145/2766462.2767712</span>. </p> <p> <span>[48]</span> Dupret G E, Piwowarski B. A user browsing model to predict search engine click data from past observations. In <span><em>Proc. the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval</em></span>, Jul. 2008, pp.331-338. DOI: <span>10.1145/1390334.1390392</span>. </p> <p> <span>[49]</span> Zheng Y, Chu Z, Li X, Mao J, Liu Y, Zhang M, Ma S. THUIR at the NTCIR-14 WWW-2 task. In <span><em>Proc. the 14th International Conference on NII Testbeds and Community for Information Access Research</em></span>, Jun. 2019, pp.165-179. DOI: <span>10.1007/978-3-030-36805-0</span>. </p> <p> <span>[50]</span> Zeiler M D. ADADELTA: An adaptive learning rate method. arXiv:1212.5701, 2012. https://arxiv.org/abs/1, May 2022. </p>
[1] 魏华鹏, 邓盈盈, 唐帆, 潘兴甲, 董未名. 基于卷积神经网络和Transformer的视觉风格迁移的比较研究[J]. 计算机科学技术学报, 2022, 37(3): 601-614.
[2] 陈铮、方晓楠、张松海. 少纹理区域的局部单应性矩阵估计[J]. 计算机科学技术学报, 2022, 37(3): 615-625.
[3] 解晓政, 牛建伟, 刘雪峰, 李青锋, 王勇, 韩洁, 唐少杰. 基于卷积神经网络并融合边界信息的乳腺癌超声图像诊断[J]. 计算机科学技术学报, 2022, 37(2): 277-294.
[4] 王新峰、周翔、饶家华、张柱金、杨跃东. 基于迁移学习的DNA甲基化缺失数据补齐[J]. 计算机科学技术学报, 2022, 37(2): 320-329.
[5] 张鑫, 陆思源, 王水花, 余翔, 王甦菁, 姚仑, 潘毅, 张煜东. 通过新型深度学习架构诊断COVID-19肺炎[J]. 计算机科学技术学报, 2022, 37(2): 330-343.
[6] Dan-Hao Zhu, Xin-Yu Dai, Jia-Jun Chen. 预训练和学习:在图神经网络中保留全局信息[J]. 计算机科学技术学报, 2021, 36(6): 1420-1430.
[7] Yi Zhong, Jian-Hua Feng, Xiao-Xin Cui, Xiao-Le Cui. 机器学习辅助的抗逻辑块加密密钥猜测攻击范式[J]. 计算机科学技术学报, 2021, 36(5): 1102-1117.
[8] Feng Wang, Guo-Jie Luo, Guang-Yu Sun, Yu-Hao Wang, Di-Min Niu, Hong-Zhong Zheng. 在忆阻器中基于模式表示法的二值神经网络权重映射法[J]. 计算机科学技术学报, 2021, 36(5): 1155-1166.
[9] Shao-Jie Qiao, Guo-Ping Yang, Nan Han, Hao Chen, Fa-Liang Huang, Kun Yue, Yu-Gen Yi, Chang-An Yuan. 基数估计器:利用垂直扫描卷积神经网络处理SQL[J]. 计算机科学技术学报, 2021, 36(4): 762-777.
[10] Chen-Chen Sun, De-Rong Shen. 面向深度实体匹配的混合层次网络[J]. 计算机科学技术学报, 2021, 36(4): 822-838.
[11] Yang Liu, Ruili He, Xiaoqian Lv, Wei Wang, Xin Sun, Shengping Zhang. 婴儿的年龄和性别容易被识别吗?[J]. 计算机科学技术学报, 2021, 36(3): 508-519.
[12] Zhang-Jin Huang, Xiang-Xiang He, Fang-Jun Wang, Qing Shen. 基于卷积神经网络的实时多阶段斑马鱼头部姿态估计框架[J]. 计算机科学技术学报, 2021, 36(2): 434-444.
[13] Bo-Wei Zou, Rong-Tao Huang, Zeng-Zhuang Xu, Yu Hong, Guo-Dong Zhou. 基于对抗神经网络的跨语言实体关系分类[J]. 计算机科学技术学报, 2021, 36(1): 207-220.
[14] Wan-Wei Liu, Fu Song, Tang-Hao-Ran Zhang, Ji Wang. 基于模型检验的ReLU神经网络验证[J]. 计算机科学技术学报, 2020, 35(6): 1365-1381.
[15] Bi-Ying Yan, Chao Yang, Pan Deng, Qiao Sun, Feng Chen, Yang Yu. 一种基于时空因果性的城市感知数据治理方法[J]. 计算机科学技术学报, 2020, 35(5): 1084-1098.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 李未;. A Structural Operational Semantics for an Edison Like Language(2)[J]. , 1986, 1(2): 42 -53 .
[2] 潘启敬;. A Routing Algorithm with Candidate Shortest Path[J]. , 1986, 1(3): 33 -52 .
[3] 黄河燕;. A Parallel Implementation Model of HPARLOG[J]. , 1986, 1(4): 27 -38 .
[4] 郑国梁; 李辉;. The Design and Implementation of the Syntax-Directed Editor Generator(SEG)[J]. , 1986, 1(4): 39 -48 .
[5] 闵应骅; 韩智德;. A Built-in Test Pattern Generator[J]. , 1986, 1(4): 62 -74 .
[6] 吴允曾;. On the Development of Applications of Logic in Programming[J]. , 1987, 2(1): 30 -34 .
[7] 张钹; 张铃;. Statistical Heuristic Search[J]. , 1987, 2(1): 1 -11 .
[8] 乔香珍;. An Efficient Parallel Algorithm for FFT[J]. , 1987, 2(3): 174 -190 .
[9] 黄国祥; 刘健;. A Key-Lock Access Control[J]. , 1987, 2(3): 236 -243 .
[10] 谢立; 陈珮珮; 杨培根; 孙钟秀;. The Design and Implementation of an OA System ZGL1[J]. , 1988, 3(1): 75 -80 .
版权所有 © 《计算机科学技术学报》编辑部
本系统由北京玛格泰克科技发展有限公司设计开发 技术支持:support@magtech.com.cn
总访问量: